From patchwork Fri Dec 6 01:52:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896243 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE18C13C677 for ; Fri, 6 Dec 2024 01:53:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450009; cv=none; b=mzrveMjcSOh1T8HOVkmw1qg14aYx7W/j1SCauKZZaX7LIzGoApeaJJATunvAZzg2RcLm1gv7whXqt7mlnOsvXwUoiholewGBcAQ8oClkPszwV6DFb3a2ZmF1k0psnHjsYenhNPvs5liGQHiwVMpBdBDGwWFawGeumMru+Em+q50= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450009; c=relaxed/simple; bh=eOuO4i4yPLsc2Im8KihmSljRQ8rTc0TOPgnR2AUqOdU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nNQkHkrhb6scDe8M8UtODEUPTBnCh0cgDk91d67BugfDNi39Dvl/msoPp9dZ1XAOCA9QIKDzc9c8hb31RASm0uZlyByCKqNHiojC3wE8bCX58rG8iVcwwyAPF5/4H11LxE6dDDLdJ+aVP5dOb7e1CFYCDLqV7GBhIGHmCWpBDDs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=Col3DdPn; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="Col3DdPn" Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603K9F013801 for ; Thu, 5 Dec 2024 17:53:27 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=qzN9V22hHNPCY40Lp/Ysqsn5kxkAoYivYUU6jILwvog=; b=Col3DdPn1sBb UqgHsnitvyXO3BGkHudQfzF4pfElsNHcdLrxtn6ywSy+EcMHbTXj+e4BbTMTDEye L3nM1cqqG7MNXiYbF8Ht0fa8/Vaxes9TWSFIhMCMQlb1gQ4vTYQg0x18e2kSmg+C RrhPKFIKooviP/kEWaOD5xBipM7b9LUy8xPb+d1fd9mOEH0wsThNWc2AU1/EJY7k x2Zylv5T3+IZNWf44G4RNHwLR72A4u+/ZgIxbnaJ4HFgV+5R39nGigGUhUC9fSC/ CWtp5h/Abe29e706d5qHBEb1L5cZp3mH+NwXYyclxYByigd9qCE/u2vY0nogEH9g LCCNqHXJzg== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bnx9rpvh-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:27 -0800 (PST) Received: from twshared11082.06.ash8.facebook.com (2620:10d:c085:208::7cb7) by mail.thefacebook.com (2620:10d:c08b:78::c78f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:23 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id CBF2215B21149; Thu, 5 Dec 2024 17:53:08 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 01/10] fs: add a write stream field to the kiocb Date: Thu, 5 Dec 2024 17:52:59 -0800 Message-ID: <20241206015308.3342386-2-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: 9fA7eeSnSuMMYgOJWlNIlKN7JuANohEn X-Proofpoint-GUID: 9fA7eeSnSuMMYgOJWlNIlKN7JuANohEn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Christoph Hellwig Prepare for io_uring passthrough of write streams. The write stream field in the kiocb structure fits into an existing 2-byte hole, so its size is not changed. Signed-off-by: Christoph Hellwig Signed-off-by: Keith Busch --- include/linux/fs.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 2cc3d45da7b01..26940c451f319 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -373,6 +373,7 @@ struct kiocb { void *private; int ki_flags; u16 ki_ioprio; /* See linux/ioprio.h */ + u8 ki_write_stream; union { /* * Only used for async buffered reads, where it denotes the From patchwork Fri Dec 6 01:53:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896250 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CD7D14D708 for ; Fri, 6 Dec 2024 01:53:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450014; cv=none; b=ZfFxAXPGE2ZGrBNTwRSyoElOV1MhV+uZOfmBO2ySp01wY9dhymM3kxNn0GWJUTTpr9QWu7kmP78CbHvZ765F7RaJHVAJiSqDmf+MJNrXVf1mXCXu8T9QzGqi/qRBgufH8r4DwiElApsQD+yIDKFUEn9tm09rHGv/6z/d4bdIz6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450014; c=relaxed/simple; bh=TShGIinhtFGgvq8AU4aYd/uSo6I+vUdgnqrheypY8Fc=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DxwwKqQL2bbVBnL8Qx3iAjQHRMvT5XHO7T73U53wwQ6XM+njjI3JafqrYn3PJqgFvrDjG+obN+D5vAoc4JzBpzWhuWwLZ6ISKiyxDMOusCJiN8sZjNXvoXKv8souGtlFrrVoQWQr3didPysOkSDNDebZ1p0+gaxl8DynnFWcvRw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=e/abHQ40; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="e/abHQ40" Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.18.1.2/8.18.1.2) with ESMTP id 4B605dJ4013872 for ; Thu, 5 Dec 2024 17:53:30 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=qdL6YTuP4C1ftheSaTqqa06tk40Qi3hfBfSgmatUaHM=; b=e/abHQ40oZkx 4PpLiZCdAbi9CnSWu/1sB1QRUfaHgxSM8Cqx4Zh+juhU3i0EUE3qoW6bz+q37dZm QI0XEDG1Em3/f9HGlDVMg9/hbjggIyP5BSZX4GXcF7qIAlutm0Sm/nSw3PxwWq3P 3IV4xDfGDObKgScfBgeg1TadSGYqOHcI2KsjyyGTZNMbn3fvl0HlZaUFKyKI3r1a VpFxBXCE9Q6y1mwUjznJ9WPlP8bR6mg6qImKxQMtgrYEczsQTH4NF4YYjzSajxQx pGQ1mSz0njwRtFjgt88xhQm+IgQC+94aPxyr1xvKIGqg779AADzDXlZ6ri2WP1Za mxgwGJ76eg== Received: from mail.thefacebook.com ([163.114.134.16]) by m0001303.ppops.net (PPS) with ESMTPS id 43bmrwh5m0-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:30 -0800 (PST) Received: from twshared11082.06.ash8.facebook.com (2620:10d:c085:108::150d) by mail.thefacebook.com (2620:10d:c08b:78::2ac9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:23 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id F086D15B2114D; Thu, 5 Dec 2024 17:53:08 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 02/10] io_uring: protection information enhancements Date: Thu, 5 Dec 2024 17:53:00 -0800 Message-ID: <20241206015308.3342386-3-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 1d9lJ6CiZflj39NIOy40-S5TcI16MEVP X-Proofpoint-ORIG-GUID: 1d9lJ6CiZflj39NIOy40-S5TcI16MEVP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Just fixing up some formatting, removing unused parameters, and paving the way to allow chaining additional arbitrary attributes. Signed-off-by: Keith Busch --- include/uapi/linux/io_uring.h | 14 ++++++++------ io_uring/rw.c | 10 +++++----- 2 files changed, 13 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 38f0d6b10eaf7..5fa38467d6070 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -115,14 +115,16 @@ struct io_uring_sqe { #define IORING_RW_ATTR_FLAG_PI (1U << 0) /* PI attribute information */ struct io_uring_attr_pi { - __u16 flags; - __u16 app_tag; - __u32 len; - __u64 addr; - __u64 seed; - __u64 rsvd; + __u16 flags; + __u16 app_tag; + __u32 len; + __u64 addr; + __u64 seed; + __u64 rsvd; }; +#define IORING_RW_ATTR_FLAGS_SUPPORTED (IORING_RW_ATTR_FLAG_PI) + /* * If sqe->file_index is set to this for opcodes that instantiate a new * direct descriptor (like openat/openat2/accept), then io_uring will allocate diff --git a/io_uring/rw.c b/io_uring/rw.c index 04e4467ab0ee8..a2987aefb2cec 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -272,14 +272,14 @@ static inline void io_meta_restore(struct io_async_rw *io, struct kiocb *kiocb) } static int io_prep_rw_pi(struct io_kiocb *req, struct io_rw *rw, int ddir, - u64 attr_ptr, u64 attr_type_mask) + u64 *attr_ptr) { struct io_uring_attr_pi pi_attr; struct io_async_rw *io; int ret; - if (copy_from_user(&pi_attr, u64_to_user_ptr(attr_ptr), - sizeof(pi_attr))) + if (copy_from_user(&pi_attr, u64_to_user_ptr(*attr_ptr), + sizeof(pi_attr))) return -EFAULT; if (pi_attr.rsvd) @@ -295,6 +295,7 @@ static int io_prep_rw_pi(struct io_kiocb *req, struct io_rw *rw, int ddir, return ret; rw->kiocb.ki_flags |= IOCB_HAS_METADATA; io_meta_save_state(io); + *attr_ptr += sizeof(pi_attr); return ret; } @@ -335,8 +336,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, if (attr_type_mask) { u64 attr_ptr; - /* only PI attribute is supported currently */ - if (attr_type_mask != IORING_RW_ATTR_FLAG_PI) + if (attr_type_mask & ~IORING_RW_ATTR_FLAGS_SUPPORTED) return -EINVAL; attr_ptr = READ_ONCE(sqe->attr_ptr); From patchwork Fri Dec 6 01:53:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896244 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0CB013C8FF for ; Fri, 6 Dec 2024 01:53:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450009; cv=none; b=RtlSffB6bD2O+KvIdyPSjlHaL6EDiM46Xnz05WISe9pZ0FysM4zSAcJlL3TQRC2uKeFU9HDs7nXY5qrHpEnRNEQ9vl6+qVOeMNgTjPEse3+6er3QMVf9WBAW80Yy7nlVYt+xvnGu5/dzOqWaf4sKTiJTUmGdJiv3LS3LqL+8NzY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450009; c=relaxed/simple; bh=vQCj16/aGlHlGXxd0RymI3pYUEiOQrHyrx/mFrcpbCE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=a1I9KIewznfN/1deS5NDpbhbThUHPAoy59l2D4Md258OfOuGPZBZOc13vTCtQKsa64hqwBZbFvc0yaJgB/+pSpi9fxXeZnubDrD4jk2cIc2cDToURXfONYXuqIubS3iqhL/Li+N+e8nQOgtlV7PXpELc1AiXIfWkZLk152BmZ+I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=J5obIwmy; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="J5obIwmy" Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603Fjf017203 for ; Thu, 5 Dec 2024 17:53:27 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=53paUo/NtLhsF7pN8C+cqET+3d6cf3Dy8gzuWV/ss+Y=; b=J5obIwmyGd33 rMG4bodX1ku1bH1vXzNy1csP/LmXIAZcXNQLdGS4OhmxrcYHBZmChjzQ15AG8yjq TjWYmDQl1HFQapS21YYGuAPkQOdU4fEMJXCV0nBl/9nkPTjXx9QfkSy/uVNis+kJ PXTbkhyrvOqtEq2J5YODrCf36FAWHtNu12jJgKTZ+T9RgixjzMF7Wl6pmADPLnKx KBNcAUCbbNb+b5n8zgdlNKKsqdiCBrRBUYQaP1evEJJaQW3+M9IjeUkmncPlQ0H5 Ge0nFqeFlrMDut2lycpYU7ZGEgcpiDTMGkCGoSLXOTZwYWlUZy6KnYYF64KtexxA pm82awKELg== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bk9uhy70-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:26 -0800 (PST) Received: from twshared32179.32.frc3.facebook.com (2620:10d:c0a8:fe::f072) by mail.thefacebook.com (2620:10d:c0a9:6f::237c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:22 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 0EB7F15B2114F; Thu, 5 Dec 2024 17:53:08 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 03/10] io_uring: add write stream attribute Date: Thu, 5 Dec 2024 17:53:01 -0800 Message-ID: <20241206015308.3342386-4-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: Woe1EAQFISaLzdyD_SfXJDVhMNHGgGqn X-Proofpoint-ORIG-GUID: Woe1EAQFISaLzdyD_SfXJDVhMNHGgGqn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Adds a new attribute type to specify a write stream per-IO. Signed-off-by: Keith Busch --- include/uapi/linux/io_uring.h | 9 ++++++++- io_uring/rw.c | 28 +++++++++++++++++++++++++++- 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 5fa38467d6070..263cd57aae72d 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -123,7 +123,14 @@ struct io_uring_attr_pi { __u64 rsvd; }; -#define IORING_RW_ATTR_FLAGS_SUPPORTED (IORING_RW_ATTR_FLAG_PI) +#define IORING_RW_ATTR_FLAG_WRITE_STREAM (1U << 1) +struct io_uring_write_stream { + __u16 write_stream; + __u8 rsvd[6]; +}; + +#define IORING_RW_ATTR_FLAGS_SUPPORTED (IORING_RW_ATTR_FLAG_PI | \ + IORING_RW_ATTR_FLAG_WRITE_STREAM) /* * If sqe->file_index is set to this for opcodes that instantiate a new diff --git a/io_uring/rw.c b/io_uring/rw.c index a2987aefb2cec..69b566e296f6d 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -299,6 +299,22 @@ static int io_prep_rw_pi(struct io_kiocb *req, struct io_rw *rw, int ddir, return ret; } +static int io_prep_rw_write_stream(struct io_rw *rw, u64 *attr_ptr) +{ + struct io_uring_write_stream write_stream; + + if (copy_from_user(&write_stream, u64_to_user_ptr(*attr_ptr), + sizeof(write_stream))) + return -EFAULT; + + if (!memchr_inv(write_stream.rsvd, 0, sizeof(write_stream.rsvd))) + return -EINVAL; + + rw->kiocb.ki_write_stream = write_stream.write_stream; + *attr_ptr += sizeof(write_stream); + return 0; +} + static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, int ddir, bool do_import) { @@ -340,7 +356,17 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, return -EINVAL; attr_ptr = READ_ONCE(sqe->attr_ptr); - ret = io_prep_rw_pi(req, rw, ddir, attr_ptr, attr_type_mask); + if (attr_type_mask & IORING_RW_ATTR_FLAG_PI) { + ret = io_prep_rw_pi(req, rw, ddir, &attr_ptr); + if (ret) + return ret; + } + + if (attr_type_mask & IORING_RW_ATTR_FLAG_WRITE_STREAM) { + ret = io_prep_rw_write_stream(rw, &attr_ptr); + if (ret) + return ret; + } } return ret; } From patchwork Fri Dec 6 01:53:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896247 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 221981474AF for ; Fri, 6 Dec 2024 01:53:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450011; cv=none; b=c5V6RmfgyL6NLZpHbkQ5b8W7AOwCrevCFDgZSq5gMeAWMNhrY5Fb0NBLP2kyyvF0e7W3wvt24BtIHc0mRhuEDfXJtBouUxJsds6CjGWBdm80vVFbQt/8P2d154Z00l5OpFvHO92yXgdJFKOOv2dqMtXHKlaKtvdAd75ZgWnck0E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450011; c=relaxed/simple; bh=5JWKIlok/LlWB3ujWNzJ8mifN5u1BcG97Q8pMGFe0PM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cmsj0fGz/INfREyyDyJB88tTgJIFEwwEKqQd6PQqmJwnxjD9vwZhIRTYJJj/YWwJLC3cKqsBYF7K7AdQM+52h+1wScwsER3GucjWU10atRNyj/QmfFVpBMzk/cvq/oKAzUHH6KKGaiUGz8LtI2v3U7TtZDprKObfTuEvKndBeYI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=is3SY7wB; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="is3SY7wB" Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603K9L013801 for ; Thu, 5 Dec 2024 17:53:29 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=IaesBJajvy2XJvCSejkRh5S+DvrgGK6zlymIWSzBoyc=; b=is3SY7wB49cy EB3OemMXg+FNcvfC0dgf5GMZTFf6pnI99KR76zYftHDeQ7bGB3NqAP4UY71yz0dG S9zrdssa03GK8kQ2590zIsilbxmzWL2sKBJAbbBvCASlQmyxmwsMdxjzDnuFKL7Z MuZMI9nvgtOQOkJ+B0GJ9Dp8Fg+Pr1btASr8L1inX3muUIiFpvImqNpo5a3mAeUC XMXE3U93+E5VkvhZ66GUFov4hYAJpIYQFmDshZ5em28MTf/udZR+Atkh84s6c7nS KjWdwOrRCdpgxpA0jH9Hu0NhVvKn3C4GKluIPk7A5bLEcekI/lZpnaiBYlM44uRx 3gZn68x22Q== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bnx9rpvh-13 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:29 -0800 (PST) Received: from twshared11082.06.ash8.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c08b:78::c78f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:23 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 1ADCA15B21151; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 04/10] block: add a bi_write_stream field Date: Thu, 5 Dec 2024 17:53:02 -0800 Message-ID: <20241206015308.3342386-5-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: sabPT88wDC81Vd-n72dQBerpxWQdi7uD X-Proofpoint-GUID: sabPT88wDC81Vd-n72dQBerpxWQdi7uD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Christoph Hellwig Add the ability to pass a write stream for placement control in the bio. Signed-off-by: Christoph Hellwig Signed-off-by: Keith Busch --- block/bio.c | 2 ++ block/blk-crypto-fallback.c | 1 + block/blk-merge.c | 4 ++++ block/bounce.c | 1 + include/linux/blk_types.h | 1 + 5 files changed, 9 insertions(+) diff --git a/block/bio.c b/block/bio.c index 699a78c85c756..2aa86edc7cd6f 100644 --- a/block/bio.c +++ b/block/bio.c @@ -251,6 +251,7 @@ void bio_init(struct bio *bio, struct block_device *bdev, struct bio_vec *table, bio->bi_flags = 0; bio->bi_ioprio = 0; bio->bi_write_hint = 0; + bio->bi_write_stream = 0; bio->bi_status = 0; bio->bi_iter.bi_sector = 0; bio->bi_iter.bi_size = 0; @@ -827,6 +828,7 @@ static int __bio_clone(struct bio *bio, struct bio *bio_src, gfp_t gfp) bio_set_flag(bio, BIO_CLONED); bio->bi_ioprio = bio_src->bi_ioprio; bio->bi_write_hint = bio_src->bi_write_hint; + bio->bi_write_stream = bio_src->bi_write_stream; bio->bi_iter = bio_src->bi_iter; if (bio->bi_bdev) { diff --git a/block/blk-crypto-fallback.c b/block/blk-crypto-fallback.c index 29a205482617c..66762243a886b 100644 --- a/block/blk-crypto-fallback.c +++ b/block/blk-crypto-fallback.c @@ -173,6 +173,7 @@ static struct bio *blk_crypto_fallback_clone_bio(struct bio *bio_src) bio_set_flag(bio, BIO_REMAPPED); bio->bi_ioprio = bio_src->bi_ioprio; bio->bi_write_hint = bio_src->bi_write_hint; + bio->bi_write_stream = bio_src->bi_write_stream; bio->bi_iter.bi_sector = bio_src->bi_iter.bi_sector; bio->bi_iter.bi_size = bio_src->bi_iter.bi_size; diff --git a/block/blk-merge.c b/block/blk-merge.c index e01383c6e534b..1e5327fb6c45b 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -866,6 +866,8 @@ static struct request *attempt_merge(struct request_queue *q, if (req->bio->bi_write_hint != next->bio->bi_write_hint) return NULL; + if (req->bio->bi_write_stream != next->bio->bi_write_stream) + return NULL; if (req->bio->bi_ioprio != next->bio->bi_ioprio) return NULL; if (!blk_atomic_write_mergeable_rqs(req, next)) @@ -987,6 +989,8 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio) return false; if (rq->bio->bi_write_hint != bio->bi_write_hint) return false; + if (rq->bio->bi_write_stream != bio->bi_write_stream) + return false; if (rq->bio->bi_ioprio != bio->bi_ioprio) return false; if (blk_atomic_write_mergeable_rq_bio(rq, bio) == false) diff --git a/block/bounce.c b/block/bounce.c index 0d898cd5ec497..fb8f60f114d7d 100644 --- a/block/bounce.c +++ b/block/bounce.c @@ -170,6 +170,7 @@ static struct bio *bounce_clone_bio(struct bio *bio_src) bio_set_flag(bio, BIO_REMAPPED); bio->bi_ioprio = bio_src->bi_ioprio; bio->bi_write_hint = bio_src->bi_write_hint; + bio->bi_write_stream = bio_src->bi_write_stream; bio->bi_iter.bi_sector = bio_src->bi_iter.bi_sector; bio->bi_iter.bi_size = bio_src->bi_iter.bi_size; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index dce7615c35e7e..4ca3449ce9c95 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -220,6 +220,7 @@ struct bio { unsigned short bi_flags; /* BIO_* below */ unsigned short bi_ioprio; enum rw_hint bi_write_hint; + u8 bi_write_stream; blk_status_t bi_status; atomic_t __bi_remaining; From patchwork Fri Dec 6 01:53:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896245 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CAFF14659C for ; Fri, 6 Dec 2024 01:53:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450011; cv=none; b=nRiLKuPlUQCqhO0+DLPL69uvdM06ma8il7GThk/BD1lMWGN54g4gRlT8GidXvvf2goPOw4uqdZxr+uMiulyAtMYvhWLgLFTxmkc+mz5p8J4ehaO2YDwS3mCsmpVSKX+wqQxlNOMhVr50drvtObjDT0E6I8ybGoTNq6iW4CD61gQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450011; c=relaxed/simple; bh=TCNBtOVAlz4CYyfyNN08IAP8Ch7Vv7aWrtZDtfmR0VA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Zighmtfk7lMM78007bVC9D+g8pzUCyCTlfwwSs8KwtaGF1XMHvp5lFjuTDFPiyc+9DfVWfKNZVIVjQ/sWvFG9ZaOKrTOR8KD6907c5+zAPD6AFA+ZvMxpqtYQ1iW3e6F1dhq0y52vqvwJwN/NnKXJhzEg5XmkrlCnLbTAU09LeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=JlMQby/y; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="JlMQby/y" Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603JgX014117 for ; Thu, 5 Dec 2024 17:53:28 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=29G8/mCPCEmN9MtdiNCjD+XztU6EWmg+eIHaaiD06/k=; b=JlMQby/y2Eku 0vOi6Y74b8SkU5qFOYpmibZ40iLDFQjQtwLD/kh5utUz2YnGIm6Dtd5oYNHXY9Wa YYsqdHSPlTM7iMhNHS7v8xQD7A2BvAf45tz1fXivx02kPwb53Z2NxXUlmcZ27pT8 Ie8xCc6MkMv4n39zNh8quAnjpRmfI7lObWGoeMiRgd5gqbt0uEZHpjh1c4a5aAoR PDqar0S+MLv0/r5+QBzfosbqH3QLm2LcJ+TktcZMmu9QB5UGeCHwf6T9O5k1zMvy klpjONh83Mfy3cbLuJx2OH/PbNWnopQ5+TGXope5tVb7sNQVDwxP2eQ4wLpx/8gL 1cKePzcb/A== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bmru153e-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:27 -0800 (PST) Received: from twshared32179.32.frc3.facebook.com (2620:10d:c0a8:1b::2d) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:22 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 20F4A15B21153; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 05/10] block: introduce max_write_streams queue limit Date: Thu, 5 Dec 2024 17:53:03 -0800 Message-ID: <20241206015308.3342386-6-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: ldeqWGZJ77P53hh2xiTisrvYi6ernjFj X-Proofpoint-ORIG-GUID: ldeqWGZJ77P53hh2xiTisrvYi6ernjFj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Drivers with hardware that support write streams need a way to export how many are available so applications can generically query this. Signed-off-by: Keith Busch [hch: renamed hints to streams, removed stacking] Signed-off-by: Christoph Hellwig --- Documentation/ABI/stable/sysfs-block | 7 +++++++ block/blk-sysfs.c | 3 +++ include/linux/blkdev.h | 9 +++++++++ 3 files changed, 19 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 0cceb2badc836..f67139b8b8eff 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -506,6 +506,13 @@ Description: [RO] Maximum size in bytes of a single element in a DMA scatter/gather list. +What: /sys/block//queue/max_write_streams +Date: November 2024 +Contact: linux-block@vger.kernel.org +Description: + [RO] Maximum number of write streams supported, 0 if not + supported. If supported, valid values are 1 through + max_write_streams, inclusive. What: /sys/block//queue/max_segments Date: March 2010 diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 4241aea84161c..c514c0cb5e93c 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -104,6 +104,7 @@ QUEUE_SYSFS_LIMIT_SHOW(max_segments) QUEUE_SYSFS_LIMIT_SHOW(max_discard_segments) QUEUE_SYSFS_LIMIT_SHOW(max_integrity_segments) QUEUE_SYSFS_LIMIT_SHOW(max_segment_size) +QUEUE_SYSFS_LIMIT_SHOW(max_write_streams) QUEUE_SYSFS_LIMIT_SHOW(logical_block_size) QUEUE_SYSFS_LIMIT_SHOW(physical_block_size) QUEUE_SYSFS_LIMIT_SHOW(chunk_sectors) @@ -446,6 +447,7 @@ QUEUE_RO_ENTRY(queue_max_hw_sectors, "max_hw_sectors_kb"); QUEUE_RO_ENTRY(queue_max_segments, "max_segments"); QUEUE_RO_ENTRY(queue_max_integrity_segments, "max_integrity_segments"); QUEUE_RO_ENTRY(queue_max_segment_size, "max_segment_size"); +QUEUE_RO_ENTRY(queue_max_write_streams, "max_write_streams"); QUEUE_RW_LOAD_MODULE_ENTRY(elv_iosched, "scheduler"); QUEUE_RO_ENTRY(queue_logical_block_size, "logical_block_size"); @@ -580,6 +582,7 @@ static struct attribute *queue_attrs[] = { &queue_max_discard_segments_entry.attr, &queue_max_integrity_segments_entry.attr, &queue_max_segment_size_entry.attr, + &queue_max_write_streams_entry.attr, &queue_hw_sector_size_entry.attr, &queue_logical_block_size_entry.attr, &queue_physical_block_size_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 08a727b408164..ce2c3ddda2411 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -399,6 +399,8 @@ struct queue_limits { unsigned short max_integrity_segments; unsigned short max_discard_segments; + unsigned short max_write_streams; + unsigned int max_open_zones; unsigned int max_active_zones; @@ -1240,6 +1242,13 @@ static inline unsigned int bdev_max_segments(struct block_device *bdev) return queue_max_segments(bdev_get_queue(bdev)); } +static inline unsigned short bdev_max_write_streams(struct block_device *bdev) +{ + if (bdev_is_partition(bdev)) + return 0; + return bdev_limits(bdev)->max_write_streams; +} + static inline unsigned queue_logical_block_size(const struct request_queue *q) { return q->limits.logical_block_size; From patchwork Fri Dec 6 01:53:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896246 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5F5313C677 for ; Fri, 6 Dec 2024 01:53:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450011; cv=none; b=n0Netxhs3vGKQ70+cWRsOH+Go9twZgtJbTW2aMNu3tMYhj4ErVtC/RAwuEw2mdG0x4LmrIycH+Mjmh8P+Yp52UKFJONIMu7pCRZI52kWDka6qanMdepLmpQH9nZrQmzpHdn/3PAnwdsHEIY19JQNMXI1cZ2e1G6WdsTZw1vkOyE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450011; c=relaxed/simple; bh=yHXFvDsEeK5TRGak0SpI6LYPt8+i/ACjWPHS2Ewq/9I=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VKXtcP8m4LpVlTyTAKyFCzL+cPp1JKjgq1hYpvKlgEj1u0YWh6ZZW3Qn7htdjiMh2eoJxZnz86hCSB4P81DQfqAcWY+aI09vDF+IF91RzeTI4PVAB1IV+5VuS1+IrHWtR3uQNlDzwrJcyDj4fMinAAj0R4sYT77IrlrlfriKWd0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=CAmUP0Ow; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="CAmUP0Ow" Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603IVL012398 for ; Thu, 5 Dec 2024 17:53:29 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=H1449KeqBHH2qLyX/UaGXiOFOGfVyFVyzOCCtWJaXls=; b=CAmUP0OwnaBe AZlMffA9J5WHd+5HmYW3a3yQ1M3kgnO5N7sxx7y9FfPPT2Jh5xEtYJ0KBZi+zSZU QDSfmTII3zudYRRZvi4o2iuDJHbd4o6iGy/YdHJh/kYN+d+VEpgNMKZrdiTwcxRK YmcsMvcH4mIF07xzggbtOS6W/tL1eqFhI2PWWpjVQL/win9zSaSmF85hs4l04LHY /0JWOCSvuOGCo+37omWz61VuSRpMio19dw0YVlywR+Q9tht44Vm7B/uMdnavV0Pa NBmyh4UIBtyp8eFBqL5e8up84IbPXiyLni/YmxhhUDltrZWa2U+IzHpxFvAhB7Ji +OlnwgVUzw== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bgr8udwa-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:28 -0800 (PST) Received: from twshared32179.32.frc3.facebook.com (2620:10d:c0a8:1b::30) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:22 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 312B315B21155; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 06/10] block: introduce a write_stream_granularity queue limit Date: Thu, 5 Dec 2024 17:53:04 -0800 Message-ID: <20241206015308.3342386-7-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: fBt9DoCXizgs7IVE0Q1zvoa4fgWq4QYc X-Proofpoint-ORIG-GUID: fBt9DoCXizgs7IVE0Q1zvoa4fgWq4QYc X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_02,2024-10-04_01,2024-09-30_01 From: Christoph Hellwig Export the granularity that write streams should be discarded with, as it is essential for making good use of them. Signed-off-by: Christoph Hellwig Signed-off-by: Keith Busch --- Documentation/ABI/stable/sysfs-block | 8 ++++++++ block/blk-sysfs.c | 3 +++ include/linux/blkdev.h | 7 +++++++ 3 files changed, 18 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index f67139b8b8eff..c454c68b68fe6 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -514,6 +514,14 @@ Description: supported. If supported, valid values are 1 through max_write_streams, inclusive. +What: /sys/block//queue/write_stream_granularity +Date: November 2024 +Contact: linux-block@vger.kernel.org +Description: + [RO] Granularity of a write stream in bytes. The granularity + of a write stream is the size that should be discarded or + overwritten together to avoid write amplification in the device. + What: /sys/block//queue/max_segments Date: March 2010 Contact: linux-block@vger.kernel.org diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index c514c0cb5e93c..525f4fa132cd3 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -105,6 +105,7 @@ QUEUE_SYSFS_LIMIT_SHOW(max_discard_segments) QUEUE_SYSFS_LIMIT_SHOW(max_integrity_segments) QUEUE_SYSFS_LIMIT_SHOW(max_segment_size) QUEUE_SYSFS_LIMIT_SHOW(max_write_streams) +QUEUE_SYSFS_LIMIT_SHOW(write_stream_granularity) QUEUE_SYSFS_LIMIT_SHOW(logical_block_size) QUEUE_SYSFS_LIMIT_SHOW(physical_block_size) QUEUE_SYSFS_LIMIT_SHOW(chunk_sectors) @@ -448,6 +449,7 @@ QUEUE_RO_ENTRY(queue_max_segments, "max_segments"); QUEUE_RO_ENTRY(queue_max_integrity_segments, "max_integrity_segments"); QUEUE_RO_ENTRY(queue_max_segment_size, "max_segment_size"); QUEUE_RO_ENTRY(queue_max_write_streams, "max_write_streams"); +QUEUE_RO_ENTRY(queue_write_stream_granularity, "write_stream_granularity"); QUEUE_RW_LOAD_MODULE_ENTRY(elv_iosched, "scheduler"); QUEUE_RO_ENTRY(queue_logical_block_size, "logical_block_size"); @@ -583,6 +585,7 @@ static struct attribute *queue_attrs[] = { &queue_max_integrity_segments_entry.attr, &queue_max_segment_size_entry.attr, &queue_max_write_streams_entry.attr, + &queue_write_stream_granularity_entry.attr, &queue_hw_sector_size_entry.attr, &queue_logical_block_size_entry.attr, &queue_physical_block_size_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index ce2c3ddda2411..7be8cc57561a1 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -400,6 +400,7 @@ struct queue_limits { unsigned short max_discard_segments; unsigned short max_write_streams; + unsigned int write_stream_granularity; unsigned int max_open_zones; unsigned int max_active_zones; @@ -1249,6 +1250,12 @@ static inline unsigned short bdev_max_write_streams(struct block_device *bdev) return bdev_limits(bdev)->max_write_streams; } +static inline unsigned int +bdev_write_stream_granularity(struct block_device *bdev) +{ + return bdev_limits(bdev)->write_stream_granularity; +} + static inline unsigned queue_logical_block_size(const struct request_queue *q) { return q->limits.logical_block_size; From patchwork Fri Dec 6 01:53:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896253 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03E5716132F for ; Fri, 6 Dec 2024 01:53:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450017; cv=none; b=r4BcZvEMfEoDmI81BHrvlZqp9uduT+vBdkvBtYfvwc5sAk/Ja9BTnjZIQhuTyp94hIZVrh9AcNbnhYnhmAOKKAWkkDGe1iwqLxJApgy/fvl3iQIpRkosdMnAcrXS6g6OhW3SxnETyw9RW54i+x0HZNUYBYzShn0e8/2i5kwFMmg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450017; c=relaxed/simple; bh=c+UVhkKbwuD8UQEwseys8Y1G4kpR1yRUBsykIZ6n1sk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=J7o4TCEEsw2a38NLhC1aQS3adQV0BgMjkPFdxlvsb1vBu6GGDokI5wTl45MP9RdkbRiOPgnbIAushRj6Gra88UhZ0hTP9CMdf2MqdCROu4RNTeIMec6P90sRLCYbfX0MzEHjYt3CGx4U9ggiF8kFuRe6QlXQR6HS3wzIaPCu1Ag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=hihcg3gY; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="hihcg3gY" Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.18.1.2/8.18.1.2) with ESMTP id 4B605dJC013872 for ; Thu, 5 Dec 2024 17:53:34 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=QQZf+H6qxvbae0if9/efVpaNOckqg81tEPEEkknyOUs=; b=hihcg3gYMgFz E94ESmiPgrPlD7LfdJt3VEa/EBxmNAGNUY0LPeQ6LBpBm97Krhjco5sFG9dNGJkl FYUNrugt0MjMxfwC8E1XNTnLfL55zptMIbhfGZ4si0sgvgEKqfHaQWePS31MsO6V Tn1wd4dqaflhknwLjT9V9XB9e56nxAeayjg1jxtfanLK24c6crgA3kpK7J5/lzPG jejQoIPB3fcm2hTiEuatTNUZ3ua4VjZYfQ/98QMqIEjKv9GG3zbp2CKXW1Q8U4eb rVjM4CG6LIteeAJ4NWIVn7wm5iHpJq6xDEw8y9P/yWrBspt95WX1aPzlUJ/osVce FFZopNNLpg== Received: from mail.thefacebook.com ([163.114.134.16]) by m0001303.ppops.net (PPS) with ESMTPS id 43bmrwh5m0-17 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:34 -0800 (PST) Received: from twshared11082.06.ash8.facebook.com (2620:10d:c085:108::150d) by mail.thefacebook.com (2620:10d:c08b:78::2ac9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:24 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 375D515B21158; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 07/10] block: expose write streams for block device nodes Date: Thu, 5 Dec 2024 17:53:05 -0800 Message-ID: <20241206015308.3342386-8-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: XcVDIGnBTrNpaZMU9_x5CneW7sQeId06 X-Proofpoint-ORIG-GUID: XcVDIGnBTrNpaZMU9_x5CneW7sQeId06 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Christoph Hellwig Export statx information about the number and granularity of write streams, use the per-kiocb write hint and map temperature hints to write streams (which is a bit questionable, but this shows how it is done). Signed-off-by: Christoph Hellwig Signed-off-by: Keith Busch --- block/bdev.c | 6 ++++++ block/fops.c | 23 +++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/block/bdev.c b/block/bdev.c index 738e3c8457e7f..c23245f1fdfe3 100644 --- a/block/bdev.c +++ b/block/bdev.c @@ -1296,6 +1296,12 @@ void bdev_statx(struct path *path, struct kstat *stat, stat->result_mask |= STATX_DIOALIGN; } + if ((request_mask & STATX_WRITE_STREAM) && + bdev_max_write_streams(bdev)) { + stat->write_stream_max = bdev_max_write_streams(bdev); + stat->result_mask |= STATX_WRITE_STREAM; + } + if (request_mask & STATX_WRITE_ATOMIC && bdev_can_atomic_write(bdev)) { struct request_queue *bd_queue = bdev->bd_queue; diff --git a/block/fops.c b/block/fops.c index 6d5c4fc5a2168..f16aa39bf5bad 100644 --- a/block/fops.c +++ b/block/fops.c @@ -73,6 +73,7 @@ static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb, } bio.bi_iter.bi_sector = pos >> SECTOR_SHIFT; bio.bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint; + bio.bi_write_stream = iocb->ki_write_stream; bio.bi_ioprio = iocb->ki_ioprio; if (iocb->ki_flags & IOCB_ATOMIC) bio.bi_opf |= REQ_ATOMIC; @@ -206,6 +207,7 @@ static ssize_t __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, for (;;) { bio->bi_iter.bi_sector = pos >> SECTOR_SHIFT; bio->bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint; + bio->bi_write_stream = iocb->ki_write_stream; bio->bi_private = dio; bio->bi_end_io = blkdev_bio_end_io; bio->bi_ioprio = iocb->ki_ioprio; @@ -333,6 +335,7 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb, dio->iocb = iocb; bio->bi_iter.bi_sector = pos >> SECTOR_SHIFT; bio->bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint; + bio->bi_write_stream = iocb->ki_write_stream; bio->bi_end_io = blkdev_bio_end_io_async; bio->bi_ioprio = iocb->ki_ioprio; @@ -398,6 +401,26 @@ static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter) if (blkdev_dio_invalid(bdev, iocb, iter)) return -EINVAL; + if (iov_iter_rw(iter) == WRITE) { + u16 max_write_streams = bdev_max_write_streams(bdev); + + if (iocb->ki_write_stream) { + if (iocb->ki_write_stream > max_write_streams) + return -EINVAL; + } else if (max_write_streams) { + enum rw_hint write_hint = + file_inode(iocb->ki_filp)->i_write_hint; + + /* + * Just use the write hint as write stream for block + * device writes. This assumes no file system is + * mounted that would use the streams differently. + */ + if (write_hint <= max_write_streams) + iocb->ki_write_stream = write_hint; + } + } + nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS + 1); if (likely(nr_pages <= BIO_MAX_VECS)) { if (is_sync_kiocb(iocb)) From patchwork Fri Dec 6 01:53:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896251 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A1F7154C12 for ; Fri, 6 Dec 2024 01:53:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450016; cv=none; b=hJhGDzMB+KhzfpCR2JX94jQY/bJIIh/nbjaVt9UNHKKdsiAcAo/ZbTAaADHGkFysw6m9rz80Cp/1nBhPa0d5q1Mk4CJ131nnZDXZTjOVJQvWcvFqaxL0ENV2YFihxdFcTTcw7PFOyULFap4AHWiPDzp4NsaKan2Qj04BugCzCfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450016; c=relaxed/simple; bh=a6bic9lWReIENxTHeZoG8ChH6lXRFB7Rkje1NanJ8/Q=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fXxudbw0kbzeBOBC8wuEFihRxoVbFDGmmFzvrWr2tOHlNbU2XQ2A93nvZFButYq8aw7M0IfyqlzHy1BRYTrjXCIwcgnwHoLPTpWRVGUJgvLQr7kYaM6Ha7UxwpB/RIzzoRms9Otd78leYfZ+o7ra/7qgAV+8DzeNFSy3qsWqw7s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=QpbdAmJk; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="QpbdAmJk" Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603MMu014363 for ; Thu, 5 Dec 2024 17:53:33 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=aqSWgHfWMagd/tVBxU1j+3PESQendeFXQhXEUWomWZI=; b=QpbdAmJkjzgF iYh3gGcrBDjNQGDCBSqOonvTJo9dHOHxO5skvx61cilBsjAGLYhg2i0kDvjAutvl TB0NzDbaLrSj2TftKl5ETJu05MWWmf2dFjLDCff1rI/r4o+1Xp9067sHobFlgQWg lqTVpswYJE4snjQ8YP5gn0GVukU95RqiZNroCilgxmIq4Vp1O4YVu54Jf/Vub8uV 5QhzQfUWj93gckRapfcZVqQ6+/PxLBeQhIkjYaR4vDpXKCzdDs7D201SX1s4Zzio JeNOSvGY8o3KkBYxu68SyLYzlY39I2LRJLa3XJDp3C95Z96RqU2sgSfJKJFJFTk8 4EB0Rqbe9w== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bmru153t-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:33 -0800 (PST) Received: from twshared11082.06.ash8.facebook.com (2620:10d:c085:108::150d) by mail.thefacebook.com (2620:10d:c08b:78::c78f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:24 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 4791315B2115A; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 08/10] nvme: add a nvme_get_log_lsi helper Date: Thu, 5 Dec 2024 17:53:06 -0800 Message-ID: <20241206015308.3342386-9-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: BaZd6h2OjSefDJpd1nWSwrMN8GDFKetf X-Proofpoint-ORIG-GUID: BaZd6h2OjSefDJpd1nWSwrMN8GDFKetf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Christoph Hellwig For log pages that need to pass in a LSI value, while at the same time not touching all the existing nvme_get_log callers. Signed-off-by: Christoph Hellwig Signed-off-by: Keith Busch --- drivers/nvme/host/core.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 571d4106d256d..36c44be98e38c 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -150,6 +150,8 @@ static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, unsigned nsid); static void nvme_update_keep_alive(struct nvme_ctrl *ctrl, struct nvme_command *cmd); +static int nvme_get_log_lsi(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page, + u8 lsp, u8 csi, void *log, size_t size, u64 offset, u16 lsi); void nvme_queue_scan(struct nvme_ctrl *ctrl) { @@ -3074,8 +3076,8 @@ static int nvme_init_subsystem(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id) return ret; } -int nvme_get_log(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page, u8 lsp, u8 csi, - void *log, size_t size, u64 offset) +static int nvme_get_log_lsi(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page, + u8 lsp, u8 csi, void *log, size_t size, u64 offset, u16 lsi) { struct nvme_command c = { }; u32 dwlen = nvme_bytes_to_numd(size); @@ -3089,10 +3091,18 @@ int nvme_get_log(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page, u8 lsp, u8 csi, c.get_log_page.lpol = cpu_to_le32(lower_32_bits(offset)); c.get_log_page.lpou = cpu_to_le32(upper_32_bits(offset)); c.get_log_page.csi = csi; + c.get_log_page.lsi = cpu_to_le16(lsi); return nvme_submit_sync_cmd(ctrl->admin_q, &c, log, size); } +int nvme_get_log(struct nvme_ctrl *ctrl, u32 nsid, u8 log_page, u8 lsp, u8 csi, + void *log, size_t size, u64 offset) +{ + return nvme_get_log_lsi(ctrl, nsid, log_page, lsp, csi, log, size, + offset, 0); +} + static int nvme_get_effects_log(struct nvme_ctrl *ctrl, u8 csi, struct nvme_effects_log **log) { From patchwork Fri Dec 6 01:53:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896249 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB32B15098A for ; Fri, 6 Dec 2024 01:53:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450013; cv=none; b=S5zhM35t1y4ljwl4IvwBiIME0HL4ezkmxMCoiM8DrXMGhZdN4pC9IzzMx0jBAQHEqmN5hHHReDHvXE/DERbsCCGSkkZM/Ldh/99w3YG4ennEyRLGN4xFXfdhMK9FBNEatGkrR3K3jCldbElU1PkEZtMbq7CSuYsVUVDXunpC7sw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450013; c=relaxed/simple; bh=jEkhKFWSiOGNqmafZm5rrOk42Ci3nP8BRG3OYmsRfC4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=S/sagpmmVat3O+37dNFaYeNy9mVjoAeLKtqouJkGztDwZ+WuGGemjjBqa2QRELiXxejWk/kJwvSMBafVxrtjABWhBymLpDRWMA19I05gZu509rSOq10yf4g/ip2BXkZqjSDndYY6zeePfqt+Mbnx7Y5zyyzHkq+Vo9834gLbLPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=FU4gvn30; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="FU4gvn30" Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603Jge014117 for ; Thu, 5 Dec 2024 17:53:31 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=/QfY4VuKFEqf3WK6nWx5RQXvAMdSdKErUcPPW9/w71A=; b=FU4gvn306t3k Nbl3PStj8eXWBYzoPkPCc+HoLsYN3OfBsqg/GGSvWy3sQFkyyJ0h85FLPKOMdbTx NzRoL5uTnLS07y38ZcmJgExeyC0Vx+0kv8AjyFenI/ZHUo5A6uM7SAT4n5SxUYxM yJfTW1W7H/Bask9xm5t/raBYT8Y39BMF6ctB4c6pZx+a6Eu+fmhU7mQoismzodJs NmF3zG4j0x0DbpeLtlWtik/RACZPrew+hpFH3CqaA/PCs2t83jypiGSiuALUAqBG FDaKV2B3bRcuAJKw5ilqMJ86BAAyJGA4gIVy6Q/sfqx6crb2wJxfXcYgPz1MuNUY hY0pXu3rdw== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bmru153e-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:30 -0800 (PST) Received: from twshared32179.32.frc3.facebook.com (2620:10d:c0a8:1b::2d) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:22 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 5C0A115B2115C; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch , Keith Busch Subject: [PATCHv11 09/10] nvme: register fdp queue limits Date: Thu, 5 Dec 2024 17:53:07 -0800 Message-ID: <20241206015308.3342386-10-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: uFNW_Wyo5Bv1P7fgoCC_LYPUEIC7p1E6 X-Proofpoint-ORIG-GUID: uFNW_Wyo5Bv1P7fgoCC_LYPUEIC7p1E6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Register the device data placement limits if supported. This is just registering the limits with the block layer. Nothing beyond reporting these attributes is happening in this patch. Signed-off-by: Keith Busch --- drivers/nvme/host/core.c | 116 +++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/nvme.h | 4 ++ include/linux/nvme.h | 73 ++++++++++++++++++++++++ 3 files changed, 193 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 36c44be98e38c..410a77de92f88 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -38,6 +38,8 @@ struct nvme_ns_info { u32 nsid; __le32 anagrpid; u8 pi_offset; + u16 endgid; + u64 runs; bool is_shared; bool is_readonly; bool is_ready; @@ -1613,6 +1615,7 @@ static int nvme_ns_info_from_identify(struct nvme_ctrl *ctrl, info->is_shared = id->nmic & NVME_NS_NMIC_SHARED; info->is_readonly = id->nsattr & NVME_NS_ATTR_RO; info->is_ready = true; + info->endgid = le16_to_cpu(id->endgid); if (ctrl->quirks & NVME_QUIRK_BOGUS_NID) { dev_info(ctrl->device, "Ignoring bogus Namespace Identifiers\n"); @@ -1653,6 +1656,7 @@ static int nvme_ns_info_from_id_cs_indep(struct nvme_ctrl *ctrl, info->is_ready = id->nstat & NVME_NSTAT_NRDY; info->is_rotational = id->nsfeat & NVME_NS_ROTATIONAL; info->no_vwc = id->nsfeat & NVME_NS_VWC_NOT_PRESENT; + info->endgid = le16_to_cpu(id->endgid); } kfree(id); return ret; @@ -2147,6 +2151,101 @@ static int nvme_update_ns_info_generic(struct nvme_ns *ns, return ret; } +static int nvme_check_fdp(struct nvme_ns *ns, struct nvme_ns_info *info, + u8 fdp_idx) +{ + struct nvme_fdp_config_log hdr, *h; + size_t size = sizeof(hdr); + int i, n, ret; + void *log; + + info->runs = 0; + ret = nvme_get_log_lsi(ns->ctrl, 0, NVME_LOG_FDP_CONFIG, 0, NVME_CSI_NVM, + (void *)&hdr, size, 0, info->endgid); + if (ret) + return ret; + + size = le32_to_cpu(hdr.sze); + log = kzalloc(size, GFP_KERNEL); + if (!log) + return 0; + + ret = nvme_get_log_lsi(ns->ctrl, 0, NVME_LOG_FDP_CONFIG, 0, NVME_CSI_NVM, + log, size, 0, info->endgid); + if (ret) + goto out; + + n = le16_to_cpu(h->numfdpc) + 1; + if (fdp_idx > n) + goto out; + + h = log; + log = h->configs; + for (i = 0; i < n; i++) { + struct nvme_fdp_config_desc *config = log; + + if (i == fdp_idx) { + info->runs = le64_to_cpu(config->runs); + break; + } + log += le16_to_cpu(config->size); + } +out: + kfree(h); + return ret; +} + +static int nvme_query_fdp_info(struct nvme_ns *ns, struct nvme_ns_info *info) +{ + struct nvme_ns_head *head = ns->head; + struct nvme_fdp_ruh_status *ruhs; + struct nvme_command c = {}; + u32 fdp, fdp_idx; + int size, ret; + + ret = nvme_get_features(ns->ctrl, NVME_FEAT_FDP, info->endgid, NULL, 0, + &fdp); + if (ret) + goto err; + + if (!(fdp & NVME_FDP_FDPE)) + goto err; + + fdp_idx = (fdp >> NVME_FDP_FDPCIDX_SHIFT) & NVME_FDP_FDPCIDX_MASK; + ret = nvme_check_fdp(ns, info, fdp_idx); + if (ret || !info->runs) + goto err; + + size = struct_size(ruhs, ruhsd, NVME_MAX_PLIDS); + ruhs = kzalloc(size, GFP_KERNEL); + if (!ruhs) { + ret = -ENOMEM; + goto err; + } + + c.imr.opcode = nvme_cmd_io_mgmt_recv; + c.imr.nsid = cpu_to_le32(head->ns_id); + c.imr.mo = NVME_IO_MGMT_RECV_MO_RUHS; + c.imr.numd = cpu_to_le32(nvme_bytes_to_numd(size)); + ret = nvme_submit_sync_cmd(ns->queue, &c, ruhs, size); + if (ret) + goto free; + + head->nr_plids = le16_to_cpu(ruhs->nruhsd); + if (!head->nr_plids) + goto free; + + kfree(ruhs); + return 0; + +free: + kfree(ruhs); +err: + head->nr_plids = 0; + info->runs = 0; + return ret; +} + static int nvme_update_ns_info_block(struct nvme_ns *ns, struct nvme_ns_info *info) { @@ -2183,6 +2282,15 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, goto out; } + if (ns->ctrl->ctratt & NVME_CTRL_ATTR_FDPS) { + ret = nvme_query_fdp_info(ns, info); + if (ret) + dev_warn(ns->ctrl->device, + "FDP failure status:0x%x\n", ret); + if (ret < 0) + goto out; + } + blk_mq_freeze_queue(ns->disk->queue); ns->head->lba_shift = id->lbaf[lbaf].ds; ns->head->nuse = le64_to_cpu(id->nuse); @@ -2216,6 +2324,12 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, if (!nvme_init_integrity(ns->head, &lim, info)) capacity = 0; + lim.max_write_streams = ns->head->nr_plids; + if (lim.max_write_streams) + lim.write_stream_granularity = info->runs; + else + lim.write_stream_granularity = 0; + ret = queue_limits_commit_update(ns->disk->queue, &lim); if (ret) { blk_mq_unfreeze_queue(ns->disk->queue); @@ -2318,6 +2432,8 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_ns_info *info) ns->head->disk->flags |= GENHD_FL_HIDDEN; else nvme_init_integrity(ns->head, &lim, info); + lim.max_write_streams = ns_lim->max_write_streams; + lim.write_stream_granularity = ns_lim->write_stream_granularity; ret = queue_limits_commit_update(ns->head->disk->queue, &lim); set_capacity_and_notify(ns->head->disk, get_capacity(ns->disk)); diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 611b02c8a8b37..5c8bdaa2c8824 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -454,6 +454,8 @@ struct nvme_ns_ids { u8 csi; }; +#define NVME_MAX_PLIDS (S8_MAX - 1) + /* * Anchor structure for namespaces. There is one for each namespace in a * NVMe subsystem that any of our controllers can see, and the namespace @@ -491,6 +493,8 @@ struct nvme_ns_head { struct device cdev_device; struct gendisk *disk; + + u16 nr_plids; #ifdef CONFIG_NVME_MULTIPATH struct bio_list requeue_list; spinlock_t requeue_lock; diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 13377dde4527b..78657a8e39561 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -275,6 +275,7 @@ enum nvme_ctrl_attr { NVME_CTRL_ATTR_HID_128_BIT = (1 << 0), NVME_CTRL_ATTR_TBKAS = (1 << 6), NVME_CTRL_ATTR_ELBAS = (1 << 15), + NVME_CTRL_ATTR_FDPS = (1 << 19), }; struct nvme_id_ctrl { @@ -761,6 +762,34 @@ struct nvme_zone_report { struct nvme_zone_descriptor entries[]; }; +struct nvme_fdp_ruh_desc { + __u8 ruht; + __u8 rsvd1[3]; +}; + +struct nvme_fdp_config_desc { + __le16 size; + __u8 fdpa; + __u8 vss; + __le32 nrg; + __le16 nruh; + __le16 maxpids; + __le32 nnss; + __le64 runs; + __le32 erutl; + __u8 rsvd28[36]; + struct nvme_fdp_ruh_desc ruhs[]; +}; + +struct nvme_fdp_config_log { + __le16 numfdpc; + __u8 ver; + __u8 rsvd3; + __le32 sze; + __u8 rsvd8[8]; + struct nvme_fdp_config_desc configs[]; +}; + enum { NVME_SMART_CRIT_SPARE = 1 << 0, NVME_SMART_CRIT_TEMPERATURE = 1 << 1, @@ -887,6 +916,7 @@ enum nvme_opcode { nvme_cmd_resv_register = 0x0d, nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, + nvme_cmd_io_mgmt_recv = 0x12, nvme_cmd_resv_release = 0x15, nvme_cmd_zone_mgmt_send = 0x79, nvme_cmd_zone_mgmt_recv = 0x7a, @@ -908,6 +938,7 @@ enum nvme_opcode { nvme_opcode_name(nvme_cmd_resv_register), \ nvme_opcode_name(nvme_cmd_resv_report), \ nvme_opcode_name(nvme_cmd_resv_acquire), \ + nvme_opcode_name(nvme_cmd_io_mgmt_recv), \ nvme_opcode_name(nvme_cmd_resv_release), \ nvme_opcode_name(nvme_cmd_zone_mgmt_send), \ nvme_opcode_name(nvme_cmd_zone_mgmt_recv), \ @@ -1059,6 +1090,7 @@ enum { NVME_RW_PRINFO_PRCHK_GUARD = 1 << 12, NVME_RW_PRINFO_PRACT = 1 << 13, NVME_RW_DTYPE_STREAMS = 1 << 4, + NVME_RW_DTYPE_DPLCMT = 2 << 4, NVME_WZ_DEAC = 1 << 9, }; @@ -1146,6 +1178,38 @@ struct nvme_zone_mgmt_recv_cmd { __le32 cdw14[2]; }; +struct nvme_io_mgmt_recv_cmd { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __le64 rsvd2[2]; + union nvme_data_ptr dptr; + __u8 mo; + __u8 rsvd11; + __u16 mos; + __le32 numd; + __le32 cdw12[4]; +}; + +enum { + NVME_IO_MGMT_RECV_MO_RUHS = 1, +}; + +struct nvme_fdp_ruh_status_desc { + u16 pid; + u16 ruhid; + u32 earutr; + u64 ruamw; + u8 rsvd16[16]; +}; + +struct nvme_fdp_ruh_status { + u8 rsvd0[14]; + __le16 nruhsd; + struct nvme_fdp_ruh_status_desc ruhsd[]; +}; + enum { NVME_ZRA_ZONE_REPORT = 0, NVME_ZRASF_ZONE_REPORT_ALL = 0, @@ -1281,6 +1345,7 @@ enum { NVME_FEAT_PLM_WINDOW = 0x14, NVME_FEAT_HOST_BEHAVIOR = 0x16, NVME_FEAT_SANITIZE = 0x17, + NVME_FEAT_FDP = 0x1d, NVME_FEAT_SW_PROGRESS = 0x80, NVME_FEAT_HOST_ID = 0x81, NVME_FEAT_RESV_MASK = 0x82, @@ -1301,6 +1366,7 @@ enum { NVME_LOG_ANA = 0x0c, NVME_LOG_FEATURES = 0x12, NVME_LOG_RMI = 0x16, + NVME_LOG_FDP_CONFIG = 0x20, NVME_LOG_DISC = 0x70, NVME_LOG_RESERVATION = 0x80, NVME_FWACT_REPL = (0 << 3), @@ -1326,6 +1392,12 @@ enum { NVME_FIS_CSCPE = 1 << 21, }; +enum { + NVME_FDP_FDPE = 1 << 0, + NVME_FDP_FDPCIDX_SHIFT = 8, + NVME_FDP_FDPCIDX_MASK = 0xff, +}; + /* NVMe Namespace Write Protect State */ enum { NVME_NS_NO_WRITE_PROTECT = 0, @@ -1888,6 +1960,7 @@ struct nvme_command { struct nvmf_auth_receive_command auth_receive; struct nvme_dbbuf dbbuf; struct nvme_directive_cmd directive; + struct nvme_io_mgmt_recv_cmd imr; }; }; From patchwork Fri Dec 6 01:53:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13896252 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A28D115098A for ; Fri, 6 Dec 2024 01:53:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450016; cv=none; b=gbszaCTMq5w/4Th+QtQ6R52kw77jk2KebGtWIHmoHFEZxA28G5PESG34vzlRBGwslqKYg0NWXwLKxcOOsPoZIgU48wrGkRacNpPlXswDU+FrQZtd4IgPI523rexiJJwlLlUP6/gMb7Q1OamECHle6kPLVgLw0psey61NlgRI7mM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733450016; c=relaxed/simple; bh=oSucAwfLbsC0ZrBCRDPimPdNR0z47lozSlk2cE3qr84=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Z/c/Alu/SacE1BJirqbm3W6DLuqz4IIDfUKaX4xXpwAjRO5/nmC/ygCZH1aCxM8zf3HcL+QIH9GqemTGTbZTSkaw2ui5rVzgJiLYHStne6A0fNAc3s9Iv84tNPebSQqzsvN99wYY1Ez+rrnfSO6nKDGzTjXT/hDU7w9UjCV1j6c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=JXsdLF5S; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="JXsdLF5S" Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B603MMv014363 for ; Thu, 5 Dec 2024 17:53:33 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=zlAb0pn1DAyajDFVPLOKoAhYRE5EXTu7nKt9gAZEWPg=; b=JXsdLF5SM5mK nkAguF4E9ngzY2b6bVDyQv5PIYAYIwM4Jt2tU/BPsCZz5OKM4CFNc7vB9qFllRcj ZSVHryvVwjTYzVXPGfhGTM7w38LwXAHcuOgGOFUEP9lyIuul+5TRC8yA9KGyqCF5 rmX5WJEYi7a2tsKCTx0EXLHo1uoFzGbTonUv9T2BXP+YcQfziggKJa4h9CYGWRFv XXzyvtaxN3TH6ZlCpMWubAR6YtgpDN4v20KaGdPK/4SnReoIOIoJhaVi9YLCTDhl Pl7sgcli9DAEYyWh9NHItfVUqXbwVjaa7j+q4UsidV6odQrP+z7KHjCQdpZjf7N9 q1FVzkGdwg== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43bmru153t-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 05 Dec 2024 17:53:33 -0800 (PST) Received: from twshared9216.15.frc2.facebook.com (2620:10d:c085:108::150d) by mail.thefacebook.com (2620:10d:c08b:78::c78f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Fri, 6 Dec 2024 01:53:25 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 6C47E15B2115E; Thu, 5 Dec 2024 17:53:09 -0800 (PST) From: Keith Busch To: , , , , , CC: , , Keith Busch Subject: [PATCHv11 10/10] nvme: use fdp streams if write stream is provided Date: Thu, 5 Dec 2024 17:53:08 -0800 Message-ID: <20241206015308.3342386-11-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241206015308.3342386-1-kbusch@meta.com> References: <20241206015308.3342386-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: TcIhh-CuIQLxQToKHoeRcRIh-Gqk-ZXV X-Proofpoint-ORIG-GUID: TcIhh-CuIQLxQToKHoeRcRIh-Gqk-ZXV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Maps a user requested write stream to an FDP placement ID if possible. Signed-off-by: Keith Busch --- drivers/nvme/host/core.c | 32 +++++++++++++++++++++++++++++++- drivers/nvme/host/nvme.h | 1 + 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 410a77de92f88..c6f48403fc51c 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -997,6 +997,18 @@ static inline blk_status_t nvme_setup_rw(struct nvme_ns *ns, if (req->cmd_flags & REQ_RAHEAD) dsmgmt |= NVME_RW_DSM_FREQ_PREFETCH; + if (op == nvme_cmd_write && ns->head->nr_plids) { + u16 write_stream = req->bio->bi_write_stream; + + if (WARN_ON_ONCE(write_stream > ns->head->nr_plids)) + return BLK_STS_INVAL; + + if (write_stream) { + dsmgmt |= ns->head->plids[write_stream - 1] << 16; + control |= NVME_RW_DTYPE_DPLCMT; + } + } + if (req->cmd_flags & REQ_ATOMIC && !nvme_valid_atomic_write(req)) return BLK_STS_INVAL; @@ -2197,11 +2209,12 @@ static int nvme_check_fdp(struct nvme_ns *ns, struct nvme_ns_info *info, static int nvme_query_fdp_info(struct nvme_ns *ns, struct nvme_ns_info *info) { + struct nvme_fdp_ruh_status_desc *ruhsd; struct nvme_ns_head *head = ns->head; struct nvme_fdp_ruh_status *ruhs; struct nvme_command c = {}; u32 fdp, fdp_idx; - int size, ret; + int size, ret, i; ret = nvme_get_features(ns->ctrl, NVME_FEAT_FDP, info->endgid, NULL, 0, &fdp); @@ -2235,6 +2248,19 @@ static int nvme_query_fdp_info(struct nvme_ns *ns, struct nvme_ns_info *info) if (!head->nr_plids) goto free; + head->nr_plids = min(head->nr_plids, NVME_MAX_PLIDS); + head->plids = kcalloc(head->nr_plids, sizeof(head->plids), + GFP_KERNEL); + if (!head->plids) { + ret = -ENOMEM; + goto free; + } + + for (i = 0; i < head->nr_plids; i++) { + ruhsd = &ruhs->ruhsd[i]; + head->plids[i] = le16_to_cpu(ruhsd->pid); + } + kfree(ruhs); return 0; @@ -2289,6 +2315,10 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, "FDP failure status:0x%x\n", ret); if (ret < 0) goto out; + } else { + ns->head->nr_plids = 0; + kfree(ns->head->plids); + ns->head->plids = NULL; } blk_mq_freeze_queue(ns->disk->queue); diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 5c8bdaa2c8824..4c12d35b3e39e 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -495,6 +495,7 @@ struct nvme_ns_head { struct gendisk *disk; u16 nr_plids; + u16 *plids; #ifdef CONFIG_NVME_MULTIPATH struct bio_list requeue_list; spinlock_t requeue_lock;