From patchwork Tue Dec 17 12:32:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Harmstone X-Patchwork-Id: 13911702 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8DE91E3DFA for ; Tue, 17 Dec 2024 12:33:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734438788; cv=none; b=ZphkI5E1eOERmbOBBwGqUXJVtTXWvx32JEjQKLj8AfHL1CjcngkBsfuyFOYSTC4iA4+o11cnPdMw/S+phEFEBZ+/n/IZgD358WvJrL6/ADKSnrFPf+g5eu7JfFqqAXKeH+XjkyMdAqzD/xWCws5G1H9JLI8+sbuKb6v2O8v669M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734438788; c=relaxed/simple; bh=nyPLepdPiFn6D+OGD+MjSqXBJcNJ9+UCod+pDt9/UQ0=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=nypjEiBiHtSjPjb352dqc9UQWiKdBu8nLazMGj4G1Uh0W0XdYmUuisxUcZ2v+SImbLkTLRHTKMJSwXjQEX/TIDJsrXVderf+b8z5N1dn47CJi3euRkWoVmifAJBcoz8vEPpSloLy4or4wpl1yCzhmP8a3IfBpaTvH+p82F14wZ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b=QxywzRdt; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="QxywzRdt" Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.18.1.2/8.18.1.2) with ESMTP id 4BHBdjoN019843 for ; Tue, 17 Dec 2024 04:33:05 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to; s=facebook; bh=bKv5BuYbPe0RVZ5KawS7S2S fYc8+BUTb6VmUSA7oGGU=; b=QxywzRdtNQYKdSmbEKRYOlXQP5wBiUvg1HkdA8W 3sRDmGb8xhhfapENGdJHKSq6jHxu/H2gOP+znqarJzaNlP1gqcn/JIcMI2zviwEz NmTpfSOne8db5lSyCMhfHz+iiLwnQ+BeR9Q7xHIYnqq6/Bvrs4WuF1YsBdJjsxmn 8Vj8= Received: from maileast.thefacebook.com ([163.114.135.16]) by m0089730.ppops.net (PPS) with ESMTPS id 43k0n22jd3-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Tue, 17 Dec 2024 04:33:05 -0800 (PST) Received: from twshared40462.17.frc2.facebook.com (2620:10d:c0a8:1c::1b) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Tue, 17 Dec 2024 12:33:03 +0000 Received: by devbig276.nha1.facebook.com (Postfix, from userid 660015) id A705D9A23362; Tue, 17 Dec 2024 12:32:56 +0000 (GMT) From: Mark Harmstone To: CC: , , Mark Harmstone Subject: [PATCH 1/2] configure: use pkg-config to find liburing Date: Tue, 17 Dec 2024 12:32:43 +0000 Message-ID: <20241217123249.2481120-1-maharmstone@fb.com> X-Mailer: git-send-email 2.43.5 Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: IKfZyKauy5FYDk8Z-2PII8G4J11iuY6P X-Proofpoint-ORIG-GUID: IKfZyKauy5FYDk8Z-2PII8G4J11iuY6P X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 Change our autoconf macros so that instead of checking for the presence of liburing.h, we use pkg-config. The benefit of this is that we can then check the version of liburing, and do conditional compilation based on this. There's a macro IO_URING_CHECK_VERSION already, but it's only in relatively recent versions of liburing.h. This replaces HAVE_URING_H, defined by AC_CHECK_HEADERS, with HAVE_URING. I also had to rename PKG_{MAJOR,MINOR,REVISION,BUILD} to start with PACKAGE_, to avoid "possibly undefined macro" errors; it looks like pkg-config assumes that anything called PKG_* is for its own use. Signed-off-by: Mark Harmstone --- VERSION | 8 ++++---- m4/package_globals.m4 | 4 ++-- m4/package_liburing.m4 | 6 +++++- release.sh | 2 +- src/feature.c | 4 ++-- src/vfs/idmapped-mounts.c | 6 +++--- src/vfs/idmapped-mounts.h | 2 +- src/vfs/tmpfs-idmapped-mounts.c | 6 +++--- src/vfs/utils.c | 4 ++-- src/vfs/utils.h | 6 +++--- src/vfs/vfstest.c | 6 +++--- 11 files changed, 29 insertions(+), 25 deletions(-) diff --git a/VERSION b/VERSION index 7294a002..afcab53e 100644 --- a/VERSION +++ b/VERSION @@ -1,7 +1,7 @@ # # This file is used by configure to get version information # -PKG_MAJOR=1 -PKG_MINOR=1 -PKG_REVISION=1 -PKG_BUILD=1 +PACKAGE_MAJOR=1 +PACKAGE_MINOR=1 +PACKAGE_REVISION=1 +PACKAGE_BUILD=1 diff --git a/m4/package_globals.m4 b/m4/package_globals.m4 index ce7a8c51..c8d5d124 100644 --- a/m4/package_globals.m4 +++ b/m4/package_globals.m4 @@ -9,9 +9,9 @@ AC_DEFUN([AC_PACKAGE_GLOBALS], AC_SUBST(pkg_name) . ./VERSION - pkg_version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} + pkg_version=${PACKAGE_MAJOR}.${PACKAGE_MINOR}.${PACKAGE_REVISION} AC_SUBST(pkg_version) - pkg_release=$PKG_BUILD + pkg_release=$PACKAGE_BUILD test -z "$BUILD_VERSION" || pkg_release="$BUILD_VERSION" AC_SUBST(pkg_release) diff --git a/m4/package_liburing.m4 b/m4/package_liburing.m4 index c92cc02a..0553966d 100644 --- a/m4/package_liburing.m4 +++ b/m4/package_liburing.m4 @@ -1,4 +1,8 @@ AC_DEFUN([AC_PACKAGE_WANT_URING], - [ AC_CHECK_HEADERS(liburing.h, [ have_uring=true ], [ have_uring=false ]) + [ PKG_CHECK_MODULES([LIBURING], [liburing], + [ AC_DEFINE([HAVE_LIBURING], [1], [Use liburing]) + have_uring=true + ], + [ have_uring=false ]) AC_SUBST(have_uring) ]) diff --git a/release.sh b/release.sh index 5b78ec79..70fbf47e 100644 --- a/release.sh +++ b/release.sh @@ -5,7 +5,7 @@ . ./VERSION -version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} +version=${PACKAGE_MAJOR}.${PACKAGE_MINOR}.${PACKAGE_REVISION} date=`date +"%-d %B %Y"` echo "Cleaning up" diff --git a/src/feature.c b/src/feature.c index 7e474ce5..7df36acf 100644 --- a/src/feature.c +++ b/src/feature.c @@ -42,7 +42,7 @@ #include #endif -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING #include #endif @@ -227,7 +227,7 @@ check_aio_support(void) static int check_uring_support(void) { -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING struct io_uring ring; int err; diff --git a/src/vfs/idmapped-mounts.c b/src/vfs/idmapped-mounts.c index f4dfc3f3..ed9992f9 100644 --- a/src/vfs/idmapped-mounts.c +++ b/src/vfs/idmapped-mounts.c @@ -2206,7 +2206,7 @@ out: } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING int tcore_io_uring_idmapped(const struct vfstest_info *info) { int fret = -1; @@ -2743,7 +2743,7 @@ out_unmap: return fret; } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ /* Validate that protected symlinks work correctly on idmapped mounts. */ int tcore_protected_symlinks_idmapped_mounts(const struct vfstest_info *info) @@ -8859,7 +8859,7 @@ static const struct test_struct t_idmapped_mounts[] = { { tcore_hardlink_crossing_idmapped_mounts, true, "cross idmapped mount hardlink", }, { tcore_hardlink_from_idmapped_mount, true, "hardlinks from idmapped mounts", }, { tcore_hardlink_from_idmapped_mount_in_userns, true, "hardlinks from idmapped mounts in user namespace", }, -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING { tcore_io_uring_idmapped, true, "io_uring from idmapped mounts", }, { tcore_io_uring_idmapped_userns, true, "io_uring from idmapped mounts in user namespace", }, { tcore_io_uring_idmapped_unmapped, true, "io_uring from idmapped mounts with unmapped ids", }, diff --git a/src/vfs/idmapped-mounts.h b/src/vfs/idmapped-mounts.h index 4a2c7b39..688394c8 100644 --- a/src/vfs/idmapped-mounts.h +++ b/src/vfs/idmapped-mounts.h @@ -30,7 +30,7 @@ int tcore_fscaps_idmapped_mounts_in_userns_separate_userns(const struct vfstest_ int tcore_hardlink_crossing_idmapped_mounts(const struct vfstest_info *info); int tcore_hardlink_from_idmapped_mount(const struct vfstest_info *info); int tcore_hardlink_from_idmapped_mount_in_userns(const struct vfstest_info *info); -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING int tcore_io_uring_idmapped(const struct vfstest_info *info); int tcore_io_uring_idmapped_userns(const struct vfstest_info *info); int tcore_io_uring_idmapped_unmapped(const struct vfstest_info *info); diff --git a/src/vfs/tmpfs-idmapped-mounts.c b/src/vfs/tmpfs-idmapped-mounts.c index 0899aed9..d8212bce 100644 --- a/src/vfs/tmpfs-idmapped-mounts.c +++ b/src/vfs/tmpfs-idmapped-mounts.c @@ -167,7 +167,7 @@ static int tmpfs_hardlink_from_idmapped_mount_in_userns(const struct vfstest_inf return tmpfs_nested_mount_setup(info, tcore_hardlink_from_idmapped_mount_in_userns); } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING static int tmpfs_io_uring_idmapped(const struct vfstest_info *info) { return tmpfs_nested_mount_setup(info, tcore_io_uring_idmapped); @@ -184,7 +184,7 @@ static int tmpfs_io_uring_idmapped_unmapped_userns(const struct vfstest_info *in { return tmpfs_nested_mount_setup(info, tcore_io_uring_idmapped_unmapped_userns); } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ static int tmpfs_protected_symlinks_idmapped_mounts(const struct vfstest_info *info) { @@ -272,7 +272,7 @@ static const struct test_struct t_tmpfs[] = { { tmpfs_hardlink_crossing_idmapped_mounts, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs cross idmapped mount hardlink", }, { tmpfs_hardlink_from_idmapped_mount, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs hardlinks from idmapped mounts", }, { tmpfs_hardlink_from_idmapped_mount_in_userns, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs hardlinks from idmapped mounts in user namespace", }, -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING { tmpfs_io_uring_idmapped, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs io_uring from idmapped mounts", }, { tmpfs_io_uring_idmapped_userns, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs io_uring from idmapped mounts in user namespace", }, { tmpfs_io_uring_idmapped_unmapped, T_REQUIRE_USERNS | T_REQUIRE_IDMAPPED_MOUNTS, "tmpfs io_uring from idmapped mounts with unmapped ids", }, diff --git a/src/vfs/utils.c b/src/vfs/utils.c index 0ab5de15..c1c7951c 100644 --- a/src/vfs/utils.c +++ b/src/vfs/utils.c @@ -502,7 +502,7 @@ out: return fret; } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING int io_uring_openat_with_creds(struct io_uring *ring, int dfd, const char *path, int cred_id, bool with_link, int *ret_cqe) { @@ -555,7 +555,7 @@ int io_uring_openat_with_creds(struct io_uring *ring, int dfd, const char *path, out: return ret; } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ /* caps_up - raise all permitted caps */ int caps_up(void) diff --git a/src/vfs/utils.h b/src/vfs/utils.h index 872fd96f..c086885a 100644 --- a/src/vfs/utils.h +++ b/src/vfs/utils.h @@ -25,7 +25,7 @@ #include #endif -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING #include #endif @@ -349,11 +349,11 @@ static inline bool switch_fsids(uid_t fsuid, gid_t fsgid) return true; } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING extern int io_uring_openat_with_creds(struct io_uring *ring, int dfd, const char *path, int cred_id, bool with_link, int *ret_cqe); -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ extern int chown_r(int fd, const char *path, uid_t uid, gid_t gid); extern int rm_r(int fd, const char *path); diff --git a/src/vfs/vfstest.c b/src/vfs/vfstest.c index f842117d..e0c897bb 100644 --- a/src/vfs/vfstest.c +++ b/src/vfs/vfstest.c @@ -1222,7 +1222,7 @@ out: return fret; } -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING static int io_uring(const struct vfstest_info *info) { int fret = -1; @@ -1495,7 +1495,7 @@ out_unmap: return fret; } -#endif /* HAVE_LIBURING_H */ +#endif /* HAVE_LIBURING */ /* The following tests are concerned with setgid inheritance. These can be * filesystem type specific. For xfs, if a new file or directory or node is @@ -2349,7 +2349,7 @@ static const struct option longopts[] = { static const struct test_struct t_basic[] = { { fscaps, T_REQUIRE_USERNS, "fscaps on regular mounts", }, { hardlink_crossing_mounts, 0, "cross mount hardlink", }, -#ifdef HAVE_LIBURING_H +#ifdef HAVE_LIBURING { io_uring, 0, "io_uring", }, { io_uring_userns, T_REQUIRE_USERNS, "io_uring in user namespace", }, #endif From patchwork Tue Dec 17 12:32:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Harmstone X-Patchwork-Id: 13911701 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A03631EB9E3 for ; Tue, 17 Dec 2024 12:33:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734438785; cv=none; b=IvK2gFPW74I+9ReXEgalHWR9M8qJEZ0iX6MOxX3VWwgHfrs3M7fpPIfpF0+rrqs57ARYG9CxyuR4twLop8fv2bBH21aIVTLeALj5IXD4l2sFF/I4SjWQbmuUpcKMej0/y2GYpdXo+dHpDRpFbcYyWled2aJLuAopFkWVOuanE0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734438785; c=relaxed/simple; bh=VmYQXiQeZyrr0gMWP/dC53J86WZa9jWHk70xC76F050=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=YxLMlAWy86wsPuYp/+Le27Zj6Wrdkk9ULBuz/CdQNnallmhbwR+OFZB4C5ANzhNjqbjOWdNd8o6f569TaYM6qd8SdxZ4iS8oGDy3SOEXAGJs43jEBIJ+vyg+EhFtVzeQySeBtpb/KWBech8X1TDvo2aJ00VgOlfIX13dVxoEJCk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b=bjrUZp3Z; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fb.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="bjrUZp3Z" Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BHBdUil007140 for ; Tue, 17 Dec 2024 04:33:03 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=facebook; bh=w 1QuYk3ltztZImSw/4lNjAM1YI6ATMLVDtU+9OR+Phc=; b=bjrUZp3ZCYFPi6hYm nOisr0sdFPydBR1VljMYTmLb38TLZda1/mLlQ0aba+cx/hKuRreyNIEVB4MjMPt4 S+FOsaIBnkbCbPetlJggHU3zL87ORKx6XtIuoXY26auWItGCdhsYerrRTiy+8BeZ TeQ4jkbDoW3/Gu/ipeH5RCG7R0= Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43k71sgrwx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Tue, 17 Dec 2024 04:33:02 -0800 (PST) Received: from twshared11145.37.frc1.facebook.com (2620:10d:c085:108::150d) by mail.thefacebook.com (2620:10d:c08b:78::c78f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Tue, 17 Dec 2024 12:33:01 +0000 Received: by devbig276.nha1.facebook.com (Postfix, from userid 660015) id AACD99A2336C; Tue, 17 Dec 2024 12:32:58 +0000 (GMT) From: Mark Harmstone To: CC: , , Mark Harmstone Subject: [PATCH 2/2] btrfs: add test for encoded reads Date: Tue, 17 Dec 2024 12:32:44 +0000 Message-ID: <20241217123249.2481120-2-maharmstone@fb.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241217123249.2481120-1-maharmstone@fb.com> References: <20241217123249.2481120-1-maharmstone@fb.com> Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: tvVoPEPz99yPpAbyNd3jj3boCDWq77TW X-Proofpoint-GUID: tvVoPEPz99yPpAbyNd3jj3boCDWq77TW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 Add btrfs/333 and its helper programs btrfs_encoded_read and btrfs_encoded_write, in order to test encoded reads. We use the BTRFS_IOC_ENCODED_WRITE ioctl to write random data into a compressed extent, then use the BTRFS_IOC_ENCODED_READ ioctl to check that it matches what we've written. If the new io_uring interface for encoded reads is supported, we also check that that matches the ioctl. Note that what we write isn't valid compressed data, so any non-encoded reads on these files will fail. Signed-off-by: Mark Harmstone --- (This is a resend, with changes made to ensure that it compiles on old versions of liburing.) .gitignore | 2 + m4/package_liburing.m4 | 2 + src/Makefile | 3 +- src/btrfs_encoded_read.c | 186 ++++++++++++++++++++++++++++++++ src/btrfs_encoded_write.c | 217 +++++++++++++++++++++++++++++++++++++ tests/btrfs/333 | 220 ++++++++++++++++++++++++++++++++++++++ tests/btrfs/333.out | 2 + 7 files changed, 631 insertions(+), 1 deletion(-) create mode 100644 src/btrfs_encoded_read.c create mode 100644 src/btrfs_encoded_write.c create mode 100755 tests/btrfs/333 create mode 100644 tests/btrfs/333.out diff --git a/.gitignore b/.gitignore index f16173d9..efd47773 100644 --- a/.gitignore +++ b/.gitignore @@ -62,6 +62,8 @@ tags /src/attr_replace_test /src/attr-list-by-handle-cursor-test /src/bstat +/src/btrfs_encoded_read +/src/btrfs_encoded_write /src/bulkstat_null_ocount /src/bulkstat_unlink_test /src/bulkstat_unlink_test_modified diff --git a/m4/package_liburing.m4 b/m4/package_liburing.m4 index 0553966d..7fbf4a5f 100644 --- a/m4/package_liburing.m4 +++ b/m4/package_liburing.m4 @@ -1,6 +1,8 @@ AC_DEFUN([AC_PACKAGE_WANT_URING], [ PKG_CHECK_MODULES([LIBURING], [liburing], [ AC_DEFINE([HAVE_LIBURING], [1], [Use liburing]) + AC_DEFINE_UNQUOTED([LIBURING_MAJOR_VERSION], [`$PKG_CONFIG --modversion liburing | cut -d. -f1`], [liburing major version]) + AC_DEFINE_UNQUOTED([LIBURING_MINOR_VERSION], [`$PKG_CONFIG --modversion liburing | cut -d. -f2`], [liburing minor version]) have_uring=true ], [ have_uring=false ]) diff --git a/src/Makefile b/src/Makefile index a0396332..b42b8147 100644 --- a/src/Makefile +++ b/src/Makefile @@ -34,7 +34,8 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ - uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment + uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ + btrfs_encoded_read btrfs_encoded_write EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \ diff --git a/src/btrfs_encoded_read.c b/src/btrfs_encoded_read.c new file mode 100644 index 00000000..c3ec7a9b --- /dev/null +++ b/src/btrfs_encoded_read.c @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) Meta Platforms, Inc. and affiliates. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "config.h" + +#ifdef HAVE_LIBURING +#include +#endif + +/* IORING_OP_URING_CMD defined from liburing 2.2 onwards */ +#if defined(HAVE_LIBURING) && (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 2)) +#define IORING_OP_URING_CMD 46 +#endif + +#define BTRFS_MAX_COMPRESSED 131072 +#define QUEUE_DEPTH 1 + +static int encoded_read_ioctl(const char *filename, long long offset) +{ + int ret, fd; + char buf[BTRFS_MAX_COMPRESSED]; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + + fd = open(filename, O_RDONLY); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + return 1; + } + + iov.iov_base = buf; + iov.iov_len = sizeof(buf); + + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.flags = 0; + + ret = ioctl(fd, BTRFS_IOC_ENCODED_READ, &enc); + + if (ret < 0) { + printf("%i\n", -errno); + close(fd); + return 0; + } + + close(fd); + + printf("%i\n", ret); + printf("%llu\n", enc.len); + printf("%llu\n", enc.unencoded_len); + printf("%llu\n", enc.unencoded_offset); + printf("%u\n", enc.compression); + printf("%u\n", enc.encryption); + + fwrite(buf, ret, 1, stdout); + + return 0; +} + +static int encoded_read_io_uring(const char *filename, long long offset) +{ +#ifdef HAVE_LIBURING + int ret, fd; + char buf[BTRFS_MAX_COMPRESSED]; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + struct io_uring ring; + struct io_uring_sqe *sqe; + struct io_uring_cqe *cqe; + + io_uring_queue_init(QUEUE_DEPTH, &ring, 0); + + fd = open(filename, O_RDONLY); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + ret = 1; + goto out_uring; + } + + iov.iov_base = buf; + iov.iov_len = sizeof(buf); + + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.flags = 0; + + sqe = io_uring_get_sqe(&ring); + if (!sqe) { + fprintf(stderr, "io_uring_get_sqe failed\n"); + ret = 1; + goto out_close; + } + + io_uring_prep_rw(IORING_OP_URING_CMD, sqe, fd, &enc, sizeof(enc), 0); + + /* sqe->cmd_op union'd to sqe->off from liburing 2.3 onwards */ +#if (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 3)) + sqe->off = BTRFS_IOC_ENCODED_READ; +#else + sqe->cmd_op = BTRFS_IOC_ENCODED_READ; +#endif + + io_uring_submit(&ring); + + ret = io_uring_wait_cqe(&ring, &cqe); + if (ret < 0) { + fprintf(stderr, "io_uring_wait_cqe returned %i\n", ret); + ret = 1; + goto out_close; + } + + io_uring_cqe_seen(&ring, cqe); + + if (cqe->res < 0) { + printf("%i\n", cqe->res); + ret = 0; + goto out_close; + } + + printf("%i\n", cqe->res); + printf("%llu\n", enc.len); + printf("%llu\n", enc.unencoded_len); + printf("%llu\n", enc.unencoded_offset); + printf("%u\n", enc.compression); + printf("%u\n", enc.encryption); + + fwrite(buf, cqe->res, 1, stdout); + + ret = 0; + +out_close: + close(fd); + +out_uring: + io_uring_queue_exit(&ring); + + return ret; +#else + fprintf(stderr, "liburing not linked in\n"); + return 1; +#endif +} + +static void usage() +{ + fprintf(stderr, "Usage: btrfs_encoded_read ioctl|io_uring filename offset\n"); +} + +int main(int argc, char *argv[]) +{ + const char *filename; + long long offset; + + if (argc != 4) { + usage(); + return 1; + } + + filename = argv[2]; + + offset = atoll(argv[3]); + if (offset == 0 && errno != 0) { + usage(); + return 1; + } + + if (!strcmp(argv[1], "ioctl")) { + return encoded_read_ioctl(filename, offset); + } else if (!strcmp(argv[1], "io_uring")) { + return encoded_read_io_uring(filename, offset); + } else { + usage(); + return 1; + } +} diff --git a/src/btrfs_encoded_write.c b/src/btrfs_encoded_write.c new file mode 100644 index 00000000..98f4a5b2 --- /dev/null +++ b/src/btrfs_encoded_write.c @@ -0,0 +1,217 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) Meta Platforms, Inc. and affiliates. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "config.h" + +#ifdef HAVE_LIBURING +#include +#endif + +/* IORING_OP_URING_CMD defined from liburing 2.2 onwards */ +#if defined(HAVE_LIBURING) && (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 2)) +#define IORING_OP_URING_CMD 46 +#endif + +#define BTRFS_MAX_COMPRESSED 131072 +#define QUEUE_DEPTH 1 + +static int encoded_write_ioctl(const char *filename, long long offset, + long long len, long long unencoded_len, + long long unencoded_offset, int compression, + char *buf, size_t size) +{ + int ret, fd; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + + fd = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0644); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + return 1; + } + + iov.iov_base = buf; + iov.iov_len = size; + + memset(&enc, 0, sizeof(enc)); + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.len = len; + enc.unencoded_len = unencoded_len; + enc.unencoded_offset = unencoded_offset; + enc.compression = compression; + + ret = ioctl(fd, BTRFS_IOC_ENCODED_WRITE, &enc); + + if (ret < 0) { + printf("%i\n", -errno); + close(fd); + return 0; + } + + printf("%i\n", ret); + + close(fd); + + return 0; +} + +static int encoded_write_io_uring(const char *filename, long long offset, + long long len, long long unencoded_len, + long long unencoded_offset, int compression, + char *buf, size_t size) +{ +#ifdef HAVE_LIBURING + int ret, fd; + struct iovec iov; + struct btrfs_ioctl_encoded_io_args enc; + struct io_uring ring; + struct io_uring_sqe *sqe; + struct io_uring_cqe *cqe; + + io_uring_queue_init(QUEUE_DEPTH, &ring, 0); + + fd = open(filename, O_CREAT | O_TRUNC | O_WRONLY, 0644); + if (fd < 0) { + fprintf(stderr, "open failed for %s\n", filename); + ret = 1; + goto out_uring; + } + + iov.iov_base = buf; + iov.iov_len = size; + + memset(&enc, 0, sizeof(enc)); + enc.iov = &iov; + enc.iovcnt = 1; + enc.offset = offset; + enc.len = len; + enc.unencoded_len = unencoded_len; + enc.unencoded_offset = unencoded_offset; + enc.compression = compression; + + sqe = io_uring_get_sqe(&ring); + if (!sqe) { + fprintf(stderr, "io_uring_get_sqe failed\n"); + ret = 1; + goto out_close; + } + + io_uring_prep_rw(IORING_OP_URING_CMD, sqe, fd, &enc, sizeof(enc), 0); + + /* sqe->cmd_op union'd to sqe->off from liburing 2.3 onwards */ +#if (LIBURING_MAJOR_VERSION < 2 || (LIBURING_MAJOR_VERSION == 2 && LIBURING_MINOR_VERSION < 3)) + sqe->off = BTRFS_IOC_ENCODED_WRITE; +#else + sqe->cmd_op = BTRFS_IOC_ENCODED_WRITE; +#endif + + io_uring_submit(&ring); + + ret = io_uring_wait_cqe(&ring, &cqe); + if (ret < 0) { + fprintf(stderr, "io_uring_wait_cqe returned %i\n", ret); + ret = 1; + goto out_close; + } + + io_uring_cqe_seen(&ring, cqe); + + if (cqe->res < 0) { + printf("%i\n", cqe->res); + ret = 0; + goto out_close; + } + + printf("%i\n", cqe->res); + + ret = 0; + +out_close: + close(fd); + +out_uring: + io_uring_queue_exit(&ring); + + return ret; +#else + fprintf(stderr, "liburing not linked in\n"); + return 1; +#endif +} + +static void usage() +{ + fprintf(stderr, "Usage: btrfs_encoded_write ioctl|io_uring filename offset len unencoded_len unencoded_offset compression\n"); +} + +int main(int argc, char *argv[]) +{ + const char *filename; + long long offset, len, unencoded_len, unencoded_offset; + int compression; + char buf[BTRFS_MAX_COMPRESSED]; + size_t size; + + if (argc != 8) { + usage(); + return 1; + } + + filename = argv[2]; + + offset = atoll(argv[3]); + if (offset == 0 && errno != 0) { + usage(); + return 1; + } + + len = atoll(argv[4]); + if (len == 0 && errno != 0) { + usage(); + return 1; + } + + unencoded_len = atoll(argv[5]); + if (unencoded_len == 0 && errno != 0) { + usage(); + return 1; + } + + unencoded_offset = atoll(argv[6]); + if (unencoded_offset == 0 && errno != 0) { + usage(); + return 1; + } + + compression = atoi(argv[7]); + if (compression == 0 && errno != 0) { + usage(); + return 1; + } + + size = fread(buf, 1, BTRFS_MAX_COMPRESSED, stdin); + + if (!strcmp(argv[1], "ioctl")) { + return encoded_write_ioctl(filename, offset, len, unencoded_len, + unencoded_offset, compression, buf, + size); + } else if (!strcmp(argv[1], "io_uring")) { + return encoded_write_io_uring(filename, offset, len, + unencoded_len, unencoded_offset, + compression, buf, size); + } else { + usage(); + return 1; + } +} diff --git a/tests/btrfs/333 b/tests/btrfs/333 new file mode 100755 index 00000000..8e4de4c0 --- /dev/null +++ b/tests/btrfs/333 @@ -0,0 +1,220 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024 Meta Platforms, Inc. All Rights Reserved. +# +# FS QA Test No. btrfs/333 +# +# Test btrfs encoded reads + +. ./common/preamble +_begin_fstest auto quick compress rw + +. ./common/filter +. ./common/btrfs + +_supported_fs btrfs + +do_encoded_read() { + local fn=$1 + local type=$2 + local exp_ret=$3 + local exp_len=$4 + local exp_unencoded_len=$5 + local exp_unencoded_offset=$6 + local exp_compression=$7 + local exp_md5=$8 + + local tmpfile=`mktemp` + + echo "running btrfs_encoded_read $type $fn 0 > $tmpfile" >>$seqres.full + src/btrfs_encoded_read $type $fn 0 > $tmpfile + + if [[ $? -ne 0 ]]; then + echo "btrfs_encoded_read failed" >>$seqres.full + rm $tmpfile + return 1 + fi + + exec {FD}< $tmpfile + + read -u ${FD} ret + + if [[ $ret == -95 && $type -eq "io_uring" ]]; then + echo "btrfs io_uring encoded read failed with -EOPNOTSUPP, skipping" >>$seqres.full + exec {FD}<&- + return 1 + fi + + if [[ $ret -lt 0 ]]; then + if [[ $ret == -1 ]]; then + echo "btrfs encoded read failed with -EPERM; are you running as root?" >>$seqres.full + else + echo "btrfs encoded read failed (errno $ret)" >>$seqres.full + fi + exec {FD}<&- + return 1 + fi + + local status=0 + + if [[ $ret -ne $exp_ret ]]; then + echo "$fn: btrfs encoded read returned $ret, expected $exp_ret" >>$seqres.full + status=1 + fi + + read -u ${FD} len + read -u ${FD} unencoded_len + read -u ${FD} unencoded_offset + read -u ${FD} compression + read -u ${FD} encryption + + local filesize=`stat -c%s $tmpfile` + local datafile=`mktemp` + + dd if=$tmpfile of=$datafile bs=1 count=$ret skip=$(($filesize-$ret)) status=none + + exec {FD}<&- + rm $tmpfile + + local md5=`md5sum $datafile | cut -d ' ' -f 1` + rm $datafile + + if [[ $len -ne $exp_len ]]; then + echo "$fn: btrfs encoded read had len of $len, expected $exp_len" >>$seqres.full + status=1 + fi + + if [[ $unencoded_len -ne $exp_unencoded_len ]]; then + echo "$fn: btrfs encoded read had unencoded_len of $unencoded_len, expected $exp_unencoded_len" >>$seqres.full + status=1 + fi + + if [[ $unencoded_offset -ne $exp_unencoded_offset ]]; then + echo "$fn: btrfs encoded read had unencoded_offset of $unencoded_offset, expected $exp_unencoded_offset" >>$seqres.full + status=1 + fi + + if [[ $compression -ne $exp_compression ]]; then + echo "$fn: btrfs encoded read had compression of $compression, expected $exp_compression" >>$seqres.full + status=1 + fi + + if [[ $encryption -ne 0 ]]; then + echo "$fn: btrfs encoded read had encryption of $encryption, expected 0" >>$seqres.full + status=1 + fi + + if [[ $md5 != $exp_md5 ]]; then + echo "$fn: data returned had hash of $md5, expected $exp_md5" >>$seqres.full + status=1 + fi + + return $status +} + +do_encoded_write() { + local fn=$1 + local exp_ret=$2 + local len=$3 + local unencoded_len=$4 + local unencoded_offset=$5 + local compression=$6 + local data_file=$7 + + local tmpfile=`mktemp` + + echo "running btrfs_encoded_write ioctl $fn 0 $len $unencoded_len $unencoded_offset $compression < $data_file > $tmpfile" >>$seqres.full + src/btrfs_encoded_write ioctl $fn 0 $len $unencoded_len $unencoded_offset $compression < $data_file > $tmpfile + + if [[ $? -ne 0 ]]; then + echo "btrfs_encoded_write failed" >>$seqres.full + rm $tmpfile + return 1 + fi + + exec {FD}< $tmpfile + + read -u ${FD} ret + + if [[ $ret -lt 0 ]]; then + if [[ $ret == -1 ]]; then + echo "btrfs encoded write failed with -EPERM; are you running as root?" >>$seqres.full + else + echo "btrfs encoded write failed (errno $ret)" >>$seqres.full + fi + exec {FD}<&- + return 1 + fi + + exec {FD}<&- + rm $tmpfile + + return 0 +} + +test_file() { + local size=$1 + local len=$2 + local unencoded_len=$3 + local unencoded_offset=$4 + local compression=$5 + + local tmpfile=`mktemp -p $SCRATCH_MNT` + local randfile=`mktemp` + + dd if=/dev/urandom of=$randfile bs=1 count=$size status=none + local md5=`md5sum $randfile | cut -d ' ' -f 1` + + do_encoded_write $tmpfile $size $len $unencoded_len $unencoded_offset \ + $compression $randfile || _fail "encoded write ioctl failed" + + rm $randfile + + do_encoded_read $tmpfile ioctl $size $len $unencoded_len \ + $unencoded_offset $compression $md5 || _fail "encoded read ioctl failed" + do_encoded_read $tmpfile io_uring $size $len $unencoded_len \ + $unencoded_offset $compression $md5 || _fail "encoded read io_uring failed" + + rm $tmpfile +} + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +sector_size=$(_scratch_btrfs_sectorsize) + +if [[ $sector_size -ne 4096 && $sector_size -ne 65536 ]]; then + _notrun "sector size $sector_size not supported by this test" +fi + +_scratch_mount "-o max_inline=2048" + +if [[ $sector_size -eq 4096 ]]; then + test_file 40960 97966 98304 0 1 # zlib + test_file 40960 97966 98304 0 2 # zstd + test_file 40960 97966 98304 0 3 # lzo 4k + test_file 40960 97966 110592 4096 1 # bookended zlib + test_file 40960 97966 110592 4096 2 # bookended zstd + test_file 40960 97966 110592 4096 3 # bookended lzo 4k +elif [[ $sector_size -eq 65536 ]]; then + test_file 65536 97966 131072 0 1 # zlib + test_file 65536 97966 131072 0 2 # zstd + test_file 65536 97966 131072 0 7 # lzo 64k + # can't test bookended extents on 64k, as max is only 2 sectors long +fi + +# btrfs won't create inline files unless PAGE_SIZE == sector size +if [[ "$(_get_page_size)" -eq $sector_size ]]; then + test_file 892 1931 1931 0 1 # inline zlib + test_file 892 1931 1931 0 2 # inline zstd + + if [[ $sector_size -eq 4096 ]]; then + test_file 892 1931 1931 0 3 # inline lzo 4k + elif [[ $sector_size -eq 65536 ]]; then + test_file 892 1931 1931 0 7 # inline lzo 64k + fi +fi + +_scratch_unmount + +echo Silence is golden +status=0 +exit diff --git a/tests/btrfs/333.out b/tests/btrfs/333.out new file mode 100644 index 00000000..60a15898 --- /dev/null +++ b/tests/btrfs/333.out @@ -0,0 +1,2 @@ +QA output created by 333 +Silence is golden