From patchwork Thu Jul 18 22:10:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 13736762 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28F58145FFC for ; Thu, 18 Jul 2024 22:11:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721340674; cv=none; b=NlaA91FBMcJM+pOLxrtKI9P3WWm4Vj4TK9QxUREqJghQKnbiMtTf1kxKSNMaMcL33UCoK8dCYjKnUZ9tVf0rTzCCQcBFf6NmSbD4FhPEWFtIR1jeSArzc6XX+CAVk4+NA3Dxb+CWIY382y/miNJilLHTYqqMAZhNwM6GH+o3kKc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721340674; c=relaxed/simple; bh=LWvNVaGthbYvI4yj3ghqWqZu/vYWrQ3wpV+A61DssWI=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W2cQlZ65gV5yD3UbsskfpGlb6UF8GbXHOibTE88XsWsStxzdUU9oamRiB5eHNSjSZZrLoJ39OkJT1O8W5xzqkwcw2Tmt7+tb3FiBbUdIpeBcLNmqxQAc8ida8TNhEd922VNjRPnt/4XwGpz9n96/q8Y/HOX4cW/rqYkZWtX1Kmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=dRq4ho/A; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=dRq4ho/A; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="dRq4ho/A"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="dRq4ho/A" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0A744216EA for ; Thu, 18 Jul 2024 22:11:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1721340670; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OTpcAHvNRgz7hHOHyV971kpgK9P/XxuDK7599zOP8bk=; b=dRq4ho/AlvVN+nvc4oCHIFpIc73c0GV00iv3TXa8ffq1iKcAuqFm0NrxZzApsOriQw7c54 liauDaAZBHsz50ThtFAHlD+AkihPiGA1u6guEY+5rPNUKHnO5C2BMOWiAxFtOo0QUVnNiE OAgX4QMSwSIvUuf2nIF/XmXc/ipGcok= Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.com header.s=susede1 header.b="dRq4ho/A" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1721340670; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OTpcAHvNRgz7hHOHyV971kpgK9P/XxuDK7599zOP8bk=; b=dRq4ho/AlvVN+nvc4oCHIFpIc73c0GV00iv3TXa8ffq1iKcAuqFm0NrxZzApsOriQw7c54 liauDaAZBHsz50ThtFAHlD+AkihPiGA1u6guEY+5rPNUKHnO5C2BMOWiAxFtOo0QUVnNiE OAgX4QMSwSIvUuf2nIF/XmXc/ipGcok= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2FD7B137EB for ; Thu, 18 Jul 2024 22:11:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6KjTNvySmWZZFwAAD6G6ig (envelope-from ) for ; Thu, 18 Jul 2024 22:11:08 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH v3 1/3] btrfs-progs: csum-change: add leaf based threshold Date: Fri, 19 Jul 2024 07:40:43 +0930 Message-ID: <7e8e75f50bb6b8d44a50bacfe803f7790e6a2a70.1721340621.git.wqu@suse.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Rspamd-Queue-Id: 0A744216EA X-Spam-Flag: NO X-Spam-Score: 0.99 X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [0.99 / 50.00]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.com:s=susede1]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim,suse.com:email]; RCVD_COUNT_TWO(0.00)[2]; PREVIOUSLY_DELIVERED(0.00)[linux-btrfs@vger.kernel.org]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.com:+] X-Spam-Level: X-Spamd-Bar: / For "btrfstune --csum", currently we do the following operations in just one transaction for each: - Delete old data csums - Change new data csums objectid Both opeartion can modify up to GiB or even TiB level of metadata, doing them in just one transaction is definitely going to cause problems. This patch adds a leaf number based threshold (32 leaves), after modifying/deleteing this many leaves, we commit a transaction to avoid huge amount of dirty leaves piling up. Signed-off-by: Qu Wenruo --- tune/change-csum.c | 68 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 52 insertions(+), 16 deletions(-) diff --git a/tune/change-csum.c b/tune/change-csum.c index 0f95cdb25533..66fdd207335c 100644 --- a/tune/change-csum.c +++ b/tune/change-csum.c @@ -371,29 +371,48 @@ static int generate_new_data_csums(struct btrfs_fs_info *fs_info, u16 new_csum_t return generate_new_data_csums_range(fs_info, 0, new_csum_type); } +/* After deleting/modifying this many leaves, commit a transaction. */ +#define CSUM_CHANGE_LEAVES_THRESHOLD 32 + static int delete_old_data_csums(struct btrfs_fs_info *fs_info) { struct btrfs_root *csum_root = btrfs_csum_root(fs_info, 0); - struct btrfs_trans_handle *trans; + struct btrfs_trans_handle *trans = 0; struct btrfs_path path = { 0 }; struct btrfs_key last_key; + unsigned int deleted_leaves = 0; int ret; last_key.objectid = BTRFS_EXTENT_CSUM_OBJECTID; last_key.type = BTRFS_EXTENT_CSUM_KEY; last_key.offset = (u64)-1; - trans = btrfs_start_transaction(csum_root, 1); - if (IS_ERR(trans)) { - ret = PTR_ERR(trans); - errno = -ret; - error("failed to start transaction to delete old data csums: %m"); - return ret; - } while (true) { int start_slot; int nr; + if (deleted_leaves >= CSUM_CHANGE_LEAVES_THRESHOLD) { + assert(trans); + ret = btrfs_commit_transaction(trans, csum_root); + if (ret < 0) { + errno = -ret; + error( + "failed to commit transaction to delete old data csums: %m"); + return ret; + } + trans = NULL; + deleted_leaves = 0; + } + if (!trans) { + trans = btrfs_start_transaction(csum_root, 1); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + errno = -ret; + error( + "failed to start transaction to delete old data csums: %m"); + return ret; + } + } ret = btrfs_search_slot(trans, csum_root, &last_key, &path, -1, 1); if (ret < 0) { errno = -ret; @@ -428,6 +447,7 @@ static int delete_old_data_csums(struct btrfs_fs_info *fs_info) break; } btrfs_release_path(&path); + deleted_leaves++; } btrfs_release_path(&path); if (ret < 0) @@ -445,9 +465,10 @@ static int delete_old_data_csums(struct btrfs_fs_info *fs_info) static int change_csum_objectids(struct btrfs_fs_info *fs_info) { struct btrfs_root *csum_root = btrfs_csum_root(fs_info, 0); - struct btrfs_trans_handle *trans; + struct btrfs_trans_handle *trans = NULL; struct btrfs_path path = { 0 }; struct btrfs_key last_key; + unsigned int changed_leaves = 0; u64 super_flags; int ret = 0; @@ -455,17 +476,32 @@ static int change_csum_objectids(struct btrfs_fs_info *fs_info) last_key.type = BTRFS_EXTENT_CSUM_KEY; last_key.offset = (u64)-1; - trans = btrfs_start_transaction(csum_root, 1); - if (IS_ERR(trans)) { - ret = PTR_ERR(trans); - errno = -ret; - error("failed to start transaction to change csum objectids: %m"); - return ret; - } while (true) { struct btrfs_key found_key; int nr; + if (changed_leaves >= CSUM_CHANGE_LEAVES_THRESHOLD) { + assert(trans); + ret = btrfs_commit_transaction(trans, csum_root); + if (ret < 0) { + errno = -ret; + error( + "failed to commit transaction to change data csum objectid: %m"); + return ret; + } + trans = NULL; + changed_leaves = 0; + } + if (!trans) { + trans = btrfs_start_transaction(csum_root, 1); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + errno = -ret; + error( + "failed to start transaction to change csum objectids: %m"); + return ret; + } + } ret = btrfs_search_slot(trans, csum_root, &last_key, &path, 0, 1); if (ret < 0) goto out; From patchwork Thu Jul 18 22:10:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 13736763 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 520D1145FFF for ; Thu, 18 Jul 2024 22:11:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721340675; cv=none; b=Rh+1M+6wvVssGz0vmcJaHSF671W6U0JvBTKQQ1N0lJe+zzdfP4xc0hL1LL/TsS3aabwfIcjZ6rjHMLuGJa5P2pzo/tVbGgGWLFMcG+R+7Q0x9Kz3q4Ujnl0P6m91lrEXRmA30WdlZrTJ7dpu8kIIYVxhxSC4iRPRJKuaJSWbNh4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721340675; c=relaxed/simple; bh=1Zz7eL23TR183MsHYUz24bI1nAG//d5VOFzMp/BmWjE=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q01H4RQRxDchFXdU1D57EltPqXBJ64kB8TphYcCNFv3lXLMZBX9fJKTmih9hOhqEXYYieuf8GgbyWgGyJY7M6MRw0PEUgN0ibbcGlTNHmKEqaX6+7odXiQGe/Mu/vtW/4Y2yTr86h5RQxagDZzEDPs2+4yem0CPDMCbZ+9icGxE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=pMj34eK/; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=pMj34eK/; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="pMj34eK/"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="pMj34eK/" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 68D1B21AC5 for ; Thu, 18 Jul 2024 22:11:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1721340671; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8hID4vMZY0qpx30RjnFp/D8y/1d+EJEWMFKqhKYPk4Q=; b=pMj34eK/qeXjssXrt9rBusHhHxsvoCW+FUkiInDpzQk428Vi0nyMqKu5+yHtxTJptTXadT W4mwopTtvFZbXi7zqa1tGwWXMEt/JZ/iuxbq39pbL7+3i4gZTsUCwJxaIC8a5M5Rlb4BUL U028d0y7QAZIVTW4LQ4pJiGWnKinY5A= Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1721340671; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8hID4vMZY0qpx30RjnFp/D8y/1d+EJEWMFKqhKYPk4Q=; b=pMj34eK/qeXjssXrt9rBusHhHxsvoCW+FUkiInDpzQk428Vi0nyMqKu5+yHtxTJptTXadT W4mwopTtvFZbXi7zqa1tGwWXMEt/JZ/iuxbq39pbL7+3i4gZTsUCwJxaIC8a5M5Rlb4BUL U028d0y7QAZIVTW4LQ4pJiGWnKinY5A= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8DCDF1379D for ; Thu, 18 Jul 2024 22:11:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YOkmEv6SmWZZFwAAD6G6ig (envelope-from ) for ; Thu, 18 Jul 2024 22:11:10 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH v3 2/3] btrfs-progs: tests: use feature output from "btrfs --version" Date: Fri, 19 Jul 2024 07:40:44 +0930 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Flag: NO X-Spam-Score: 1.20 X-Spamd-Result: default: False [1.20 / 50.00]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.997]; MIME_GOOD(-0.10)[text/plain]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; PREVIOUSLY_DELIVERED(0.00)[linux-btrfs@vger.kernel.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.com:email]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Spam-Level: * With the new feature matrix output in "btrfs --version" there is no need to do the config.h hack to determine if we have certain feature. This would provide a more reliable way to detect features. Signed-off-by: Qu Wenruo --- tests/common | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/tests/common b/tests/common index e996b35af787..4800ef8e77c9 100644 --- a/tests/common +++ b/tests/common @@ -12,11 +12,7 @@ _test_config() { local feature="$1" - if [ ! -f "$TOP/include/config.h" ]; then - echo "include/config.h not exists" - exit 1 - fi - if grep -q "$feature.*1" "${TOP}/include/config.h"; then + if "$TOP/btrfs" --version | grep -q "+${feature}"; then return 0 fi return 1 From patchwork Thu Jul 18 22:10:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 13736764 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 827CE146016 for ; Thu, 18 Jul 2024 22:11:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721340676; cv=none; b=IHJAegJSm6lYjpgVeV6LMa1StHFgQAHgNIFl9tPzGw9cHpnhxC8Ty+UCUvxgrSB5mZw3n/1tIgAhfhoOxnyIzAZZqdxIEYcvtuGaPYWJOvFX8E+G1C170DETLQO8kt4rQSUvS6Chi9AOAAD5YHOFhKRIDn6B6K+vNKdbxFSpJgY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721340676; c=relaxed/simple; bh=YMNwuKHxrWOZIrVrNc73wXjzmTBEn2MSb89DH7ekUHM=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aWVzzc5WcoxzSx7eYi1VyXE+u1mcNyBocUa5NddvvZWMQeX4Bv0Rfqia55ouwh61I1xY1iI1rO4mWF5oAIN+2z97+1tFECm1kc+fG9u7FsAbTRhPpoNyW5t0nd5vzUh6mkqUFLlQ+yqD3YqhMOD5LxdbSxk/Si1Z5SXAdNKbyyE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=Sx/yAe0c; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=Sx/yAe0c; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="Sx/yAe0c"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="Sx/yAe0c" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C697721AF5 for ; Thu, 18 Jul 2024 22:11:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1721340672; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3hOOFTf0JQg/6KOxdrdzzjfFshogkW9/gE4fpxqBFcM=; b=Sx/yAe0cZSpoMCk57gj4WRiHe7kkS8JA7guiMtLOsrOFgtAIdTdR1gX04jChHORvZU2IRx zYbbOqnlCHOGrkXDtYw3+V/JHS3R8A00Z0fu+jhLf3guceV+ZNGGLHlBxj/MCj7aKwhW9y Jz3GxL9NsRzOYOh9ro6p2hklvcEJ1kQ= Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.com header.s=susede1 header.b="Sx/yAe0c" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1721340672; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3hOOFTf0JQg/6KOxdrdzzjfFshogkW9/gE4fpxqBFcM=; b=Sx/yAe0cZSpoMCk57gj4WRiHe7kkS8JA7guiMtLOsrOFgtAIdTdR1gX04jChHORvZU2IRx zYbbOqnlCHOGrkXDtYw3+V/JHS3R8A00Z0fu+jhLf3guceV+ZNGGLHlBxj/MCj7aKwhW9y Jz3GxL9NsRzOYOh9ro6p2hklvcEJ1kQ= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EB49B1379D for ; Thu, 18 Jul 2024 22:11:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id wAr9KP+SmWZZFwAAD6G6ig (envelope-from ) for ; Thu, 18 Jul 2024 22:11:11 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH v3 3/3] btrfs-progs: misc-tests: add a basic resume test using error injection Date: Fri, 19 Jul 2024 07:40:45 +0930 Message-ID: <4556c3b266244552477186a27fb4b4a0c40c0310.1721340621.git.wqu@suse.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Rspamd-Queue-Id: C697721AF5 X-Spam-Flag: NO X-Spam-Score: 0.99 X-Rspamd-Action: no action X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [0.99 / 50.00]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.com:s=susede1]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; FUZZY_BLOCKED(0.00)[rspamd.com]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.com:dkim,suse.com:email]; RCVD_COUNT_TWO(0.00)[2]; PREVIOUSLY_DELIVERED(0.00)[linux-btrfs@vger.kernel.org]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.com:+] X-Spam-Level: X-Spamd-Bar: / The new test case does: - Make sure the build has error injection support This is done by checking "btrfs --version" output. - Inject error at the last commit transaction of new data csum generation - Resume the csum conversion and make sure it works Signed-off-by: Qu Wenruo --- tests/common | 7 +++ .../065-csum-conversion-inject/test.sh | 45 +++++++++++++++++++ 2 files changed, 52 insertions(+) create mode 100755 tests/misc-tests/065-csum-conversion-inject/test.sh diff --git a/tests/common b/tests/common index 4800ef8e77c9..f6b68a418087 100644 --- a/tests/common +++ b/tests/common @@ -402,6 +402,13 @@ check_regular_build() fi } +check_injection() +{ + if ! _test_config "INJECT"; then + _not_run "This test requires error injection support (make D=1)" + fi +} + check_prereq() { # Internal tools for testing, not shipped with the package diff --git a/tests/misc-tests/065-csum-conversion-inject/test.sh b/tests/misc-tests/065-csum-conversion-inject/test.sh new file mode 100755 index 000000000000..715349c4d403 --- /dev/null +++ b/tests/misc-tests/065-csum-conversion-inject/test.sh @@ -0,0 +1,45 @@ +#!/bin/bash +# Verify the csum conversion can still resume after an interruption + +source "$TEST_TOP/common" || exit +source "$TEST_TOP/common.convert" || exit + +check_experimental_build +check_injection +setup_root_helper +prepare_test_dev + +test_resume_data_csum_generation() +{ + local new_csum="$1" + local tmp=$(_mktemp "csum-convert") + + # Error at the end of the data csum generation. + export INJECT="0x4de02239" + run_mustfail_stdout "error injection not working" \ + "$TOP/btrfstune" --csum "$new_csum" "$TEST_DEV" &> $tmp + cat "$tmp" >> "$RESULTS" + if ! grep -q "$INJECT" "$tmp"; then + rm -f -- "$tmp" + _fail "csum conversion failed to unexpected reason" + fi + rm -f -- "$tmp" + unset INJECT + run_check "$TOP/btrfstune" --csum "$new_csum" "$TEST_DEV" + run_check "$TOP/btrfs" check --check-data-csum "$TEST_DEV" +} + +check_injection + +run_check_mkfs_test_dev --csum crc32c + +# We only mount the filesystem once to populate its contents, later one we +# would never mount the fs (to reduce the dependency on kernel features). +run_check_mount_test_dev +populate_fs +run_check_umount_test_dev + +test_resume_data_csum_generation xxhash +test_resume_data_csum_generation blake2 +test_resume_data_csum_generation sha256 +test_resume_data_csum_generation crc32c