From patchwork Thu May 2 09:07:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 13651471 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5603547772 for ; Thu, 2 May 2024 09:08:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714640900; cv=none; b=iJ785Uk7fZbQJqhUHMIQkrEF/SOdkXBB75Na0bLR160760BBmwV2klOCy3mIVIMgHffUfKuwplboKJarFrAGRKlY8AfcOeZqG0l0nZB4/jcSS1ytbwiqG0ZdThMmOE5fYYlaRZgLMpWNvOOy6pqpNAkc9FdrJdhjhw4SU+yO1f0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714640900; c=relaxed/simple; bh=JqUSTZhLCyvKaz6onISwYB82AImKxk/JDVc2eK2nr8Q=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Aesq+HD96gOv0Aj/ws0DGyMxPbH29Rm8/oUyFzr4mkfMI4aB19xgTjKseXn7E0L2hWUwaxaHO7DCtqBdzXQ0bgg7OHgVb87yFXjtj4QuoyqxW3OFTZJfJZex4hFQurGxSKJXiKId454NgwrMtHBDxcjFcgNkr5xOXYujRHIzQ04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=g0n1ZWz1; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=g0n1ZWz1; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="g0n1ZWz1"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="g0n1ZWz1" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9981F1FBDB for ; Thu, 2 May 2024 09:08:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1714640896; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xEFjm3L55zz33ExDCw708NJKMIWxIU8WoFUm4qcscVk=; b=g0n1ZWz1E6WpcyGtg06rnExn7AbAIPKAft9zdDI2NHb22vxnmQAJFsmAcA3izs/wjeBE+X hwvFnSl/aQafvkGIArxY7dL+JfKG8P60Y04lXkFqOmglX7f16kmRGXI2jYrWmnfaO6eOgo HHj5jOQqDwLGJD4rHhoUb56KiJIUuaE= Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.com header.s=susede1 header.b=g0n1ZWz1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1714640896; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xEFjm3L55zz33ExDCw708NJKMIWxIU8WoFUm4qcscVk=; b=g0n1ZWz1E6WpcyGtg06rnExn7AbAIPKAft9zdDI2NHb22vxnmQAJFsmAcA3izs/wjeBE+X hwvFnSl/aQafvkGIArxY7dL+JfKG8P60Y04lXkFqOmglX7f16kmRGXI2jYrWmnfaO6eOgo HHj5jOQqDwLGJD4rHhoUb56KiJIUuaE= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id ADF941386E for ; Thu, 2 May 2024 09:08:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([10.150.64.162]) by imap1.dmz-prg2.suse.org with ESMTPSA id MHDNGf9XM2ZZcAAAD6G6ig (envelope-from ) for ; Thu, 02 May 2024 09:08:15 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 1/3] btrfs-progs: check/lowmem: detect and repair mismatched ram_bytes Date: Thu, 2 May 2024 18:37:53 +0930 Message-ID: X-Mailer: git-send-email 2.45.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Flag: NO X-Spam-Score: -5.01 X-Rspamd-Action: no action X-Rspamd-Queue-Id: 9981F1FBDB X-Spam-Level: X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spamd-Result: default: False [-5.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; DWL_DNSWL_MED(-2.00)[suse.com:dkim]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.com:s=susede1]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-btrfs@vger.kernel.org]; FROM_EQ_ENVFROM(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; TO_DN_NONE(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:dkim,suse.com:email]; RCVD_COUNT_TWO(0.00)[2]; DKIM_TRACE(0.00)[suse.com:+] For non-compressed non-hole file extent items, the ram_bytes should match disk_num_bytes. But due to kernel bugs, we have several cases where ram_bytes is not correctly updated. Thankfully this is really a very minor mismatch and can never cause data corruption since the kernel does not utilize ram_bytes for non-compressed extents at all. So here we just detect and repair it for lowmem mode. Signed-off-by: Qu Wenruo --- check/mode-lowmem.c | 69 +++++++++++++++++++++++++++++++++++++++++++++ check/mode-lowmem.h | 1 + 2 files changed, 70 insertions(+) diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c index fd9b975c4e5f..99e1305b1f3e 100644 --- a/check/mode-lowmem.c +++ b/check/mode-lowmem.c @@ -2081,6 +2081,61 @@ static int check_file_extent_inline(struct btrfs_root *root, return err; } +static int repair_ram_bytes_mismatch(struct btrfs_root *root, + struct btrfs_path *path) +{ + struct btrfs_trans_handle *trans; + struct btrfs_key key; + struct btrfs_file_extent_item *fi; + u64 disk_num_bytes; + int recover_ret; + int ret; + + btrfs_item_key_to_cpu(path->nodes[0], &key, path->slots[0]); + btrfs_release_path(path); + UASSERT(key.type == BTRFS_EXTENT_DATA_KEY); + + trans = btrfs_start_transaction(root, 1); + if (IS_ERR(trans)) { + ret = PTR_ERR(trans); + errno = -ret; + error_msg(ERROR_MSG_START_TRANS, "%m"); + return ret; + } + + ret = btrfs_search_slot(trans, root, &key, path, 0, 1); + /* Not really possible. */ + if (ret > 0) { + ret = -ENOENT; + btrfs_release_path(path); + goto recover; + } + + if (ret < 0) + goto recover; + + fi = btrfs_item_ptr(path->nodes[0], path->slots[0], + struct btrfs_file_extent_item); + disk_num_bytes = btrfs_file_extent_disk_num_bytes(path->nodes[0], fi); + btrfs_set_file_extent_ram_bytes(path->nodes[0], fi, disk_num_bytes); + btrfs_mark_buffer_dirty(path->nodes[0]); + + ret = btrfs_commit_transaction(trans, root); + if (ret < 0) { + errno = -ret; + error_msg(ERROR_MSG_COMMIT_TRANS, "%m"); + } else { + printf( + "Successfully repaired ram_bytes for non-compressed extent at root %llu ino %llu file_pos %llu\n", + root->objectid, key.objectid, key.offset); + } + return ret; +recover: + recover_ret = btrfs_search_slot(NULL, root, &key, path, 0, 0); + UASSERT(recover_ret == 0); + return ret; +} + /* * Check file extent datasum/hole, update the size of the file extents, * check and update the last offset of the file extent. @@ -2106,6 +2161,7 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, u64 csum_found; /* In byte size, sectorsize aligned */ u64 search_start; /* Logical range start we search for csum */ u64 search_len; /* Logical range len we search for csum */ + u64 ram_bytes; u64 gen; u64 super_gen; unsigned int extent_type; @@ -2140,6 +2196,7 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, extent_num_bytes = btrfs_file_extent_num_bytes(node, fi); extent_offset = btrfs_file_extent_offset(node, fi); compressed = btrfs_file_extent_compression(node, fi); + ram_bytes = btrfs_file_extent_ram_bytes(node, fi); is_hole = (disk_bytenr == 0) && (disk_num_bytes == 0); super_gen = btrfs_super_generation(gfs_info->super_copy); @@ -2150,6 +2207,18 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, err |= INVALID_GENERATION; } + if (!compressed && disk_bytenr && disk_num_bytes != ram_bytes) { + error( + "minor ram_bytes mismatch for non-compressed data extents, have %llu expect %llu", + ram_bytes, disk_num_bytes); + if (opt_check_repair) { + ret = repair_ram_bytes_mismatch(root, path); + if (ret < 0) + err |= RAM_BYTES_MISMATCH; + } else { + err |= RAM_BYTES_MISMATCH; + } + } /* * Check EXTENT_DATA csum * diff --git a/check/mode-lowmem.h b/check/mode-lowmem.h index b45e6bc137f3..b3e212165519 100644 --- a/check/mode-lowmem.h +++ b/check/mode-lowmem.h @@ -47,6 +47,7 @@ #define INODE_MODE_ERROR (1U << 25) /* Bad inode mode */ #define INVALID_GENERATION (1U << 26) /* Generation is too new */ #define SUPER_BYTES_USED_ERROR (1U << 27) /* Super bytes_used is invalid */ +#define RAM_BYTES_MISMATCH (1U << 27) /* Non-compressed extent has wrong ram_bytes */ /* * Error bit for low memory mode check. From patchwork Thu May 2 09:07:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 13651472 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD76B52F61 for ; Thu, 2 May 2024 09:08:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714640901; cv=none; b=Sv7q/6t3nqDVEMnALqNvADfJy60T2l6EUJNTBb9q5lgu5WnbZuLiBF9g55SzibyTWP531JCTok7w3Fdd+ZSmMGom0V6wgDpDRsPysw2h1epJRBCetA0HOVFfD1b5JTaegJg+2cjc7k78ucOhiLX0ScKLEf/6GJtdxWujnE/I2gw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714640901; c=relaxed/simple; bh=aXcxRuVh8qc7ojMDnBWTXZxiq82OWEuzeRoURwPvLp0=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LqxiYrcuNZ68qStQeofdLm4G1L+pSiNM0g8ZF2RFZz3tsX6T2ekXDuJl1MoeQ4M+oL+kLfIY1a+0S9LHr1FCgkKzTUgzJ4DVBqAYYJj1Sh0BI+kUfyCyGdbunBbKOSs099vVcuGRLO2sF6RSQ+rmBV4ta9DofgeGndgGfmsKicI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=EaBFbc6x; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=EaBFbc6x; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="EaBFbc6x"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="EaBFbc6x" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1BB851FBDD for ; Thu, 2 May 2024 09:08:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1714640898; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hfDX6d1qf7GRGl/E9RhfxyC6+CtW9MO/FDMqvXwxmv8=; b=EaBFbc6xpr2Kt6yVb9ktCQ3RthIkUl9IjvNBYYj0BZvijxPPNR5vqHY/OsLsIQmf8G5a7x jGNElBEaw9eAZO6tN5Jdo1dIhyZVfxhdH4bOaTDZI8bY8xTIBxRMSZo9W276aHDzjbk8NF IVIZeFI15c5wRx754zaRdZQFJ5VMCRU= Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1714640898; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hfDX6d1qf7GRGl/E9RhfxyC6+CtW9MO/FDMqvXwxmv8=; b=EaBFbc6xpr2Kt6yVb9ktCQ3RthIkUl9IjvNBYYj0BZvijxPPNR5vqHY/OsLsIQmf8G5a7x jGNElBEaw9eAZO6tN5Jdo1dIhyZVfxhdH4bOaTDZI8bY8xTIBxRMSZo9W276aHDzjbk8NF IVIZeFI15c5wRx754zaRdZQFJ5VMCRU= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 29E341386E for ; Thu, 2 May 2024 09:08:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([10.150.64.162]) by imap1.dmz-prg2.suse.org with ESMTPSA id WF6iMwBYM2ZZcAAAD6G6ig (envelope-from ) for ; Thu, 02 May 2024 09:08:16 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 2/3] btrfs-progs: check/original: detect and repair ram_bytes mismatch Date: Thu, 2 May 2024 18:37:54 +0930 Message-ID: <71ec5089099c57cddb6e004b8f79a476dd2874b4.1714640642.git.wqu@suse.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-btrfs@vger.kernel.org]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO For non-compressed non-hole file extent items, the ram_bytes should match disk_num_bytes. But due to kernel bugs, we have several cases where ram_bytes is not correctly updated. Thankfully this is really a very minor mismatch and can never cause data corruption since the kernel does not utilize ram_bytes for non-compressed extents at all. So here we just detect and repair it for original mode. Signed-off-by: Qu Wenruo --- check/main.c | 126 ++++++++++++++++++++++++++++++++++++++++-- check/mode-original.h | 8 +++ 2 files changed, 130 insertions(+), 4 deletions(-) diff --git a/check/main.c b/check/main.c index 93af325f2525..6c5f61fe1bb1 100644 --- a/check/main.c +++ b/check/main.c @@ -493,6 +493,33 @@ static int device_record_compare(const struct rb_node *node1, const struct rb_no return 0; } +static int add_mismatch_ram_bytes_record(struct inode_record *inode_rec, + struct btrfs_key *key) +{ + struct mismatch_ram_bytes_record *record; + + record = malloc(sizeof(*record)); + if (!record) { + error_msg(ERROR_MSG_MEMORY, "mismatch ram bytes record"); + return -ENOMEM; + } + memcpy(&record->key, key, sizeof(*key)); + list_add_tail(&record->list, &inode_rec->mismatch_ram_bytes); + return 0; +} + +static void free_mismatch_ram_bytes_records(struct inode_record *inode_rec) +{ + if (!list_empty(&inode_rec->mismatch_ram_bytes)) { + struct mismatch_ram_bytes_record *ram; + + ram = list_entry(inode_rec->mismatch_ram_bytes.next, + struct mismatch_ram_bytes_record, list); + list_del(&ram->list); + free(ram); + } +} + static struct inode_record *clone_inode_rec(struct inode_record *orig_rec) { struct inode_record *rec; @@ -501,6 +528,7 @@ static struct inode_record *clone_inode_rec(struct inode_record *orig_rec) struct inode_backref *tmp; struct mismatch_dir_hash_record *hash_record; struct mismatch_dir_hash_record *new_record; + struct mismatch_ram_bytes_record *ram_record; struct unaligned_extent_rec_t *src; struct unaligned_extent_rec_t *dst; struct rb_node *rb; @@ -514,6 +542,7 @@ static struct inode_record *clone_inode_rec(struct inode_record *orig_rec) rec->refs = 1; INIT_LIST_HEAD(&rec->backrefs); INIT_LIST_HEAD(&rec->mismatch_dir_hash); + INIT_LIST_HEAD(&rec->mismatch_ram_bytes); INIT_LIST_HEAD(&rec->unaligned_extent_recs); rec->holes = RB_ROOT; @@ -537,6 +566,11 @@ static struct inode_record *clone_inode_rec(struct inode_record *orig_rec) memcpy(&new_record, hash_record, size); list_add_tail(&new_record->list, &rec->mismatch_dir_hash); } + list_for_each_entry(ram_record, &orig_rec->mismatch_ram_bytes, list) { + ret = add_mismatch_ram_bytes_record(rec, &ram_record->key); + if (ret < 0) + goto cleanup; + } list_for_each_entry(src, &orig_rec->unaligned_extent_recs, list) { size = sizeof(*src); dst = malloc(size); @@ -578,6 +612,7 @@ cleanup: free(hash_record); } } + free_mismatch_ram_bytes_records(rec); if (!list_empty(&rec->unaligned_extent_recs)) list_for_each_entry_safe(src, dst, &rec->unaligned_extent_recs, list) { @@ -619,6 +654,8 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec) fprintf(stderr, ", odd file extent"); if (errors & I_ERR_BAD_FILE_EXTENT) fprintf(stderr, ", bad file extent"); + if (errors & I_ERR_RAM_BYTES_MISMATCH) + fprintf(stderr, ", bad ram bytes for non-compressed extents"); if (errors & I_ERR_FILE_EXTENT_OVERLAP) fprintf(stderr, ", file extent overlap"); if (errors & I_ERR_FILE_EXTENT_TOO_LARGE) @@ -637,8 +674,6 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec) fprintf(stderr, ", link count wrong"); if (errors & I_ERR_ODD_INODE_FLAGS) fprintf(stderr, ", odd inode flags"); - if (errors & I_ERR_INLINE_RAM_BYTES_WRONG) - fprintf(stderr, ", invalid inline ram bytes"); if (errors & I_ERR_INVALID_IMODE) fprintf(stderr, ", invalid inode mode bit 0%o", rec->imode & ~07777); @@ -699,6 +734,17 @@ static void print_inode_error(struct btrfs_root *root, struct inode_record *rec) hash_record->key.offset); } } + if (errors & I_ERR_RAM_BYTES_MISMATCH) { + struct mismatch_ram_bytes_record *ram_record; + + fprintf(stderr, + "Non-compressed file extents with invalid ram_bytes (minor errors):\n"); + list_for_each_entry(ram_record, &rec->mismatch_ram_bytes, list) { + fprintf(stderr, "\tino=%llu offset=%llu\n", + ram_record->key.objectid, + ram_record->key.offset); + } + } } static void print_ref_error(int errors) @@ -760,6 +806,7 @@ static struct inode_record *get_inode_rec(struct cache_tree *inode_cache, rec->refs = 1; INIT_LIST_HEAD(&rec->backrefs); INIT_LIST_HEAD(&rec->mismatch_dir_hash); + INIT_LIST_HEAD(&rec->mismatch_ram_bytes); INIT_LIST_HEAD(&rec->unaligned_extent_recs); rec->holes = RB_ROOT; @@ -811,6 +858,14 @@ static void free_inode_rec(struct inode_record *rec) list_del(&backref->list); free(backref); } + while (!list_empty(&rec->mismatch_ram_bytes)) { + struct mismatch_ram_bytes_record *ram; + + ram = list_entry(rec->mismatch_ram_bytes.next, + struct mismatch_ram_bytes_record, list); + list_del(&ram->list); + free(ram); + } list_for_each_entry_safe(hash, next, &rec->mismatch_dir_hash, list) free(hash); free_unaligned_extent_recs(&rec->unaligned_extent_recs); @@ -821,7 +876,8 @@ static void free_inode_rec(struct inode_record *rec) static bool can_free_inode_rec(struct inode_record *rec) { if (!rec->errors && rec->checked && rec->found_inode_item && - rec->nlink == rec->found_link && list_empty(&rec->backrefs)) + rec->nlink == rec->found_link && list_empty(&rec->backrefs) && + list_empty(&rec->mismatch_ram_bytes)) return true; return false; } @@ -1742,6 +1798,14 @@ static int process_file_extent(struct btrfs_root *root, rec->errors |= I_ERR_BAD_FILE_EXTENT; if (compression && rec->nodatasum) rec->errors |= I_ERR_BAD_FILE_EXTENT; + if (disk_bytenr && !compression && + btrfs_file_extent_ram_bytes(eb, fi) != + btrfs_file_extent_disk_num_bytes(eb, fi)) { + rec->errors |= I_ERR_RAM_BYTES_MISMATCH; + ret = add_mismatch_ram_bytes_record(rec, key); + if (ret < 0) + return ret; + } if (disk_bytenr > 0) rec->found_size += num_bytes; } else { @@ -3044,6 +3108,57 @@ static int repair_inode_gen_original(struct btrfs_trans_handle *trans, return 0; } +static int repair_ram_bytes(struct btrfs_trans_handle *trans, + struct btrfs_root *root, + struct btrfs_path *path, + struct inode_record *rec) +{ + struct mismatch_ram_bytes_record *record; + struct mismatch_ram_bytes_record *tmp; + int ret = 0; + + btrfs_release_path(path); + list_for_each_entry_safe(record, tmp, &rec->mismatch_ram_bytes, list) { + struct btrfs_file_extent_item *fi; + struct extent_buffer *leaf; + int type; + int slot; + int search_ret; + + search_ret = btrfs_search_slot(trans, root, &record->key, path, 0, 1); + if (search_ret > 0) + search_ret = -ENOENT; + if (search_ret < 0) { + ret = search_ret; + btrfs_release_path(path); + continue; + } + leaf = path->nodes[0]; + slot = path->slots[0]; + fi = btrfs_item_ptr(leaf, slot, struct btrfs_file_extent_item); + type = btrfs_file_extent_type(leaf, fi); + if (type != BTRFS_FILE_EXTENT_REG && + type != BTRFS_FILE_EXTENT_PREALLOC) { + ret = -EUCLEAN; + btrfs_release_path(path); + continue; + } + if (btrfs_file_extent_disk_bytenr(path->nodes[0], fi) == 0 || + btrfs_file_extent_compression(path->nodes[0], fi)) { + ret = -EUCLEAN; + btrfs_release_path(path); + continue; + } + btrfs_set_file_extent_ram_bytes(leaf, fi, + btrfs_file_extent_disk_num_bytes(leaf, fi)); + btrfs_mark_buffer_dirty(leaf); + btrfs_release_path(path); + } + if (!ret) + rec->errors &= ~I_ERR_RAM_BYTES_MISMATCH; + return ret; +} + static int try_repair_inode(struct btrfs_root *root, struct inode_record *rec) { struct btrfs_trans_handle *trans; @@ -3066,7 +3181,8 @@ static int try_repair_inode(struct btrfs_root *root, struct inode_record *rec) I_ERR_MISMATCH_DIR_HASH | I_ERR_UNALIGNED_EXTENT_REC | I_ERR_INVALID_IMODE | - I_ERR_INVALID_GEN))) + I_ERR_INVALID_GEN | + I_ERR_RAM_BYTES_MISMATCH))) return rec->errors; /* @@ -3106,6 +3222,8 @@ static int try_repair_inode(struct btrfs_root *root, struct inode_record *rec) ret = repair_unaligned_extent_recs(trans, root, &path, rec); if (!ret && rec->errors & I_ERR_INVALID_GEN) ret = repair_inode_gen_original(trans, root, &path, rec); + if (!ret && rec->errors & I_ERR_RAM_BYTES_MISMATCH) + ret = repair_ram_bytes(trans, root, &path, rec); btrfs_release_path(&path); ret = btrfs_commit_transaction(trans, root); if (ret < 0) { diff --git a/check/mode-original.h b/check/mode-original.h index ac8de57cc5d4..fbc6c2e5bd0f 100644 --- a/check/mode-original.h +++ b/check/mode-original.h @@ -189,6 +189,8 @@ struct unaligned_extent_rec_t { #define I_ERR_INVALID_GEN (1U << 20) #define I_ERR_INVALID_NLINK (1U << 21) #define I_ERR_INVALID_XATTR (1U << 22) +/* Ram_bytes mismatch for non-compressed data extents. */ +#define I_ERR_RAM_BYTES_MISMATCH (1U << 23) struct inode_record { struct list_head backrefs; @@ -216,6 +218,7 @@ struct inode_record { u64 extent_end; struct rb_root holes; struct list_head mismatch_dir_hash; + struct list_head mismatch_ram_bytes; u32 refs; }; @@ -232,6 +235,11 @@ struct mismatch_dir_hash_record { /* namebuf follows here */ }; +struct mismatch_ram_bytes_record { + struct list_head list; + struct btrfs_key key; +}; + struct root_backref { struct list_head list; unsigned int found_dir_item:1; From patchwork Thu May 2 09:07:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 13651473 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4118535BF for ; Thu, 2 May 2024 09:08:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714640904; cv=none; b=L7vXoI/U2aadlOAU22yYV8U+WtWRCJ8GYgtGkZFU7ReW2fIdBPE+Bo0swaf0F2Wp4Mj45jKtJlfPqQmEiXSJ+DW5GebWV2KmjL7GcaTFAkd84pqjWq3nBEbfhwNI9iW3XFK7PtpbJ/gK36CchgfDwT/Bmt14AtPIzKg6yzhPdWs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714640904; c=relaxed/simple; bh=YNxl+LD4rysdtMn350m04o3CtftIBeZ2LhHYg1VL9s8=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T1Y+PwNF87RrheKBW7g9TbMYXBYeSvN9tdiaJe0ojKoTDdzQh47dvLt59Q26cfpLNjGbXZNS+yzBV+2U3b/lDSc2zli/nP9y+3ObF6yhg7AzimxryWq5+JIgC4JnuA/AoHsRUzRyw41f1ODeFEv2TXivbbMYTfERE3i90L3pD6U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com; spf=pass smtp.mailfrom=suse.com; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=esy/Z0G5; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b=CQ6KFfIJ; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=suse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="esy/Z0G5"; dkim=pass (1024-bit key) header.d=suse.com header.i=@suse.com header.b="CQ6KFfIJ" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9FB091FBBF for ; Thu, 2 May 2024 09:08:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1714640900; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9GIq8464ohgwLlwW8ZpjOaHMUoqQNOB9WQtDtllgvqw=; b=esy/Z0G5pOHdOs1pbSlVp4qNGD9QQp2/+8WLg+APWSBFM+gv0ARMI5w6UtHP1JNRG2lxub 6s4CFKPeWDL9t1SmA7EippjxxHXAhZCWO6J4+HTI3+ZbxxzxypWDm/g5BNA5q1iebMcZED eR6+XPvDYsREx9sRtAvfP/surJPy9e8= Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1714640899; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9GIq8464ohgwLlwW8ZpjOaHMUoqQNOB9WQtDtllgvqw=; b=CQ6KFfIJ4J0VljD/6fdgEJSyuID9xsf+hhkp4X0qlVdNL4B6LCtv6XTWIt6UcwakHB2B0C c8Gl6aK159pHKvnofFyGo4vAu/v+D3HHPSRfqnDLqybGkTe6/fenVzhoJKn/XoA3ake5G1 gUKadNLWjlOzlQTag8A2PYkQPtqHDqE= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id ACE4C1386E for ; Thu, 2 May 2024 09:08:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([10.150.64.162]) by imap1.dmz-prg2.suse.org with ESMTPSA id yOr4FwJYM2ZZcAAAD6G6ig (envelope-from ) for ; Thu, 02 May 2024 09:08:18 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 3/3] btrfs-progs: tests/fsck: add test case for ram_bytes detection and repair Date: Thu, 2 May 2024 18:37:55 +0930 Message-ID: <06aecdc98f472fad3f1d3a5558fe6985561191dc.1714640642.git.wqu@suse.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-btrfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; DKIM_SIGNED(0.00)[suse.com:s=susede1]; DBL_BLOCKED_OPENRESOLVER(0.00)[4var:email,ckdt:email,8z:email,acrg:email,s:email,ywipn:email,ysv:email,yfpj:email,gii3gazzn1mrq2fyz2owm:email,dn:email,bwv:email,ayekn:email,74cc:email,o:email,c:email,suse.com:email]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-btrfs@vger.kernel.org]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO The new test case would contain a file system image, with the following file: item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160 generation 7 transid 8 size 16384 nbytes 16384 block group 0 mode 100600 links 1 uid 0 gid 0 rdev 0 sequence 258 flags 0x0(none) atime 1714635006.328482575 (2024-05-02 17:00:06) ctime 1714635013.394980640 (2024-05-02 17:00:13) mtime 1714635013.394980640 (2024-05-02 17:00:13) otime 1714635006.328482575 (2024-05-02 17:00:06) item 5 key (257 INODE_REF 256) itemoff 15869 itemsize 14 index 2 namelen 4 name: file item 6 key (257 EXTENT_DATA 0) itemoff 15816 itemsize 53 generation 7 type 1 (regular) extent data disk byte 13631488 nr 1048576 extent data offset 0 nr 16384 ram 16384 extent compression 0 (none) Note the ram bytes, which should be 1048576. Furthermore, the inode size is trucnated to 16K (originally 1M), so that offset + num_bytes would still be no larger than ram_bytes. So the only error is the mismatch between ram_bytes and disk_num_bytes for the non-compressed data extent. The image is hand crafted for now, as btrfs-corrupt-block doesn not yet support corrupting the ram_bytes field. Signed-off-by: Qu Wenruo --- .../default.img.xz | Bin 0 -> 2076 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 tests/fsck-tests/062-noncompressed-ram-bytes-mismatch/default.img.xz GIT binary patch literal 2076 zcmV+%2;=wtH+ooF000E$*0e?hz}pXiARz%3000000001m;;ozD5B~?#T>wRyj;C3^ zv%$$4d1r37hBKh;&NC{9Vlybk2gs>;O=+*j*Fys=1WO!J9Q0^Gqiqs0EHbPio`0K_ z(aS^?hy(XE6kon2MI{3*trchQh3!0HI=~or6w^ zfAFP4!p4~xq_|ly!>RmQ(l-r8$wr4N)u z=C6FiY@YFpj>TOYv+8{cp!|mbO$?Ps>X*;9dRPngyyYo2^@1@Dn)r8$2!AQv#M7=r zN<$6lUR@G%ii3gaZZn1mRq2FYZ2owm?oR|zDH(R6!h#g7p-#>_rwtpxYAP=rv$hxtn$nddZ;7JegkSHv}@B}ep%?U7V^1YGCD#>q{% z{A-225;3In*f_g_j3lgvd6dT}uv(v0mvt!F*urq8>a%VPle~RBuG~Edim#HSOcFC^ z<7Ud{DTd=!;}=s*^{Wj2s-MY6k{!zuZfHv!JfoelTDwy~qAMbx*2G}uAw}V_{nW`& z0?WxQ^IC&;tedpOI%X2+ zB3aqG0#J@@5GfAjJ=8rBD5#yeYzW`Rs{VyspYWN`1IB-=s;dk*UqbgphLKt;aW`%g zKOlytm>*;m*U?YQ`>f)4rO^sj9BzPYRLMT94zl_NZC%EZvHa^p`8QJ?noEsLf9swE z-~(0$6_XDW9WJa@CkDt`LJ^S^JZjGA2f-_D^kdc_gmWpk;`g9wZeG3DlC8W2=fx5_ z_;40A;0X^K?qvyWHs*h-MfHCkBDH>>N+54mx3wUxI+Q3bh~X?qF5ITsXC83M&$#w!|m6WF{(5&2*} zE*`G6@C~Wxj96`Rm;)wOh?90N2PeR7!=DCo;ti&{0R5|In$M4ME_t_RJ+{>*vkPXL z7p{MdJSe!3r=z)4;3K&%^`PmW++M%LN1($?&-V2EKRQGLRm8oVMX;?FxXdCg4)QRY2}IA1^e3cNZU?`rO%JKtly^ibzm5#W2(3y$kKJj zoKn_hN{a36JS_SElIG^P z8&a7ll`6qM>yN(E$vf~BI>D2iaw=n(fzrrXzbuddI4w`fZ$NSqECVP3o5=?aN5jKP{B$^UXjKF6~*kOdK7 zYcU-uw_Td}UfVFT)`dp+^ z4syA;n9v)jF0FTF@YsV+97@acrg;A@74cc^)ZZb4a&xzL zpV!7~I`tI%$Roq5CGqHGaa}e;AW)<#je4JGa}Jl z^LG(z`gLbRZ49PL9&fo$XxhPnbVKEglmlFBN>0{BsbC%S^8U=kr!sQz)f6?v<%X-haNG7j;SHJEMy9oRq`fRwQYx_U$cbB|sa)|Z z^<)^VOV2LPyOoIpXR@#wJ(e z%O{ELRDzEXYr54==CoJ!I~wAD00000nsdux%pJ=B0rwApAOHY%nueUQ#Ao{g00000 G1X)@Hrty6M literal 0 HcmV?d00001 diff --git a/tests/fsck-tests/062-noncompressed-ram-bytes-mismatch/default.img.xz b/tests/fsck-tests/062-noncompressed-ram-bytes-mismatch/default.img.xz new file mode 100644 index 0000000000000000000000000000000000000000..94a587a83e64a9d1981d8b74e313b4ff7e497c68