From patchwork Mon Apr 20 17:07:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499471 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B6076CA for ; Mon, 20 Apr 2020 17:09:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 63DF0208E4 for ; Mon, 20 Apr 2020 17:09:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402540; bh=tPDZwM/4wQHICe/qYNofhfEPheexviXoxm4aIF+HkMM=; h=From:To:Cc:Subject:Date:List-ID:From; b=MCgqrakB+nOuHJzjgeCQyFs2KyWXe9TAoom6iN1Yc35hxq+XEvKDXMKtLEudphJfq lklgYsa6Zw9fY47Mw2pCbK/RybG81wyLqa/VsFOOm/9pfxBELYlyKuOKbMKWuz/NtV 1xGQ5Cj5KfWlkCZU/9QyFRC2/YfU0/HBn8hiR2o4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726373AbgDTRJA (ORCPT ); Mon, 20 Apr 2020 13:09:00 -0400 Received: from mail.kernel.org ([198.145.29.99]:42654 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725784AbgDTRI7 (ORCPT ); Mon, 20 Apr 2020 13:08:59 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DA258206D9; Mon, 20 Apr 2020 17:08:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402539; bh=tPDZwM/4wQHICe/qYNofhfEPheexviXoxm4aIF+HkMM=; h=From:To:Cc:Subject:Date:From; b=ZodWwYFXm/PO6kL7uIaDEoy5WDPQfGh/BuqL/cXnoHzPt7N+Sb3IGKy9HHogrOu6P glYU41MYxa7VfXOvbFskposhG9HEtLvgw+fT0+yRGPtXq2OHiw8kCPNjVqcaN//svc AmREQUqe4TU5Ri1ZvaRbttxMWA+fMyqkMeJ/pFp8= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 1/4] fsx: allow zero range operations to cross eof Date: Mon, 20 Apr 2020 18:07:38 +0100 Message-Id: <20200420170738.9879-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana Currently we are limiting the range for zero range operations to stay within the i_size boundary. This is not optimal because like this we lose coverage of the filesystem's zero range implementation, since zero range operations are allowed to cross the i_size. Fix this by limiting the range to 'maxfilelen' and not 'file_size', and update the 'file_size' after each zero range operation if needed. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 9d598a4f..56479eda 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1244,6 +1244,17 @@ do_zero_range(unsigned offset, unsigned length, int keep_size) } memset(good_buf + offset, '\0', length); + + if (!keep_size && end_offset > file_size) { + /* + * If there's a gap between the old file size and the offset of + * the zero range operation, fill the gap with zeroes. + */ + if (offset > file_size) + memset(good_buf + file_size, '\0', offset - file_size); + + file_size = end_offset; + } } #else @@ -2141,7 +2152,7 @@ have_op: do_punch_hole(offset, size); break; case OP_ZERO_RANGE: - TRIM_OFF_LEN(offset, size, file_size); + TRIM_OFF_LEN(offset, size, maxfilelen); do_zero_range(offset, size, keep_size); break; case OP_COLLAPSE_RANGE: From patchwork Mon Apr 20 17:09:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499475 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 895836CA for ; Mon, 20 Apr 2020 17:09:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 687DF22209 for ; Mon, 20 Apr 2020 17:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402550; bh=/09lFz86nyjgOxhAxgGqNyvCbXjNDXalZtzZZTnqodE=; h=From:To:Cc:Subject:Date:List-ID:From; b=ZJu+GynUQfXyHADW5clEz758Dr94i5mK2BzPpAL1NQEBSYNsMWVMmz8CY4c/sE0Om EzfJDYFrALYktC4MObzYIF+MxxRI4l5NYWJZfB6SDWTE2EEDZ1XKtVAGa8+8fe5MjY ejPBbtemGGdWT7fGGhNdHDsz/47MVabqLrprmsWk= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726100AbgDTRJK (ORCPT ); Mon, 20 Apr 2020 13:09:10 -0400 Received: from mail.kernel.org ([198.145.29.99]:42710 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725784AbgDTRJK (ORCPT ); Mon, 20 Apr 2020 13:09:10 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BBBF4206D9; Mon, 20 Apr 2020 17:09:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402549; bh=/09lFz86nyjgOxhAxgGqNyvCbXjNDXalZtzZZTnqodE=; h=From:To:Cc:Subject:Date:From; b=oIvhkGvh8dNq1KRn0wMOb5dpf7Heg+flEnRxEb+xiCTGqgBib7EsCIdC/D7hY14WW FB0tIf5KtSYrlkU6eK6r2EumZ6YwynScbH+SkU9GThHXcaqlEliDKqNpZCTzRnT0KC uusGL6hlPGtrMCjLEZGygRZwJORLaMIR0gqK8mwI= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 2/4] fsx: fix infinite/too long loops when generating ranges for clone operations Date: Mon, 20 Apr 2020 18:09:05 +0100 Message-Id: <20200420170905.9944-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana While running generic/457 I've had fsx taking a lot of CPU time and not making any progress for over an hour. Attaching gdb to the fsx process revealed that fsx was in the loop that generates the ranges for a clone operation, in particular the loop seemed to never end because the range defined by 'offset2' kept overlapping with the range defined by 'offset'. So far this happened two times in one of my test VMs with generic/457. Fix this by breaking out of the loop after trying 30 times, like we currently do for dedupe operations, which results in logging the operation as skipped. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 56479eda..ab64b50a 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -2013,16 +2013,24 @@ test(void) keep_size = random() % 2; break; case OP_CLONE_RANGE: - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset = offset & ~(block_size - 1); + size = size & ~(block_size - 1); + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 = offset2 & ~(block_size - 1); + } while (range_overlaps(offset, offset2, size) || + offset2 + size > maxfilelen); + break; + } case OP_DEDUPE_RANGE: { int tries = 0; From patchwork Mon Apr 20 17:09:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499479 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 689A96CA for ; Mon, 20 Apr 2020 17:09:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 477C3208E4 for ; Mon, 20 Apr 2020 17:09:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402562; bh=1NURecO0mrbzqRJJhqwcnUklSVWjq4mXylGXv77qwzU=; h=From:To:Cc:Subject:Date:List-ID:From; b=msLPkbLZEz32le21K91oM5yxT3wy1163HeMEOdaTBRFkubGLx5/3E3h+MqANJ4Cit nfxaAQumYblVWaa49wqtB1TMn9lCppfgGn99uhRnN66+ZuF60sTbSIf+heqAXodXtg uATxjA/7q4CCm+KJivKQgXhYDk1nmPK08/nEy4vk= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726522AbgDTRJW (ORCPT ); Mon, 20 Apr 2020 13:09:22 -0400 Received: from mail.kernel.org ([198.145.29.99]:42760 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726296AbgDTRJV (ORCPT ); Mon, 20 Apr 2020 13:09:21 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C0D49206D9; Mon, 20 Apr 2020 17:09:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402561; bh=1NURecO0mrbzqRJJhqwcnUklSVWjq4mXylGXv77qwzU=; h=From:To:Cc:Subject:Date:From; b=peRQsf5jD8aed7u1ZY4wNJK+QO4AgM4RRvq4VZgOP6ZyOV+jbl1zygQBSGAAqmxQ2 fuSl2M9t7MkNFHsUIW7tQYhU5oInyVJA26MDEHAf07q7W//7rjzCFUgfbyO4Ar4wtf xV5RRCht31X+atU411k4d7RV4cglJfDMA7nQLGMA= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 3/4] fsx: fix infinite/too long loops when generating ranges for copy_file_range Date: Mon, 20 Apr 2020 18:09:17 +0100 Message-Id: <20200420170917.9994-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana While running generic/521 I've had fsx taking a lot of CPU time and not making any progress for several hours. Attaching gdb to the fsx process revealed that fsx was in the loop that generates the ranges for a copy_file_range operation, in particular the loop seemed to never end because the range defined by 'offset2' kept overlapping with the range defined by 'offset'. So far this happened one time only in one of my test VMs with generic/521. Fix this by breaking out of the loop after trying 30 times, like we currently do for dedupe operations, which results in logging the operation as skipped. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index ab64b50a..40cbd401 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -2051,17 +2051,25 @@ test(void) break; } case OP_COPY_RANGE: - TRIM_OFF_LEN(offset, size, file_size); - offset -= offset % readbdy; - if (o_direct) - size -= size % readbdy; - do { - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 -= offset2 % writebdy; - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; + { + int tries = 0; + + TRIM_OFF_LEN(offset, size, file_size); + offset -= offset % readbdy; + if (o_direct) + size -= size % readbdy; + do { + if (tries++ >= 30) { + size = 0; + break; + } + offset2 = random(); + TRIM_OFF(offset2, maxfilelen); + offset2 -= offset2 % writebdy; + } while (range_overlaps(offset, offset2, size) || + offset2 + size > maxfilelen); + break; + } } have_op: From patchwork Mon Apr 20 17:09:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 11499483 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A79E6CA for ; Mon, 20 Apr 2020 17:09:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71DBC22244 for ; Mon, 20 Apr 2020 17:09:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402576; bh=WcE77tt/bEU+P5Fom4lh616Ry8OSojCe1UWaZIb3OsQ=; h=From:To:Cc:Subject:Date:List-ID:From; b=VulHI+JOidPqWYAjYnD6HYeL1ockWlqM+AklxtzTW+yfNe8H/0r3oxiS+DFpgZmhi jvM5v++KjtY84+h3P+y2neMErjxQ845DSA+m/92YO834DxyYLDSg//yHviRpP5JFsK mmk0pGR2KRSaiGSIx5FI2QtvhzS3jQ//twU5rlEo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726587AbgDTRJg (ORCPT ); Mon, 20 Apr 2020 13:09:36 -0400 Received: from mail.kernel.org ([198.145.29.99]:42844 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726534AbgDTRJf (ORCPT ); Mon, 20 Apr 2020 13:09:35 -0400 Received: from debian7.Home (bl8-197-74.dsl.telepac.pt [85.241.197.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 67F9A206D9; Mon, 20 Apr 2020 17:09:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587402575; bh=WcE77tt/bEU+P5Fom4lh616Ry8OSojCe1UWaZIb3OsQ=; h=From:To:Cc:Subject:Date:From; b=GjEBAfi4jMYQJJO6Dh2OGAVT7U4Iv7XsGCyjUF/VJg3UgkouxgxjPIzg3s9c/iXSr dm8x3vIZ7frGmHRQkp18iBNTF3p9vqQoUjnTINJ/vTG1dR/RKVo+BclEBnD9am9gdA FByEflh3OK3QA3+4i3aNPCxGsBwRIvY87DrNs094= From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 4/4] fsx: move range generation logic into a common helper Date: Mon, 20 Apr 2020 18:09:31 +0100 Message-Id: <20200420170931.10047-1-fdmanana@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: fstests-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana We have very similar code that generates the destination range for clone, dedupe and copy_file_range operations, so avoid duplicating the code three times and move it into a helper function. Signed-off-by: Filipe Manana Reviewed-by: Brian Foster --- ltp/fsx.c | 94 ++++++++++++++++++++++++++------------------------------------- 1 file changed, 39 insertions(+), 55 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 40cbd401..7c76655a 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -1939,6 +1939,39 @@ range_overlaps( return llabs((unsigned long long)off1 - off0) < size; } +static void generate_dest_range(bool bdy_align, + unsigned long max_range_end, + unsigned long *src_offset, + unsigned long *size, + unsigned long *dst_offset) +{ + int tries = 0; + + TRIM_OFF_LEN(*src_offset, *size, file_size); + if (bdy_align) { + *src_offset -= *src_offset % readbdy; + if (o_direct) + *size -= *size % readbdy; + } else { + *src_offset = *src_offset & ~(block_size - 1); + *size = *size & ~(block_size - 1); + } + + do { + if (tries++ >= 30) { + *size = 0; + break; + } + *dst_offset = random(); + TRIM_OFF(*dst_offset, max_range_end); + if (bdy_align) + *dst_offset -= *dst_offset % writebdy; + else + *dst_offset = *dst_offset & ~(block_size - 1); + } while (range_overlaps(*src_offset, *dst_offset, *size) || + *dst_offset + *size > max_range_end); +} + int test(void) { @@ -2013,63 +2046,14 @@ test(void) keep_size = random() % 2; break; case OP_CLONE_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; - } + generate_dest_range(false, maxfilelen, &offset, &size, &offset2); + break; case OP_DEDUPE_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, file_size); - offset2 = offset2 & ~(block_size - 1); - } while (range_overlaps(offset, offset2, size) || - offset2 + size > file_size); - break; - } + generate_dest_range(false, file_size, &offset, &size, &offset2); + break; case OP_COPY_RANGE: - { - int tries = 0; - - TRIM_OFF_LEN(offset, size, file_size); - offset -= offset % readbdy; - if (o_direct) - size -= size % readbdy; - do { - if (tries++ >= 30) { - size = 0; - break; - } - offset2 = random(); - TRIM_OFF(offset2, maxfilelen); - offset2 -= offset2 % writebdy; - } while (range_overlaps(offset, offset2, size) || - offset2 + size > maxfilelen); - break; - } + generate_dest_range(true, maxfilelen, &offset, &size, &offset2); + break; } have_op: