| Message ID | 1344975738-1415-1-git-send-email-jbacik@fusionio.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Tue, Aug 14, 2012 at 3:22 PM, Josef Bacik <jbacik@fusionio.com> wrote: > Swinging this pendulum back the other way. We've been allocating chunks up > to 2% of the disk no matter how much we actually have allocated. So instead > fix this calculation to only allocate chunks if we have more than 80% of the > space available allocated. Please test this as it will likely cause all > sorts of ENOSPC problems to pop up suddenly. Thanks, > > Signed-off-by: Josef Bacik <jbacik@fusionio.com> I've been testing this patch with my multiple rsync test (On a 3.5.1 kernel merged with for-linus). I tested without compression, and with lzo compression, and I haven't run into any ENOSPC issues. I still have ENOSPC issues with zlib, with or without this patch. I made a series of runs with and without this patch (on an uncompressed, newly formatted partition), and some of the results were not what I anticipated. 1) I found that *MORE* metadata space was being allocated with this patch than when using an unpatched baseline kernel. The total allocated space was exactly the same in each run (I saw a slight variation in the amount of used Metadata). On the unpatched baseline kernel, at the end of the run, the 'btrfs fi df' command would show: # btrfs fi df /mnt/benchmark/ Data: total=10.01GB, used=6.99GB System: total=4.00MB, used=4.00KB Metadata: total=776.00MB, used=481.38MB With this patch applied, the 'btrfs fi df' command would show: # btrfs fi df /mnt/benchmark/ Data: total=10.01GB, used=6.99GB System: total=4.00MB, used=4.00KB Metadata: total=1.01GB, used=480.94MB 2) The multiple rsync's would run significantly faster with the patched kernel. Unpatched baseline kernel: Time to run 7 rysncs: 348.3 sec (+/- 9.7 sec) Patched kernel: Time to run 7 rsyncs: 316.6 sec (+/- 6.5 sec) Perhaps the extra allocated metadata space made things run better, or perhaps something else was going on. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Aug 15, 2012 at 11:29:11AM -0600, Mitch Harder wrote: > On Tue, Aug 14, 2012 at 3:22 PM, Josef Bacik <jbacik@fusionio.com> wrote: > > Swinging this pendulum back the other way. We've been allocating chunks up > > to 2% of the disk no matter how much we actually have allocated. So instead > > fix this calculation to only allocate chunks if we have more than 80% of the > > space available allocated. Please test this as it will likely cause all > > sorts of ENOSPC problems to pop up suddenly. Thanks, > > > > Signed-off-by: Josef Bacik <jbacik@fusionio.com> > > I've been testing this patch with my multiple rsync test (On a 3.5.1 > kernel merged with for-linus). > > I tested without compression, and with lzo compression, and I haven't > run into any ENOSPC issues. I still have ENOSPC issues with zlib, > with or without this patch. > > I made a series of runs with and without this patch (on an > uncompressed, newly formatted partition), and some of the results were > not what I anticipated. > > 1) I found that *MORE* metadata space was being allocated with this > patch than when using an unpatched baseline kernel. The total > allocated space was exactly the same in each run (I saw a slight > variation in the amount of used Metadata). > > On the unpatched baseline kernel, at the end of the run, the 'btrfs fi > df' command would show: > > # btrfs fi df /mnt/benchmark/ > Data: total=10.01GB, used=6.99GB > System: total=4.00MB, used=4.00KB > Metadata: total=776.00MB, used=481.38MB > > With this patch applied, the 'btrfs fi df' command would show: > > # btrfs fi df /mnt/benchmark/ > Data: total=10.01GB, used=6.99GB > System: total=4.00MB, used=4.00KB > Metadata: total=1.01GB, used=480.94MB > > > 2) The multiple rsync's would run significantly faster with the patched kernel. > > Unpatched baseline kernel: Time to run 7 rysncs: 348.3 sec (+/- 9.7 sec) > Patched kernel: Time to run 7 rsyncs: 316.6 sec (+/- 6.5 sec) > > Perhaps the extra allocated metadata space made things run better, or > perhaps something else was going on. Well that's odd, I wonder if we're doing the limited dance more often. Once I've finished my fsync work I'll come back to this. I know for sure in my tests it's allocating chunks way too often, so I imagine your test is just tickling a different aspect of the chunk allocator. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index ce494b9..eaf1a9e 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -3487,7 +3487,8 @@ static int should_alloc_chunk(struct btrfs_root *root, * and purposes it's used space. Don't worry about locking the * global_rsv, it doesn't change except when the transaction commits. */ - num_allocated += global_rsv->size; + if (sinfo->flags & BTRFS_BLOCK_GROUP_METADATA) + num_allocated += global_rsv->size; /* * in limited mode, we want to have some free space up to @@ -3501,15 +3502,8 @@ static int should_alloc_chunk(struct btrfs_root *root, if (num_bytes - num_allocated < thresh) return 1; } - thresh = btrfs_super_total_bytes(root->fs_info->super_copy); - /* 256MB or 2% of the FS */ - thresh = max_t(u64, 256 * 1024 * 1024, div_factor_fine(thresh, 2)); - /* system chunks need a much small threshold */ - if (sinfo->flags & BTRFS_BLOCK_GROUP_SYSTEM) - thresh = 32 * 1024 * 1024; - - if (num_bytes > thresh && sinfo->bytes_used < div_factor(num_bytes, 8)) + if (num_allocated + alloc_bytes < div_factor(num_bytes, 8)) return 0; return 1; }
Swinging this pendulum back the other way. We've been allocating chunks up to 2% of the disk no matter how much we actually have allocated. So instead fix this calculation to only allocate chunks if we have more than 80% of the space available allocated. Please test this as it will likely cause all sorts of ENOSPC problems to pop up suddenly. Thanks, Signed-off-by: Josef Bacik <jbacik@fusionio.com> --- fs/btrfs/extent-tree.c | 12 +++--------- 1 files changed, 3 insertions(+), 9 deletions(-)