From patchwork Mon Jan 8 20:21:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Lyle X-Patchwork-Id: 10150553 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9C35A602CA for ; Mon, 8 Jan 2018 20:22:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2EDE1FFDA for ; Mon, 8 Jan 2018 20:22:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 97E5522F3E; Mon, 8 Jan 2018 20:22:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7C2D28A23 for ; Mon, 8 Jan 2018 20:21:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757216AbeAHUVt (ORCPT ); Mon, 8 Jan 2018 15:21:49 -0500 Received: from mail-pg0-f65.google.com ([74.125.83.65]:34827 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757214AbeAHUVp (ORCPT ); Mon, 8 Jan 2018 15:21:45 -0500 Received: by mail-pg0-f65.google.com with SMTP id d6so5243895pgv.2 for ; Mon, 08 Jan 2018 12:21:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lyle-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7WokGLswrzgwdnrw0obtBT8a2by12YRBW3glxhMQLyw=; b=Loxu7Ucjwc1cUHkMJfIkvJvAq/iktXg5MMn59S2nGR+H8TWOVlQ9i2gpi/LxwAWUNA DdCgL59KiknWVZ4YGmN6EjVzqLFHcIBUw7wYuN0gkI5zvnbYHR9JjxGEU222lPnNdLQO Y6hvOoO9NiYcuAmcxWICCXkpor0maeLgN1RPJPBRzQK3tGQU+VzunbOLqAVYIEs/2SrC opbF2eWvnFTPjyfFBWUFjxF72E1AzNcUgYs9n2s3ptwn3SBsG73CD0WkDmxPe0XxKocW IrmI9hAuVPRhspk0hu8MYXvE1eWq4Jdby6K1lxn6SX4MudM8PH72AMZa8r1I8YNqHGGp NNKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7WokGLswrzgwdnrw0obtBT8a2by12YRBW3glxhMQLyw=; b=KPYctfNCY6nFEOc/4BYJc9TBHgkHcT5XlOKhKvVH7iekw/jAF7ckbpFEoT+yc5bIlL BAHw7H6c0x/RU7AgmleummdFTrVVAt3uEvBqoO189baxPgZXxuoVfgWkWgZMstFfryyU ihdu5LGWs07L5K2xHC+p8V3C1uEPyPmBvUn/qSUjMLPSd9ko/GW65QO4wW0Z1zPRDZ7h A3vyG7rpld0jIe+lKMZ2QBL1shOH1l4R3FjxIAN2IhChpqFo7K1kmY5IyUer72HZ7Uct DhCiUdyfebMcbDGerNlSQGIPgXm28Ezg4GQ3zYZDdKqsN+HZw9MX7eSi4HMfNhZwYGr+ W0iQ== X-Gm-Message-State: AKGB3mKt0pIX2/hv+BiO94QSb9dj+AVUHjgOpuxxqx+yU1zRgtMVnXil SsT7iwRRLGeR/wlw660jhx8YMNn22IQ= X-Google-Smtp-Source: ACJfBosCuDhJ5LzslgJ8C4CLZ4UsSu2e5vDD3dmiUKXQJSIoz1fL22TS/olnhsAPVt1t+GuqO+uaTA== X-Received: by 10.84.169.4 with SMTP id g4mr3377329plb.245.1515442904318; Mon, 08 Jan 2018 12:21:44 -0800 (PST) Received: from midnight.lan (2600-6c52-6200-383d-a0f8-4aea-fac9-9f39.dhcp6.chtrptr.net. [2600:6c52:6200:383d:a0f8:4aea:fac9:9f39]) by smtp.gmail.com with ESMTPSA id o70sm28540227pfk.79.2018.01.08.12.21.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Jan 2018 12:21:43 -0800 (PST) From: Michael Lyle To: linux-bcache@vger.kernel.org, linux-block@vger.kernel.org Cc: axboe@fb.com, Tang Junhui , Michael Lyle Subject: [416 PATCH 04/13] bcache: segregate flash only volume write streams Date: Mon, 8 Jan 2018 12:21:21 -0800 Message-Id: <20180108202130.31303-5-mlyle@lyle.org> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180108202130.31303-1-mlyle@lyle.org> References: <20180108202130.31303-1-mlyle@lyle.org> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tang Junhui In such scenario that there are some flash only volumes , and some cached devices, when many tasks request these devices in writeback mode, the write IOs may fall to the same bucket as bellow: | cached data | flash data | cached data | cached data| flash data| then after writeback of these cached devices, the bucket would be like bellow bucket: | free | flash data | free | free | flash data | So, there are many free space in this bucket, but since data of flash only volumes still exists, so this bucket cannot be reclaimable, which would cause waste of bucket space. In this patch, we segregate flash only volume write streams from cached devices, so data from flash only volumes and cached devices can store in different buckets. Compare to v1 patch, this patch do not add a additionally open bucket list, and it is try best to segregate flash only volume write streams from cached devices, sectors of flash only volumes may still be mixed with dirty sectors of cached device, but the number is very small. [mlyle: fixed commit log formatting, permissions, line endings] Signed-off-by: Tang Junhui Reviewed-by: Michael Lyle Signed-off-by: Michael Lyle --- drivers/md/bcache/alloc.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/md/bcache/alloc.c b/drivers/md/bcache/alloc.c index a0cc1bc6d884..6cc6c0f9c3a9 100644 --- a/drivers/md/bcache/alloc.c +++ b/drivers/md/bcache/alloc.c @@ -525,15 +525,21 @@ struct open_bucket { /* * We keep multiple buckets open for writes, and try to segregate different - * write streams for better cache utilization: first we look for a bucket where - * the last write to it was sequential with the current write, and failing that - * we look for a bucket that was last used by the same task. + * write streams for better cache utilization: first we try to segregate flash + * only volume write streams from cached devices, secondly we look for a bucket + * where the last write to it was sequential with the current write, and + * failing that we look for a bucket that was last used by the same task. * * The ideas is if you've got multiple tasks pulling data into the cache at the * same time, you'll get better cache utilization if you try to segregate their * data and preserve locality. * - * For example, say you've starting Firefox at the same time you're copying a + * For example, dirty sectors of flash only volume is not reclaimable, if their + * dirty sectors mixed with dirty sectors of cached device, such buckets will + * be marked as dirty and won't be reclaimed, though the dirty data of cached + * device have been written back to backend device. + * + * And say you've starting Firefox at the same time you're copying a * bunch of files. Firefox will likely end up being fairly hot and stay in the * cache awhile, but the data you copied might not be; if you wrote all that * data to the same buckets it'd get invalidated at the same time. @@ -550,7 +556,10 @@ static struct open_bucket *pick_data_bucket(struct cache_set *c, struct open_bucket *ret, *ret_task = NULL; list_for_each_entry_reverse(ret, &c->data_buckets, list) - if (!bkey_cmp(&ret->key, search)) + if (UUID_FLASH_ONLY(&c->uuids[KEY_INODE(&ret->key)]) != + UUID_FLASH_ONLY(&c->uuids[KEY_INODE(search)])) + continue; + else if (!bkey_cmp(&ret->key, search)) goto found; else if (ret->last_write_point == write_point) ret_task = ret;