From patchwork Tue Nov 15 06:37:10 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Butsykin X-Patchwork-Id: 9429085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 035CB60469 for ; Tue, 15 Nov 2016 07:13:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D4C4C2864B for ; Tue, 15 Nov 2016 07:13:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C73E3289F4; Tue, 15 Nov 2016 07:13:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 219EC2864B for ; Tue, 15 Nov 2016 07:13:45 +0000 (UTC) Received: from localhost ([::1]:44615 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c6Xw8-0000rU-Bq for patchwork-qemu-devel@patchwork.kernel.org; Tue, 15 Nov 2016 02:13:44 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47709) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c6Xvb-0000p5-Kf for qemu-devel@nongnu.org; Tue, 15 Nov 2016 02:13:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c6XvY-0000wW-FP for qemu-devel@nongnu.org; Tue, 15 Nov 2016 02:13:11 -0500 Received: from mail-ve1eur01on0113.outbound.protection.outlook.com ([104.47.1.113]:61661 helo=EUR01-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c6XvY-0000vy-58; Tue, 15 Nov 2016 02:13:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=vP4hrVlOjoX2YA7I6ONiDWb2kAH+ADHDRntSkD0DP5k=; b=RBmItrU2KFscjuXQKSAnCOitJcqi9ChNvccOVVy7rPQrtzM93O7u83pVGW/0ICoFiQRu1PdMX8VCDMgF5L2gqLx9wxofvvPJBydzQuu9GH3o07zwGG5v00Lh4quEcPg/wIoY2tg2WeGIlqVBWHHTWKnduXQdC8JUvlxvPpkn0Tg= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=pbutsykin@virtuozzo.com; Received: from pavelb-Z68P-DS3.sw.ru (195.214.232.10) by VI1PR0802MB2558.eurprd08.prod.outlook.com (10.172.255.136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.721.10; Tue, 15 Nov 2016 06:38:27 +0000 From: Pavel Butsykin To: , Date: Tue, 15 Nov 2016 09:37:10 +0300 Message-ID: <20161115063715.12561-14-pbutsykin@virtuozzo.com> X-Mailer: git-send-email 2.10.1 In-Reply-To: <20161115063715.12561-1-pbutsykin@virtuozzo.com> References: <20161115063715.12561-1-pbutsykin@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: AM5PR0101CA0010.eurprd01.prod.exchangelabs.com (10.169.240.20) To VI1PR0802MB2558.eurprd08.prod.outlook.com (10.172.255.136) X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2558; 2:Q0kbAprkayiW/WbwUfn2T0Vl8ZZPHRkp3yTESbeHn98eZKYXLfz9mS795Wm2WSy9JlXXDXXs5L7VQKsBn3Dj1cUXJnygbNbrtnnQ95GABSjUao7SFIO1AiOuO3EK9hGKlFAwEfu4s4fOJB4feKuIL67LBvelTBgoXyT/Dl3P+KI=; 3:Mv+4UQJk5qaqohVyxZUnnsgW47H1pct9Hz/uw2tjorN/1ZtsgS2uulPaEoloJzyRKJVQ4YnSVOUtGRxXVu7DJc3lY8CKjq6XS0liWovxK8St0T54qzTTKuW/rYkMV9GtHRXfLG1rCbuyiyXQPC/Z95+Fi91dRL+xVHYHbXfjb28= X-MS-Office365-Filtering-Correlation-Id: e47806d1-7127-4aff-60be-08d40d22022c X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:VI1PR0802MB2558; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2558; 25:IZBqfe8XMM4bvJ/UrkmqlQF4aUF10ny6ZTS+c+tie5Kpq+HmEf/zvJFldrUohWcyLj4GXR2GNZxWpPPgnzlcJ6Y5wPaRtaFyjqjOWYFFDThZvg96SrTFxZJBgRxjj+IimyUyaMMQRfvIb11KJPCYd/1YluvXqz0Yf+2CfS2C+FouJfbMzt8QYORqHdx3TQyTGSo1EQBsQjMTs5DfUYaxtsFPZo7svjPsb2VLY5PT+sUsIQKCGFQ/gFlV2tdCwLMrh7iLpLfhPYx3eK2TagR/b1CGZi6ApoEyV33K4Aezc2B6w7sc6ge2UnThX6JlwSf1fxpsbntBMH1yJ1V63HKvo57QGhf3axtM9byYgqyg3qo0vebGbLlQi4ufLFH0ZeLNvRiqFljdxO4yGfu/vIkXv2cqbvU7ROCcRa0pHzNbE5yAPJClpa0fpHneVMQ7VhT7O1ZpqjrhooqzNmkRjTQi37Y5MXu2SrQTmkVgnk9kD2L5HbUOF0bTBEzucrIfvLVaT1t3d4n16SOdlWyFmNkUe7MyxAh7pSvzf2CoIZbLvdXZbGsLqOYKCaWQzyCseidWHp1mIXrnKgLmeMXI2mDZKDyI2OUVUEiM+JiOpEvihkiCq1N/2caCuCmIdyK43zEzJYpQSXN768Ya2isNvByEuKZ+ipvxXib1M5P95pUeUoWhZCEc2+2dL3Gzh9MOLnj93ZJ1ZHJ9NuxO6H8grhcg4j7G6Ru44yVHHW4g3GRZ1p4= X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2558; 31:9NnLyt4R9G+BJVrjo2jKIcYOxyLJaim+29QY0fSJYY7K6TOm8E9MIbmuU9kiCNSATT2LRcdSzyid2E3JixS7FvC0KoYB5XVBM+J0SwmzOKz7ya+gAAUkiMrNu/EnKGNGG66MQbYL6V4EFXCQzxnjurumYoGxGvUNuOTo95+/ZNaq+mQnJfeCVf8v5SsHMTTxXjZcFn9rjoAuo202uNjQMcFWCGIPf1JFlCTiux0fdIoGCf7CEq8cNH7sNd23WFKirme5YUulup5TxvQhatv2KFbwvYRcecIGimgXFK0aoB0=; 20:AjCZMv0Z9i9oD95wRm69yGCKmgUDiiJyVztepTBh9yVGvk35N2eA3lgku+rPRhB0jc8vwALdZZLJZdJz3y2RwtjFkUz5YiMX1zxfHS+b3sfLPEWqstOmUOW7TvQDcWWpJwUvsfsEVeL3FAeo8XqPWOdbyw58vmv84Rvq34oudWM=; 4:gqq8gxXDoSqm2lCzarryAu+9gWwi6joTPgZDHAyYCGXtKlrJw1rou2YNpYhVE3noQ4uG3pIiZu+QEplj1wbLEKu758QTUG+zeCpPJyNskUedYI+n9EMm/VMhImasi7FsZe48FX3Ido5gF/6ZoKZK2NNhVQoC8dtiVybOxcPHSgfM+5lc4xTRdzqO9dPazQBmLG0NC8TuUfzbOBKm/azurSVhC1ckBuaxfQ2ntvPlHdR5xR/OeGLZBQX9O7k1R2qaoRLzD9PH04vycXhX0N+bDqWeq6u4JKpMg+riic4iMxv3PpW11IjIUm6OaQHIeFnVhn/V1BxHvkANSZ0xQCM0MdQxYuW2oplP4M18Swua/jXKxMLhJY4j2NrgcYnCj7WUOqkQnF3dJlpZooIuWZZomDjKLxSKOl/AuFc0+CgusZLyMd7kGj1WnsY+dM2NQiVx X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6060326)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6061324)(6043046); SRVR:VI1PR0802MB2558; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0802MB2558; X-Forefront-PRVS: 012792EC17 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(7916002)(199003)(189002)(47776003)(105586002)(101416001)(1076002)(68736007)(106356001)(6666003)(76176999)(305945005)(4326007)(97736004)(66066001)(53416004)(189998001)(42186005)(69596002)(2906002)(7736002)(5001770100001)(7846002)(86362001)(48376002)(50226002)(2950100002)(50466002)(50986999)(5003940100001)(33646002)(77096005)(81156014)(8676002)(92566002)(81166006)(3846002)(36756003)(6116002)(5660300001); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0802MB2558; H:pavelb-Z68P-DS3.sw.ru; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0802MB2558; 23:bF4hQVlxuSnY3BsX51b/z0dJv4R5W58+zyuiQ/m?= =?us-ascii?Q?CcJ7IVbfpwqmOMTf8s/rCvUq5HLErvNxLWjQbMK7gdgiMxMhphbSMhBrO0kR?= =?us-ascii?Q?kYXATsFKQl7tEcajUQHYWYy2stuDzMUwRk9H3moVM+ifwR2mdyM51F+0473D?= =?us-ascii?Q?61pIiViGXGOq78p+P6PKQTm5Oc6NGkMvKWxMToQJPAfwYhsAhW9vob3sZtXW?= =?us-ascii?Q?OE08X0/qqVBkLpXua08zkKnmZ9whzm2+eTzDZlemaUeo6G/A4iiic47qGLe7?= =?us-ascii?Q?nx4cOpCTKBocLH4mWv09HMnXGJucvxc3NVPGYzzx1uPqA2zKW9boQlbASboS?= =?us-ascii?Q?SQCGwcZAxd/5+z/hGpkrAAHvshdQHK4w/GZawQxH3nkiJMmB7YPTCnx8Fpxq?= =?us-ascii?Q?JAgM64jj5kRClx5+QNSQRrXCfH8CYawnTMzkdynzkYglRVlOo67MJbuIJnsm?= =?us-ascii?Q?Shc4WZmiuMy50DnCvNrB9cTKqZrJ2aqd8rzANWLsXEaIrS/x4Y3XLY6SKclD?= =?us-ascii?Q?kZoP23wFPga6ivrgmWhcWdmH0NiNV9wwCp/7rOhC3QdH/ID4+trBUqq5IOus?= =?us-ascii?Q?VwzQq1dQEo4OQySM2tkSutrTf1BMZk4U2v0gABp4DNUP4x1HwPX2WcgWw/nE?= =?us-ascii?Q?DStt0xwZojgPhPWzPclwwPZxDk7Sg32QsiPQS0jcFJSb9LlWpkJSi8M3L0h7?= =?us-ascii?Q?XcxZ4Nkfe7KwHKGJDTfzZrYhGEjUmF+8t+s1FHHLnKlR1FkqHgh+ml2m93H1?= =?us-ascii?Q?FkdV0Xj+shBtaMhVgFgb7hJd++AfTB+sCT7XIQx8RDQ1gWMoJgdejGRv0AJ+?= =?us-ascii?Q?Cu0EN0zmYZAvb6JECA5Ct4+QRLIPQYh4G4rhVMTO4yY8tyDU1oTt69rVeyZS?= =?us-ascii?Q?BaIiPSQr7OD4Eiqk+5U1qb0PfVXKoC+fuJ/K9FIFuLqnU98up6Mp4OBSDDlH?= =?us-ascii?Q?WK630X76ofSpItQQAF1fZrhM48QiuT7WPqSQtCdBWmtUabpwgi9Atv8y1z3U?= =?us-ascii?Q?7wpM2kGYBdxAGQYhnQyP1466gqUSS0xPmntPoa6BX1EP1hBNMpSp7aXNMT5e?= =?us-ascii?Q?53cArw61VrQRvWGJe/m5CamK74Dwj?= X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2558; 6:+FmXaBHIO+Tz+0byObF98+RmebkqzvJZ3pIeMhybdDmnzBmCoe0TsQZnlBl4LQlknFUgKUHy80CS5WbhF7UVkVaJxXZ0O6BDYhOnBsbiSvGg/ArgKNXbWKQP3zbPkdaTngg6P8/0g9ZG0Q62LtWb/gqt88l7WpKGBTz75i9IshBmaBFzhgtPOrkyqyam9i9sHMqTxfg+dIZfRXdrr7u6OtDLgF1rz7W29BM2/xnfxaMP5sZSvU+wxSTuABWr+/HxAT7SAS+9jkMeJ4GHKyzc9DU3EXh+ys/mEFGHEq7UlwG9FtjYgh59tw3iCKmPKaHZO5kR3flyIpVRiMu17ID2CK4eqOyO8mv2uBiyUT3OlWU=; 5:7SPg1duqqHliyV8qkyn8TWcrDMPlmrL2SNJeI+HEYggU4PdMT27z16QjHucZtxpJI/bO0y8XY0U05ALff7OJWR5ezV0F2KeC8GYljwgbcp2YBsghhmAuNVPfd7t8a96qq03gmfCKcIMon5SUGIEhKKjL4/rafXkEuyyNOK5iZPI=; 24:wD6s8bfEI9Ey2eIrkF3zPKmda2KeVgBUtbreXTrR13dUc2F2XA/FGEXA3BLYBbRurZXxrE5INj9v+MUzfghlROP6uNlVuE5DLLe9v8/LWZA= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0802MB2558; 7:dDNsw+I50PhcHGJx6iolziWJBUuWTDkm/hB6BSAcmd6hNAnadbR73koVx83AhywMT/tLLWf9uStnlGBt+MjtAxGLSTakQL3gdRjZWFn3Pufuquq5PltDLcl5wEHAw6GSdj3u4c9BX19ja2AKr9mZr41cg0rdk7+JU08mVdgBUART6f1m7HeIxeMWsDF9o8/0CCS+7WhHWvQYtksthQY1PZGCPMVzcmR40S15Y6v6VC92r6S3204d5PsC3yJOtwd3R6uqxkSoMSgKP3VbdAqweZVW9DoQKWaqDNGi2xJs49g2C6762wqSTh5Nqak94djWTTiFnZ1vnyEJ6hv48qZwg/mbwBA+N7LL8k4S2yG6D8o=; 20:fV3YRqn0B/G42kw6FlqAlcV7LaWBsJdpo+MVCIVpE4RRb4a/XR2PumbL+sWdpTopSH8NlPYde6HDjPtvV9AyAlOGhpRJtzAMSko9BaqCGbTOEsO3v05UUSDWgd/1ad9nuJz6T/QDwjfVrI4AisVNLDXCD0gQI50LtWzqIWGXx6Y= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Nov 2016 06:38:27.5808 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2558 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 104.47.1.113 Subject: [Qemu-devel] [PATCH v1 13/18] block/pcache: inflight readahead request waiting for aio read X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP If there was a cache hit, but the status of the node is NODE_STATUS_INFLIGHT, then this means that a readahead request for this node is in flight. In this case, we can wait for the completion of the readahead request, and then copy the data. It allows us even more to optimize aio read requests. Signed-off-by: Pavel Butsykin --- block/pcache.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 65 insertions(+), 8 deletions(-) diff --git a/block/pcache.c b/block/pcache.c index 0f918d0..55d1061 100644 --- a/block/pcache.c +++ b/block/pcache.c @@ -67,9 +67,15 @@ typedef struct BDRVPCacheState { uint64_t readahead_size; } BDRVPCacheState; +typedef struct ACBLinkEntry { + QTAILQ_ENTRY(ACBLinkEntry) entry; + struct PCacheAIOCBRead *acb; +} ACBLinkEntry; + typedef struct PCacheNode { RBCacheNode common; uint8_t *data; + QTAILQ_HEAD(, ACBLinkEntry) wait_list; enum { NODE_STATUS_NEW = 0, NODE_STATUS_INFLIGHT = 1, @@ -94,6 +100,7 @@ typedef struct PCacheAIOCBRead { uint64_t offset; uint64_t bytes; QEMUIOVector *qiov; + int ref; int ret; } PCacheAIOCBRead; @@ -147,18 +154,49 @@ static void read_cache_data(PCacheAIOCBRead *acb, PCacheNode *node, assert(copy == size); } +static void pcache_aio_read(PCacheAIOCBRead *acb, PCacheNode *node) +{ + ACBLinkEntry *link; + + if (node->status == NODE_STATUS_COMPLETED) { + read_cache_data(acb, node, acb->offset, acb->bytes); + return; + } + + assert(node->status == NODE_STATUS_INFLIGHT || + node->status == NODE_STATUS_REMOVE); + + link = g_malloc(sizeof(*link)); + link->acb = acb; + + QTAILQ_INSERT_HEAD(&node->wait_list, link, entry); + acb->ref++; +} + +static void aio_read_complete(PCacheAIOCBRead *acb, int ret) +{ + if (ret < 0 && acb->ret == 0) { + acb->ret = ret; + } + + assert(acb->ref > 0); + if (--acb->ref == 0) { + qemu_coroutine_enter(acb->co); + } +} + static void pcache_aio_read_cb(void *opaque, int ret) { PCacheAIOCBRead *acb = opaque; - acb->ret = ret; - qemu_coroutine_enter(acb->co); + aio_read_complete(acb, ret); } static void pcache_aio_readahead_cb(void *opaque, int ret) { PCacheAIOCBReadahead *acb = opaque; PCacheNode *node = acb->node; + ACBLinkEntry *link, *next; assert(node->status == NODE_STATUS_INFLIGHT || node->status == NODE_STATUS_REMOVE); @@ -171,6 +209,20 @@ static void pcache_aio_readahead_cb(void *opaque, int ret) rbcache_remove(s->cache, &node->common); } } + + QTAILQ_FOREACH_SAFE(link, &node->wait_list, entry, next) { + PCacheAIOCBRead *acb_read = link->acb; + + QTAILQ_REMOVE(&node->wait_list, link, entry); + g_free(link); + + if (ret == 0) { + read_cache_data(acb_read, node, acb_read->offset, acb_read->bytes); + } + + aio_read_complete(acb_read, ret); + } + pcache_node_unref(node); qemu_coroutine_enter(acb->co); @@ -254,6 +306,7 @@ static RBCacheNode *pcache_node_alloc(uint64_t offset, uint64_t bytes, node->data = g_malloc(bytes); node->status = NODE_STATUS_NEW; node->ref = 1; + QTAILQ_INIT(&node->wait_list); return &node->common; } @@ -379,7 +432,7 @@ static int pcache_lookup_data(PCacheAIOCBRead *acb) BDRVPCacheState *s = acb->bs->opaque; PCacheNode *node = rbcache_search(s->cache, acb->offset, acb->bytes); - if (node == NULL || node->status != NODE_STATUS_COMPLETED) { + if (node == NULL) { return CACHE_MISS; } @@ -387,7 +440,7 @@ static int pcache_lookup_data(PCacheAIOCBRead *acb) if (node->common.offset <= acb->offset && node->common.offset + node->common.bytes >= acb->offset + acb->bytes) { - read_cache_data(acb, node, acb->offset, acb->bytes); + pcache_aio_read(acb, node); return CACHE_HIT; } @@ -405,6 +458,7 @@ static coroutine_fn int pcache_co_preadv(BlockDriverState *bs, uint64_t offset, .offset = offset, .bytes = bytes, .qiov = qiov, + .ref = 1, }; int status; @@ -417,14 +471,17 @@ static coroutine_fn int pcache_co_preadv(BlockDriverState *bs, uint64_t offset, update_req_stats(s->req_stats, offset, bytes); status = pcache_lookup_data(&acb); - if (status == CACHE_HIT) { - return 0; + if (status == CACHE_MISS || status == PARTIAL_CACHE_HIT) { + bdrv_aio_preadv(bs->file, offset, qiov, bytes, + pcache_aio_read_cb, &acb); } - bdrv_aio_preadv(bs->file, offset, qiov, bytes, pcache_aio_read_cb, &acb); - pcache_readahead_request(&acb); + if (status == CACHE_HIT && --acb.ref == 0) { + return 0; + } + out: qemu_coroutine_yield();