From patchwork Mon Aug 29 17:10:08 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Butsykin X-Patchwork-Id: 9304451 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2317860756 for ; Mon, 29 Aug 2016 20:45:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 137352876F for ; Mon, 29 Aug 2016 20:45:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 07BF828942; Mon, 29 Aug 2016 20:45:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2EA952876F for ; Mon, 29 Aug 2016 20:45:51 +0000 (UTC) Received: from localhost ([::1]:45696 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1beTRG-0008Hu-Ay for patchwork-qemu-devel@patchwork.kernel.org; Mon, 29 Aug 2016 16:45:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59692) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1beTQk-0008Dc-S3 for qemu-devel@nongnu.org; Mon, 29 Aug 2016 16:45:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1beTQj-00076H-1J for qemu-devel@nongnu.org; Mon, 29 Aug 2016 16:45:18 -0400 Received: from mail-eopbgr30122.outbound.protection.outlook.com ([40.107.3.122]:55008 helo=EUR03-AM5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1beTQc-00074p-2q; Mon, 29 Aug 2016 16:45:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=gasDlAAuspOAZXtGat9GpYIO3kD68ZRD3y7+3hFUPEI=; b=XtCOfvEJGrX/l8xO3IYu14NqTOsFd2nx8MSVMgHZyeaiL+kHHJJjjWITD8pLyvu0u7DC3ALvSXPounAc8YLVqu1LJn4pMq5UDxuwsrbzGqtxNgk4lSwm031nkYT4Nk//2ZmgJ1UO+4qs4/GXLasUEzy30hs0tqoGZx5hlel1e94= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=pbutsykin@virtuozzo.com; Received: from pavelb-Z68P-DS3.sw.ru (195.214.232.10) by AM5PR0802MB2547.eurprd08.prod.outlook.com (10.175.45.23) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.13; Mon, 29 Aug 2016 17:11:22 +0000 From: Pavel Butsykin To: , Date: Mon, 29 Aug 2016 20:10:08 +0300 Message-ID: <20160829171021.4902-10-pbutsykin@virtuozzo.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20160829171021.4902-1-pbutsykin@virtuozzo.com> References: <20160829171021.4902-1-pbutsykin@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: AM5PR0901CA0011.eurprd09.prod.outlook.com (10.164.186.149) To AM5PR0802MB2547.eurprd08.prod.outlook.com (10.175.45.23) X-MS-Office365-Filtering-Correlation-Id: 8a07a550-673a-464d-e124-08d3d02f8081 X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 2:2fG1QL1Sb6sOnX/ZQqSBt4km5keLTBU/ERSN7dfiPIraRy5k4822LgRrKkw5SAhZXQCYpfgVjJhytK7/L+5J7yjMdDSDT65X+LKmKYxEZeobHLzGmjhLSOkHhfUTsS5e68cRkmMlgkNDef/9ThUpQDlLAoq0BMQjJOgMYotiokoaWOMMi9fgvOGBoeh6qG7J; 3:fQFenSQY7vk0HC7pfxAcUy9fR6L6NMeGakO9D0ZCh6UPqscRP9f0tVROaS+OlZVE+6PAkOLBf866poF3Q4UMoKHZtzjyrHFu5fC+UbFImlIKqxttvgpZiCUUAkEM/lXD X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM5PR0802MB2547; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 25:lrzWjweYKfiCLaeNlKk+TS45rLVrfr5LJjaxyzOYyTSXBdtMLUlm6Bs825wnquRYQc8k0Tsz0TeVz414N2PjKDUa0rtDhPwA8xsMbABmftXxyCGVbanBudq/tydtRAiOtHONaiof92kbyVM48CsQ3WA6ncll4bCsEBlK+bbAqgXQrEwReyGU5ARBHOV0LohmWvfyUWkpcdMeI5GyTpcPdQYmjGvZwy3WJzTo7BA2Xib10ybafnsmhOkAeF5uIhiKYRWWSJmEDGKYV5WJ++fvlB6hatnNTkI4TAwHmczqvYtLQOpcFN7ByWdQNwWCgr0Xlj389stfnTZ0jXT5HddZNrV4US6x3RWCxs0xx7roy2szOX0bUcqEHf8kwBLSk2qgD0B54/hslxZOO5rb3szNdDi8yAJGb4gIiehZLdwxySFNHgl5Z9tbXUfR9jdGyQImqiuZZJ8zvmSdV8c1XmM/zUzmVGGl359nFMWpOQbkyY8bDiKkPABn+mzCwJSFHQU2sM1JbYmG07AkzCfQCAxuwG8cs9JCf/jEp6yoGhGjKULXL2KaKjFoJRGXRzPRufipZ1m3EbBHBXxg8w0Dsx0gc0uBKo+04xllVYGgMF8wSKQHp+ZL/vmb19tRqhlvViLkeh4aRVSmE8dXsAtkZ5w0eT5A8SAk3JXih+Sd8qCoHIlQsqZ3L+GTmhG6V3wjynSjD8MS3OnzH+EkMqu+jHc1VA==; 31:F/CVMPbyW4pW+Mzeb1OJUrLq2km5bxLk1+VgVnGaLNDD2i3WD9sDTu3k10UhfnghS/I9Lc2ZcpDjIC4/Sh41i0FCMO0juQefGICCk31yPGY2c7Vsy4AMLvp4hIEAQ1i58gnrMLpeMyL7Wtwfe2763iOTI8i7/Xuc+Yl3XpQybo/N5pRJQwDas+RqEkuUaI93ONC0XVImRoSfceQaeoXEs13sD5Y4FCLB5htmI9mN4VI= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6042046)(6043046); SRVR:AM5PR0802MB2547; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0802MB2547; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 4:C9V0zhY7FlqzBd1GkLMQI03etYZugpo/SA5MH6CWDVhegDr/ncOHzwbRYYVaN0sT0lKPnbgwUWXpnvovm0k4opzty5n1hS8u+ZIXmnJucN+zYTuLNqzCPiF+ImmH3jmiPzgtobyekrvOrzAqpGzkMLtjEbjmDbdtw5DvoXNDYEQoUaxZc+gQJ1cAoVy7XTcK8fA+Uu0GrB1pMUuKnhUe8gQDXHdMjoZPejDizd63m/3fOCsMCMfQBqNcL/e1hT5JxAa5Ev9B2IwI4faw3bO1FhrpKF7fSFgjDl76UD9NWOW30p9+SpW68Wf5hHlosWc4ZA/CQ8OEjZzv+4QhLl9U02qy6x9WR+ck073PlAzZbbilgN4AHA7abmvKKBNYB+KrGsQ+VeGfvXU1p1zkwBFWF1UbzgWgUAp8P20Bmk7tOn4= X-Forefront-PRVS: 0049B3F387 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6009001)(7916002)(199003)(189002)(3846002)(305945005)(77096005)(86362001)(1076002)(53416004)(92566002)(68736007)(42186005)(97736004)(5001770100001)(47776003)(105586002)(2906002)(6116002)(189998001)(50986999)(4326007)(48376002)(586003)(2950100001)(5003940100001)(50466002)(66066001)(19580405001)(19580395003)(76176999)(69596002)(5660300001)(229853001)(101416001)(36756003)(7736002)(8676002)(106356001)(50226002)(7846002)(81156014)(33646002)(81166006)(21314002)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:AM5PR0802MB2547; H:pavelb-Z68P-DS3.sw.ru; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0802MB2547; 23:TsqY0zkcRuNj75BJtJ5ZI/gWuoqElvnu7o9cXod?= =?us-ascii?Q?dQ8346A8q0Uq49itlDPws9nD81m8/n6rPvaibKVNBE+mCqYwzr+Ma/2mbb66?= =?us-ascii?Q?hW1CX6eMlIT5dzScYD5VWqtIYsbhjvOqQclwOeoRxf2BvrgBlwrE5Av9VvcC?= =?us-ascii?Q?yShb+yOHvGsGq08qJpjcCEOt/BQFh/yEM7bsdiy6EdnLrvYktFfVu5VPMu12?= =?us-ascii?Q?sLL1WE0Kb9nE8VG9pjI/faJonNCtH+xK9p60wqkKBQHm8s3pqPgp58R/5pzK?= =?us-ascii?Q?bTQKXIRbKdzI9WNhV/O4TNPa9M1DWn+Ws1IBFs+mMJ5kL/i+E/eJPPZaW32s?= =?us-ascii?Q?NJqteV7hkxYVArcZM1kc3WcNXUzEPaRCxlKITvPSjxzG9h5SsbtvD13Ajz/Q?= =?us-ascii?Q?O61ouCja84EhZlfCk/I10KPYi4gSm/8n3TP6l3jf8c7oBvQZ3y1+xQ0B3JQj?= =?us-ascii?Q?UZjVFb1va/sceH5ysaclP98OxTrmSU+aRgqNzXCvbFVvZd8kJrH7OgiWL0S+?= =?us-ascii?Q?SOakYrU81aRcQUCxKKjyvm1S5zYUp5Sldkw7SP29uef8Fq1x+1pVZlZYv+0Q?= =?us-ascii?Q?zMbALjaNExbAcNRbUWouM43QX8/dU9KA/Xtv6/YZfEY8GJpZe9tHFN46hyOK?= =?us-ascii?Q?jsAB15WD7RjEJWW0rWl3lVHBDt5Azfrl6hVbusLqrTBgNyWKOVhWHHBIS5BG?= =?us-ascii?Q?DDVnYn60DCEO7Ur6rAoy42LCb+Gis4fqnqIPHV5RDK+dZHlNxm5GBJsDGCpF?= =?us-ascii?Q?tHW1j7Js3UIDV4gDNOVSVvHziERswTHG5g6fKqxcpdq9pCXk/IvzmPX49gxA?= =?us-ascii?Q?YznQTjpsaD8wHmHktfXLrZiqZILx+8bCdY1fUPOcG3iwOe0HGiVeF1mvrgb0?= =?us-ascii?Q?j5IS6Zn5aF0jAXzRJkaPrpUxSPzegyJ1t+uYE98UyjKd3IDC5Bz8Zzxxj1be?= =?us-ascii?Q?/ulL4TqmJthpac/AhqG2GVxu/eNh85Gjzdw1DjTzBGvUSt2WQwO9TJTA61K0?= =?us-ascii?Q?eCIOCcRzDNCybvRXRPEulTg2wS68sZTtFFyzWGDoS0g3f7LDr5GPx/R7uswN?= =?us-ascii?Q?QoTrJIARuK9yPe8+wn3qKRJjoaWaFAvdlCCR7umRwjWKb75TMUnWKVSmw8QB?= =?us-ascii?Q?sLq2f0nQnFfWGuEwLgMG+/r0XbiLM1zXU4q/FQBqjtww87BWYM3j5j4LX4FG?= =?us-ascii?Q?gzF5BuqYmh0EudL4=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 6:2NfCUPGlvgUhahpdLgpI7Xlyo7lVooTSIZgBPk520gLgu8SvvqWMiTdVVFb8lwpE06aCFISgkY012PtJmDA71F23s0YKzm/Xj3jjLPRXQqSPops9JOmIuzFxP+7MfpaeR24zgnVxRoAUNkRHt+eTLPyrHAcQWL8rUbSzZFP6ZCGVYNMIqE16FwXA3wQbu6f8WLEDdBN1TMZJQnjwISvJBP6N0CUVQKCf7BPOBAzIDhfl5HSagYfH64l2SwpaRnHo5akDfKQY6+lLwXgUJGbkUFUJFR8vmRmYvc1Q7/P4rqy27CqJwBEFhhqz9ya5kuEp; 5:1tDlTZbmrYXtocQdmHEXIQw+UZFp0hDuec3RGQtu+GbEubSihV29+AcRlOZpz3v2VUxsKDc3J1S51zP1jdHUI4wLDb5bTXRB5f8XRlBjXiO6fSSdkA0bi7mjxTAoznOFuzIoCp9lWNRZZz+HXSvymw==; 24:CDOXHBEHpiqtXexa8fIHXPSHpKWTX2L1chAlE1lsTfNRJckgqSbJDu/sYgmmjc47fV4VlwGHI1HQOFbchiZVQ07o+7pyi89aE+dMh6utkFw=; 7:bKuh3aVQIglWyvpSqRw26GSnevpCKMfogzD5Q9FlTTV/RgCw1r/JtFCBeE3QrdyOLsvTcP8+IV4rIz4sd8ctTMmAS0SgieWOjwzBGA180a1ZI5z6TAmDBQMXrScwgp7jZGSP4mMMbgASZTcso2jyiGuUGh4Tw8kk152PgV7MWEBfgUHpX49EhAg8IprSS13LoKhjS03vVV4heTGAw53JcfbBdqdsdc+9GWmJzYGWnwsGq0DxqP6/EojcckYM3Pzs SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0802MB2547; 20:xrqvjLGVDaIEJOshZGzi45UH8elkDt7lG/iLQsreopNJCtxOlt+SGMQRP7Bj+u1zB4pf1rMMtwwmkBQaj1+OOyGXQINUphrJU4dFrRhN34J3uV14Ut73X1wpJu9rRVqTNAj3vc5Qpk0O+OoQX2wLEq2w9M0nNNzTiE6wU5EqNxY= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Aug 2016 17:11:22.1983 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2547 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.3.122 Subject: [Qemu-devel] [PATCH RFC v2 09/22] block/pcache: separation AIOCB on requests X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, famz@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP for case when the cache partially covers request we are part of the request is filled from the cache, and the other part request from disk. Also add reference counting for nodes, as way to maintain multithreading. There is still no full synchronization in multithreaded mode. Signed-off-by: Pavel Butsykin --- block/pcache.c | 169 ++++++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 155 insertions(+), 14 deletions(-) diff --git a/block/pcache.c b/block/pcache.c index 28bd056..6114289 100644 --- a/block/pcache.c +++ b/block/pcache.c @@ -58,7 +58,10 @@ typedef struct BlockNode { typedef struct PCNode { BlockNode cm; + uint32_t status; + uint32_t ref; uint8_t *data; + CoMutex lock; } PCNode; typedef struct ReqStor { @@ -95,9 +98,23 @@ typedef struct PrefCacheAIOCB { uint64_t sector_num; uint32_t nb_sectors; int aio_type; + struct { + QTAILQ_HEAD(req_head, PrefCachePartReq) list; + CoMutex lock; + } requests; int ret; } PrefCacheAIOCB; +typedef struct PrefCachePartReq { + uint64_t sector_num; + uint32_t nb_sectors; + + QEMUIOVector qiov; + PCNode *node; + PrefCacheAIOCB *acb; + QTAILQ_ENTRY(PrefCachePartReq) entry; +} PrefCachePartReq; + static const AIOCBInfo pcache_aiocb_info = { .aiocb_size = sizeof(PrefCacheAIOCB), }; @@ -126,8 +143,39 @@ static QemuOptsList runtime_opts = { #define MB_BITS 20 #define PCACHE_DEFAULT_CACHE_SIZE (4 << MB_BITS) +enum { + NODE_SUCCESS_STATUS = 0, + NODE_WAIT_STATUS = 1, + NODE_REMOVE_STATUS = 2, + NODE_GHOST_STATUS = 3 /* only for debugging */ +}; + #define PCNODE(_n) ((PCNode *)(_n)) +static inline void pcache_node_unref(PCNode *node) +{ + assert(node->status == NODE_SUCCESS_STATUS || + node->status == NODE_REMOVE_STATUS); + + if (atomic_fetch_dec(&node->ref) == 0) { + assert(node->status == NODE_REMOVE_STATUS); + + node->status = NODE_GHOST_STATUS; + g_free(node->data); + g_slice_free1(sizeof(*node), node); + } +} + +static inline PCNode *pcache_node_ref(PCNode *node) +{ + assert(node->status == NODE_SUCCESS_STATUS || + node->status == NODE_WAIT_STATUS); + assert(atomic_read(&node->ref) == 0);/* XXX: only for sequential requests */ + atomic_inc(&node->ref); + + return node; +} + static int pcache_key_cmp(const RbNodeKey *key1, const RbNodeKey *key2) { assert(key1 != NULL); @@ -184,13 +232,7 @@ static void *node_insert(struct RbRoot *root, BlockNode *node) static inline PCNode *pcache_node_insert(struct RbRoot *root, PCNode *node) { - return node_insert(root, &node->cm); -} - -static inline void pcache_node_free(PCNode *node) -{ - g_free(node->data); - g_slice_free1(sizeof(*node), node); + return pcache_node_ref(node_insert(root, &node->cm)); } static inline void *pcache_node_alloc(RbNodeKey* key) @@ -199,6 +241,9 @@ static inline void *pcache_node_alloc(RbNodeKey* key) node->cm.sector_num = key->num; node->cm.nb_sectors = key->size; + node->ref = 0; + node->status = NODE_WAIT_STATUS; + qemu_co_mutex_init(&node->lock); node->data = g_malloc(node->cm.nb_sectors << BDRV_SECTOR_BITS); return node; @@ -206,6 +251,12 @@ static inline void *pcache_node_alloc(RbNodeKey* key) static void pcache_node_drop(BDRVPCacheState *s, PCNode *node) { + uint32_t prev_status = atomic_xchg(&node->status, NODE_REMOVE_STATUS); + if (prev_status == NODE_REMOVE_STATUS) { + return; + } + assert(prev_status != NODE_GHOST_STATUS); + atomic_sub(&s->pcache.curr_size, node->cm.nb_sectors); qemu_co_mutex_lock(&s->pcache.lru.lock); @@ -216,7 +267,7 @@ static void pcache_node_drop(BDRVPCacheState *s, PCNode *node) rb_erase(&node->cm.rb_node, &s->pcache.tree.root); qemu_co_mutex_unlock(&s->pcache.tree.lock); - pcache_node_free(node); + pcache_node_unref(node); } static void pcache_try_shrink(BDRVPCacheState *s) @@ -234,6 +285,30 @@ static void pcache_try_shrink(BDRVPCacheState *s) } } +static PrefCachePartReq *pcache_req_get(PrefCacheAIOCB *acb, PCNode *node) +{ + PrefCachePartReq *req = g_slice_alloc(sizeof(*req)); + + req->nb_sectors = node->cm.nb_sectors; + req->sector_num = node->cm.sector_num; + req->node = node; + req->acb = acb; + + assert(acb->sector_num <= node->cm.sector_num + node->cm.nb_sectors); + + qemu_iovec_init(&req->qiov, 1); + qemu_iovec_add(&req->qiov, node->data, + node->cm.nb_sectors << BDRV_SECTOR_BITS); + return req; +} + +static inline void push_node_request(PrefCacheAIOCB *acb, PCNode *node) +{ + PrefCachePartReq *req = pcache_req_get(acb, node); + + QTAILQ_INSERT_HEAD(&acb->requests.list, req, entry); +} + static inline void pcache_lru_node_up(BDRVPCacheState *s, PCNode *node) { qemu_co_mutex_lock(&s->pcache.lru.lock); @@ -253,16 +328,17 @@ static bool pcache_node_find_and_create(PrefCacheAIOCB *acb, RbNodeKey *key, found = pcache_node_insert(&s->pcache.tree.root, new_node); qemu_co_mutex_unlock(&s->pcache.tree.lock); if (found != new_node) { - pcache_node_free(new_node); - pcache_lru_node_up(s, found); + g_free(new_node->data); + g_slice_free1(sizeof(*new_node), new_node); + if (found->status == NODE_SUCCESS_STATUS) { + pcache_lru_node_up(s, found); + } *out_node = found; return false; } atomic_add(&s->pcache.curr_size, new_node->cm.nb_sectors); - qemu_co_mutex_lock(&s->pcache.lru.lock); - QTAILQ_INSERT_HEAD(&s->pcache.lru.list, &new_node->cm, entry); - qemu_co_mutex_unlock(&s->pcache.lru.lock); + push_node_request(acb, new_node); pcache_try_shrink(s); @@ -291,6 +367,7 @@ static void pcache_pickup_parts_of_cache(PrefCacheAIOCB *acb, PCNode *node, up_size = lc_key.size; if (!pcache_node_find_and_create(acb, &lc_key, &new_node)) { + pcache_node_unref(node); node = new_node; continue; } @@ -300,6 +377,8 @@ static void pcache_pickup_parts_of_cache(PrefCacheAIOCB *acb, PCNode *node, /* XXX: node read */ up_size = MIN(node->cm.sector_num + node->cm.nb_sectors - num, size); + pcache_node_unref(node); + size -= up_size; num += up_size; if (size != 0) { @@ -336,6 +415,8 @@ static int32_t pcache_prefetch(PrefCacheAIOCB *acb) node->cm.sector_num + node->cm.nb_sectors >= acb->sector_num + acb->nb_sectors) { + /* XXX: node read */ + pcache_node_unref(node); return PREFETCH_FULL_UP; } pcache_pickup_parts_of_cache(acb, node, key.num, key.size); @@ -343,10 +424,56 @@ static int32_t pcache_prefetch(PrefCacheAIOCB *acb) return PREFETCH_PART_UP; } +static void pcache_node_submit(PrefCachePartReq *req) +{ + PCNode *node = req->node; + BDRVPCacheState *s = req->acb->s; + + assert(node != NULL); + assert(atomic_read(&node->ref) != 0); + assert(node->data != NULL); + + qemu_co_mutex_lock(&node->lock); + if (node->status == NODE_WAIT_STATUS) { + qemu_co_mutex_lock(&s->pcache.lru.lock); + QTAILQ_INSERT_HEAD(&s->pcache.lru.list, &node->cm, entry); + qemu_co_mutex_unlock(&s->pcache.lru.lock); + + node->status = NODE_SUCCESS_STATUS; + } + qemu_co_mutex_unlock(&node->lock); +} + +static void pcache_merge_requests(PrefCacheAIOCB *acb) +{ + PrefCachePartReq *req, *next; + + qemu_co_mutex_lock(&acb->requests.lock); + QTAILQ_FOREACH_SAFE(req, &acb->requests.list, entry, next) { + QTAILQ_REMOVE(&acb->requests.list, req, entry); + + assert(req != NULL); + assert(req->node->status == NODE_WAIT_STATUS); + + pcache_node_submit(req); + + /* XXX: pcache read */ + + pcache_node_unref(req->node); + + g_slice_free1(sizeof(*req), req); + } + qemu_co_mutex_unlock(&acb->requests.lock); +} + static void pcache_aio_cb(void *opaque, int ret) { PrefCacheAIOCB *acb = opaque; + if (acb->aio_type & QEMU_AIO_READ) { + pcache_merge_requests(acb); + } + acb->common.cb(acb->common.opaque, ret); qemu_aio_unref(acb); @@ -366,6 +493,9 @@ static PrefCacheAIOCB *pcache_aio_get(BlockDriverState *bs, int64_t sector_num, acb->aio_type = type; acb->ret = 0; + QTAILQ_INIT(&acb->requests.list); + qemu_co_mutex_init(&acb->requests.lock); + return acb; } @@ -445,6 +575,17 @@ fail: return ret; } +static void pcache_node_check_and_free(BDRVPCacheState *s, PCNode *node) +{ + assert(node->status == NODE_SUCCESS_STATUS); + assert(node->ref == 0); + + node->status = NODE_REMOVE_STATUS; + rb_erase(&node->cm.rb_node, &s->pcache.tree.root); + g_free(node->data); + g_slice_free1(sizeof(*node), node); +} + static void pcache_close(BlockDriverState *bs) { uint32_t cnt = 0; @@ -452,7 +593,7 @@ static void pcache_close(BlockDriverState *bs) BlockNode *node, *next; QTAILQ_FOREACH_SAFE(node, &s->pcache.lru.list, entry, next) { QTAILQ_REMOVE(&s->pcache.lru.list, node, entry); - pcache_node_free(PCNODE(node)); + pcache_node_check_and_free(s, PCNODE(node)); cnt++; } DPRINTF("used %d nodes\n", cnt);