From patchwork Wed Sep 28 09:42:17 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konrad Rzeszutek Wilk X-Patchwork-Id: 9353663 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EC23A6077A for ; Wed, 28 Sep 2016 09:44:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB71629478 for ; Wed, 28 Sep 2016 09:44:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CFD012947C; Wed, 28 Sep 2016 09:44:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 99A4129478 for ; Wed, 28 Sep 2016 09:44:56 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bpBO1-00004v-8p; Wed, 28 Sep 2016 09:42:45 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bpBO0-0008Vk-AU for xen-devel@lists.xenproject.org; Wed, 28 Sep 2016 09:42:44 +0000 Received: from [193.109.254.147] by server-5.bemta-6.messagelabs.com id DB/A2-29563-3909BE75; Wed, 28 Sep 2016 09:42:43 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBLMWRWlGSWpSXmKPExsXSO6nOVXfihNf hBsc3mFt83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBl//t9gK9gTUnH77V3mBsYHDl2MXBxCAh1M Ev0961ggnG+MEtd/3WWFcDYySnz9+AsowwnkTGCUaLzn18XIwcEmYCLxZpUjSFhEwFRi+8STY CXMAm4Sh89/ZAexhQW8JWZuX8QIYrMIqEpcu3yIDcTmBao5/uoXG8gYCQF5iWe360FMTgF3if uHfSEWuUl8b3gMNlFCwFiib1YfywRGvgWMDKsY1YtTi8pSi3TN9ZKKMtMzSnITM3N0DQ3M9HJ Ti4sT01NzEpOK9ZLzczcxAkOEAQh2MM687H+IUZKDSUmUd3P463AhvqT8lMqMxOKM+KLSnNTi Q4wyHBxKErwqwJATEixKTU+tSMvMAQYrTFqCg0dJhFcQJM1bXJCYW5yZDpE6xagoJc77sB8oI QCSyCjNg2uDRcglRlkpYV5GoEOEeApSi3IzS1DlXzGKczAqCfPKgoznycwrgZv+CmgxE9DipS degCwuSURISTUwbvfdq3QzriF9p+LttZ9uJQvMP5KX9f04R29T4i6B3S99/npz/SsTnfF71sv aonXtrNJvBW0FC3y//VqyrWfT3V3CMZMDM9nuxvfoxAgKPZl5eRd/hAzPvNxNH9ds+6iec+SD fvMZRu4nR+boStx6vZD748y2uECpbpm3968JXl0554lCU+zFDCWW4oxEQy3mouJEAM6Yjx6LA gAA X-Env-Sender: konrad.wilk@oracle.com X-Msg-Ref: server-9.tower-27.messagelabs.com!1475055760!61918365!1 X-Originating-IP: [141.146.126.69] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTQxLjE0Ni4xMjYuNjkgPT4gMjc3MjE4\n X-StarScan-Received: X-StarScan-Version: 8.84; banners=-,-,- X-VirusChecked: Checked Received: (qmail 16420 invoked from network); 28 Sep 2016 09:42:41 -0000 Received: from aserp1040.oracle.com (HELO aserp1040.oracle.com) (141.146.126.69) by server-9.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 28 Sep 2016 09:42:41 -0000 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u8S9gbRR028678 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Sep 2016 09:42:38 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u8S9gbcq009231 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 28 Sep 2016 09:42:37 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u8S9gZTC019382; Wed, 28 Sep 2016 09:42:36 GMT Received: from localhost.localdomain.com (/209.6.196.81) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Sep 2016 02:42:35 -0700 From: Konrad Rzeszutek Wilk To: xen-devel@lists.xenproject.org, konrad@kernel.org Date: Wed, 28 Sep 2016 05:42:17 -0400 Message-Id: <1475055746-22401-4-git-send-email-konrad.wilk@oracle.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1475055746-22401-1-git-send-email-konrad.wilk@oracle.com> References: <1475055746-22401-1-git-send-email-konrad.wilk@oracle.com> X-Source-IP: aserv0021.oracle.com [141.146.126.233] Cc: Konrad Rzeszutek Wilk Subject: [Xen-devel] [PATCH v1 03/12] tmem: Wrap tmem dedup code with CONFIG_TMEM_DEDUP X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This config is not defined anywhere but it makes it way easier to figure out what code to deal with. Signed-off-by: Konrad Rzeszutek Wilk --- v1: First submission. --- xen/common/tmem.c | 57 ++++++++++++++++++++++++++++++++++++++++++---- xen/common/tmem_control.c | 2 ++ xen/common/tmem_xen.c | 2 ++ xen/include/xen/tmem_xen.h | 4 ++++ 4 files changed, 60 insertions(+), 5 deletions(-) diff --git a/xen/common/tmem.c b/xen/common/tmem.c index efaafc4..8327218 100644 --- a/xen/common/tmem.c +++ b/xen/common/tmem.c @@ -71,10 +71,14 @@ struct tmem_page_descriptor { pagesize_t size; /* 0 == PAGE_SIZE (pfp), -1 == data invalid, else compressed data (cdata). */ uint32_t index; +#ifdef CONFIG_TMEM_DEDUP /* Must hold pcd_tree_rwlocks[firstbyte] to use pcd pointer/siblings. */ uint16_t firstbyte; /* NON_SHAREABLE->pfp otherwise->pcd. */ +#endif bool_t eviction_attempted; /* CHANGE TO lifetimes? (settable). */ +#ifdef CONFIG_TMEM_DEDUP struct list_head pcd_siblings; +#endif union { struct page_info *pfp; /* Page frame pointer. */ char *cdata; /* Compressed data. */ @@ -94,15 +98,20 @@ struct tmem_page_content_descriptor { char *cdata; /* If compression_enabled. */ char *tze; /* If !compression_enabled, trailing zeroes eliminated. */ }; +#ifdef CONFIG_TMEM_DEDUP struct list_head pgp_list; struct rb_node pcd_rb_tree_node; uint32_t pgp_ref_count; +#endif pagesize_t size; /* If compression_enabled -> 0 *pfp. */ }; + +#ifdef CONFIG_TMEM_DEDUP struct rb_root pcd_tree_roots[256]; /* Choose based on first byte of page. */ rwlock_t pcd_tree_rwlocks[256]; /* Poor man's concurrency for now. */ +#endif static int tmem_initialized = 0; @@ -262,6 +271,7 @@ static void tmem_persistent_pool_page_put(void *page_va) */ #define NOT_SHAREABLE ((uint16_t)-1UL) +#ifdef CONFIG_TMEM_DEDUP static int pcd_copy_to_client(xen_pfn_t cmfn, struct tmem_page_descriptor *pgp) { uint8_t firstbyte = pgp->firstbyte; @@ -485,6 +495,7 @@ unlock: write_unlock(&pcd_tree_rwlocks[firstbyte]); return ret; } +#endif /************ PAGE DESCRIPTOR MANIPULATION ROUTINES *******************/ @@ -503,12 +514,14 @@ static struct tmem_page_descriptor *pgp_alloc(struct tmem_object_root *obj) INIT_LIST_HEAD(&pgp->global_eph_pages); INIT_LIST_HEAD(&pgp->us.client_eph_pages); pgp->pfp = NULL; +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() ) { pgp->firstbyte = NOT_SHAREABLE; pgp->eviction_attempted = 0; INIT_LIST_HEAD(&pgp->pcd_siblings); } +#endif pgp->size = -1; pgp->index = -1; pgp->timestamp = get_cycles(); @@ -533,9 +546,12 @@ static void pgp_free_data(struct tmem_page_descriptor *pgp, struct tmem_pool *po if ( pgp->pfp == NULL ) return; +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() && pgp->firstbyte != NOT_SHAREABLE ) pcd_disassociate(pgp,pool,0); /* pgp->size lost. */ - else if ( pgp_size ) + else +#endif + if ( pgp_size ) tmem_free(pgp->cdata, pool); else tmem_free_page(pgp->us.obj->pool,pgp->pfp); @@ -1085,6 +1101,8 @@ static struct client *client_create(domid_t cli_id) client->cli_id = cli_id; client->compress = tmem_compression_enabled(); +#ifdef CONFIG_TMEM_DEDUP +#endif client->shared_auth_required = tmem_shared_auth(); for ( i = 0; i < MAX_GLOBAL_SHARED_POOLS; i++) client->shared_auth_uuid[i][0] = @@ -1141,13 +1159,16 @@ static bool_t tmem_try_to_evict_pgp(struct tmem_page_descriptor *pgp, bool_t *ho { struct tmem_object_root *obj = pgp->us.obj; struct tmem_pool *pool = obj->pool; +#ifdef CONFIG_TMEM_DEDUP struct client *client = pool->client; uint16_t firstbyte = pgp->firstbyte; +#endif if ( pool->is_dying ) return 0; if ( spin_trylock(&obj->obj_spinlock) ) { +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() ) { firstbyte = pgp->firstbyte; @@ -1166,6 +1187,7 @@ static bool_t tmem_try_to_evict_pgp(struct tmem_page_descriptor *pgp, bool_t *ho goto pcd_unlock; } } +#endif if ( obj->pgp_count > 1 ) return 1; if ( write_trylock(&pool->pool_rwlock) ) @@ -1173,10 +1195,12 @@ static bool_t tmem_try_to_evict_pgp(struct tmem_page_descriptor *pgp, bool_t *ho *hold_pool_rwlock = 1; return 1; } +#ifdef CONFIG_TMEM_DEDUP pcd_unlock: if ( tmem_dedup_enabled() ) write_unlock(&pcd_tree_rwlocks[firstbyte]); obj_unlock: +#endif spin_unlock(&obj->obj_spinlock); } return 0; @@ -1232,11 +1256,13 @@ found: ASSERT_SPINLOCK(&obj->obj_spinlock); pgp_del = pgp_delete_from_obj(obj, pgp->index); ASSERT(pgp_del == pgp); +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() && pgp->firstbyte != NOT_SHAREABLE ) { ASSERT(pgp->pcd->pgp_ref_count == 1 || pgp->eviction_attempted); pcd_disassociate(pgp,pool,1); } +#endif /* pgp already delist, so call pgp_free directly. */ pgp_free(pgp); @@ -1303,9 +1329,11 @@ static int do_tmem_put_compress(struct tmem_page_descriptor *pgp, xen_pfn_t cmfn else if ( (size == 0) || (size >= tmem_mempool_maxalloc) ) { ret = 0; goto out; +#ifdef CONFIG_TMEM_DEDUP } else if ( tmem_dedup_enabled() && !is_persistent(pgp->us.obj->pool) ) { if ( (ret = pcd_associate(pgp,dst,size)) == -ENOMEM ) goto out; +#endif } else if ( (p = tmem_malloc(size,pgp->us.obj->pool)) == NULL ) { ret = -ENOMEM; goto out; @@ -1365,11 +1393,13 @@ copy_uncompressed: ret = tmem_copy_from_client(pgp->pfp, cmfn, tmem_cli_buf_null); if ( ret < 0 ) goto bad_copy; +#ifdef CONFIG_PAGE_DEDUP if ( tmem_dedup_enabled() && !is_persistent(pool) ) { if ( pcd_associate(pgp,NULL,0) == -ENOMEM ) goto failed_dup; } +#endif done: /* Successfully replaced data, clean up and return success. */ @@ -1506,6 +1536,7 @@ copy_uncompressed: if ( ret < 0 ) goto bad_copy; +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() && !is_persistent(pool) ) { if ( pcd_associate(pgp, NULL, 0) == -ENOMEM ) @@ -1514,6 +1545,7 @@ copy_uncompressed: goto del_pgp_from_obj; } } +#endif insert_page: if ( !is_persistent(pool) ) @@ -1601,10 +1633,13 @@ static int do_tmem_get(struct tmem_pool *pool, return 0; } ASSERT(pgp->size != -1); +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() && !is_persistent(pool) && pgp->firstbyte != NOT_SHAREABLE ) rc = pcd_copy_to_client(cmfn, pgp); - else if ( pgp->size != 0 ) + else +#endif + if ( pgp->size != 0 ) { rc = tmem_decompress_to_client(cmfn, pgp->cdata, pgp->size, clibuf); } @@ -2362,29 +2397,41 @@ unsigned long tmem_freeable_pages(void) /* Called at hypervisor startup. */ static int __init init_tmem(void) { - int i; if ( !tmem_enabled() ) return 0; +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() ) + { + unsigned int i; + for (i = 0; i < 256; i++ ) { pcd_tree_roots[i] = RB_ROOT; rwlock_init(&pcd_tree_rwlocks[i]); } - + } +#endif if ( !tmem_mempool_init() ) return 0; if ( tmem_init() ) { printk("tmem: initialized comp=%d dedup=%d tze=%d\n", - tmem_compression_enabled(), tmem_dedup_enabled(), tmem_tze_enabled()); + tmem_compression_enabled(), +#ifdef CONFIG_TMEM_DEDUP + tmem_dedup_enabled(), +#else + 0, +#endif + tmem_tze_enabled()); +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled()&&tmem_compression_enabled()&&tmem_tze_enabled() ) { tmem_tze_disable(); printk("tmem: tze and compression not compatible, disabling tze\n"); } +#endif tmem_initialized = 1; } else diff --git a/xen/common/tmem_control.c b/xen/common/tmem_control.c index ca34852..565e50b 100644 --- a/xen/common/tmem_control.c +++ b/xen/common/tmem_control.c @@ -274,6 +274,7 @@ static int __tmemc_set_var(struct client *client, uint32_t subop, uint32_t arg1) atomic_add(client->weight,&tmem_global.client_weight_total); break; case XEN_SYSCTL_TMEM_OP_SET_COMPRESS: +#ifdef CONFIG_TMEM_DEDUP if ( tmem_dedup_enabled() ) { tmem_client_warn("tmem: compression %s for all %ss, cannot be changed when tmem_dedup is enabled\n", @@ -281,6 +282,7 @@ static int __tmemc_set_var(struct client *client, uint32_t subop, uint32_t arg1) tmem_client_str); return -1; } +#endif client->compress = arg1 ? 1 : 0; tmem_client_info("tmem: compression %s for %s=%d\n", arg1 ? "enabled" : "disabled",tmem_cli_id_str,cli_id); diff --git a/xen/common/tmem_xen.c b/xen/common/tmem_xen.c index 71cb7d5..9a2bfed 100644 --- a/xen/common/tmem_xen.c +++ b/xen/common/tmem_xen.c @@ -20,8 +20,10 @@ boolean_param("tmem", opt_tmem); bool_t __read_mostly opt_tmem_compress = 0; boolean_param("tmem_compress", opt_tmem_compress); +#ifdef CONFIG_TMEM_DEDUP bool_t __read_mostly opt_tmem_dedup = 0; boolean_param("tmem_dedup", opt_tmem_dedup); +#endif bool_t __read_mostly opt_tmem_tze = 0; boolean_param("tmem_tze", opt_tmem_tze); diff --git a/xen/include/xen/tmem_xen.h b/xen/include/xen/tmem_xen.h index 582951a..52f87b1 100644 --- a/xen/include/xen/tmem_xen.h +++ b/xen/include/xen/tmem_xen.h @@ -41,11 +41,13 @@ static inline bool_t tmem_compression_enabled(void) return opt_tmem_compress; } +#ifdef CONFIG_TMEM_DEDUP extern bool_t opt_tmem_dedup; static inline bool_t tmem_dedup_enabled(void) { return opt_tmem_dedup; } +#endif extern bool_t opt_tmem_tze; static inline bool_t tmem_tze_enabled(void) @@ -192,6 +194,7 @@ static inline struct client *tmem_client_from_cli_id(domid_t cli_id) return c; } +#ifdef CONFIG_TMEM_DEDUP static inline uint8_t tmem_get_first_byte(struct page_info *pfp) { const uint8_t *p = __map_domain_page(pfp); @@ -288,6 +291,7 @@ static inline void tmem_tze_copy_from_pfp(void *tva, struct page_info *pfp, page unmap_domain_page(p); } +#endif /* these typedefs are in the public/tmem.h interface typedef XEN_GUEST_HANDLE(void) cli_mfn_t;