From patchwork Mon Aug 21 18:05:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 9913453 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9DCC0603F9 for ; Mon, 21 Aug 2017 18:08:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93D07287A5 for ; Mon, 21 Aug 2017 18:08:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 88C5F287AB; Mon, 21 Aug 2017 18:08:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4AF5C287BB for ; Mon, 21 Aug 2017 18:08:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1djr5K-0001hV-AQ; Mon, 21 Aug 2017 18:05:58 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1djr5I-0001gR-SE for xen-devel@lists.xen.org; Mon, 21 Aug 2017 18:05:57 +0000 Received: from [193.109.254.147] by server-2.bemta-6.messagelabs.com id 42/8B-27137-4012B995; Mon, 21 Aug 2017 18:05:56 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrMLMWRWlGSWpSXmKPExsVyuP0Ovy6z4ux Ig0OtQhZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa8b712tYCybkV+xetoqpgXFRSBcjJ4eEgJHE 24n/mLoYuTiEBBYySrStu88IkmATUJXYcP0UK4gtIiAtce3zZUaQImaBx4wSzbv3gBUJCwRK7 H28BcxmAWp42r8KrIFXwFhi+drvTBAb5CU6DkxmAbE5BUwk1u89zAxiCwHVTPg8m20CI/cCRo ZVjBrFqUVlqUW6RhZ6SUWZ6RkluYmZObqGBmZ6uanFxYnpqTmJScV6yfm5mxiBHmYAgh2M59c GHmKU5GBSEuXdkDorUogvKT+lMiOxOCO+qDQntfgQowYHh0DfmtUXGKVY8vLzUpUkeHfJz44U EixKTU+tSMvMAYYgTKkEB4+SCO9qkDRvcUFibnFmOkTqFKOilDivuAJQQgAkkVGaB9cGC/tLj LJSwryMQEcJ8RSkFuVmlqDKv2IU52BUEua9CzKeJzOvBG76K6DFTECLDVungSwuSURISTUwBp vfs4/cfssob/K0RamMWX8UrXfl9B/IvH5u6sVT1xZuS9wS2xV1OEMwf7r3sdStXeEGSxnPqsn P+cDwe/+s/+sTZ8QWz9Z7/PSH3oo2iacKdWvn3Pnk8nrXva0HdHodWQ6L+T3znbWZ58Xv29y3 X831jJ1aceWg9XbHr+KCgZs3K9XX7ttQtUCJpTgj0VCLuag4EQARVOl/dgIAAA== X-Env-Sender: jgross@suse.com X-Msg-Ref: server-12.tower-27.messagelabs.com!1503338755!111317174!1 X-Originating-IP: [195.135.220.15] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 31099 invoked from network); 21 Aug 2017 18:05:55 -0000 Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by server-12.tower-27.messagelabs.com with DHE-RSA-CAMELLIA256-SHA encrypted SMTP; 21 Aug 2017 18:05:55 -0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D38B8ADDF; Mon, 21 Aug 2017 18:05:54 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xen.org Date: Mon, 21 Aug 2017 20:05:48 +0200 Message-Id: <20170821180549.3989-5-jgross@suse.com> X-Mailer: git-send-email 2.12.3 In-Reply-To: <20170821180549.3989-1-jgross@suse.com> References: <20170821180549.3989-1-jgross@suse.com> Cc: Juergen Gross , sstabellini@kernel.org, wei.liu2@citrix.com, George.Dunlap@eu.citrix.com, andrew.cooper3@citrix.com, ian.jackson@eu.citrix.com, tim@xen.org, jbeulich@suse.com Subject: [Xen-devel] [PATCH 4/5] xen: support different gnttab_max_frames for grant v1 and v2 X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The number of grants a domain can setup is limited by the maximum number of grant frames it is allowed to use. Today the limit is the same regardless whether the domain uses grant v1 or v2. Using v2 will therefor be a disadvantage for the domain as only half the number of grants compared to v1 can be used, because a grant v2 entry is twice as large as the v1 entry. This is the reason for the lack of grant v2 support in the Linux kernel (in fact grant v2 support has been removed from Linux for this reason). OTOH using only grant v1 will limit a pv domain to the low 16TB of memory of the host, as grant v1 entries only use a 32 bit mfn. So if we want to support more than 16TB of memory and be able to use that memory in pv domains, we have to remove the disadvantage of using grant v2 by being able to setup the same number of grants as with v1. In order to achieve this add support for limiting the number of grant frames for v1 and v2 independently from each other. Per default let the v2 number be twice the value of the v1 number. Modify the boot parameter gnttab_max_frames to accept either a single value which will set the v1 limit to that value and the v2 limit to 2*value, or two values separated by a comma to set both limits to dedicated values. Add some sanity checks to make sure the maximum number of frames isn't lower than the initial number, as this leads to rather strange crashes. Signed-off-by: Juergen Gross --- xen/arch/arm/domain.c | 2 +- xen/arch/arm/domain_build.c | 2 +- xen/common/compat/grant_table.c | 9 ++-- xen/common/grant_table.c | 107 ++++++++++++++++++++++++++------------ xen/include/asm-arm/grant_table.h | 3 +- xen/include/xen/grant_table.h | 5 +- 6 files changed, 84 insertions(+), 44 deletions(-) diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index eeebbdb39a..5a0bec0b4b 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -469,7 +469,7 @@ struct domain *alloc_domain_struct(void) return NULL; clear_page(d); - d->arch.grant_table_gfn = xzalloc_array(gfn_t, max_grant_frames); + d->arch.grant_table_gfn = xzalloc_array(gfn_t, max_grant_frames()); return d; } diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 1bec4fa23d..ffc5e5e87c 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -2095,7 +2095,7 @@ static void __init find_gnttab_region(struct domain *d, kinfo->gnttab_size = (_etext - _stext) & PAGE_MASK; /* Make sure the grant table will fit in the region */ - if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames ) + if ( (kinfo->gnttab_size >> PAGE_SHIFT) < max_grant_frames() ) panic("Cannot find a space for the grant table region\n"); #ifdef CONFIG_ARM_32 diff --git a/xen/common/compat/grant_table.c b/xen/common/compat/grant_table.c index f8c60a1bdf..a437340215 100644 --- a/xen/common/compat/grant_table.c +++ b/xen/common/compat/grant_table.c @@ -157,11 +157,12 @@ int compat_grant_table_op(unsigned int cmd, unsigned int max_frame_list_size_in_page = (COMPAT_ARG_XLAT_SIZE - sizeof(*nat.setup)) / sizeof(*nat.setup->frame_list.p); - if ( max_frame_list_size_in_page < max_grant_frames ) + + if ( max_frame_list_size_in_page < max_grant_frames() ) { gdprintk(XENLOG_WARNING, "max_grant_frames is too large (%u,%u)\n", - max_grant_frames, max_frame_list_size_in_page); + max_grant_frames(), max_frame_list_size_in_page); rc = -EINVAL; } else @@ -295,11 +296,11 @@ int compat_grant_table_op(unsigned int cmd, break; } if ( max_frame_list_size_in_pages < - grant_to_status_frames(max_grant_frames) ) + grant_to_status_frames(max_grant_frames_v2) ) { gdprintk(XENLOG_WARNING, "grant_to_status_frames(max_grant_frames) is too large (%u,%u)\n", - grant_to_status_frames(max_grant_frames), + grant_to_status_frames(max_grant_frames_v2), max_frame_list_size_in_pages); rc = -EINVAL; break; diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index 01d09945da..33cf1f9ea2 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -76,8 +76,34 @@ struct grant_table { #define DEFAULT_MAX_NR_GRANT_FRAMES 32 #endif -unsigned int __read_mostly max_grant_frames; -integer_param("gnttab_max_frames", max_grant_frames); +static unsigned int __read_mostly max_grant_frames_v1; +static unsigned int __read_mostly max_grant_frames_v2; + +static int set_max_grant_frames(const char *str) +{ + const char *s; + unsigned int val; + + val = simple_strtoul(str, &s, 0); + if ( val < INITIAL_NR_GRANT_FRAMES ) + return -EINVAL; + max_grant_frames_v1 = val; + max_grant_frames_v2 = 2 * max_grant_frames_v1; + if ( !*s ) + return 0; + + if ( *s != ',' ) + return -EINVAL; + + val = simple_strtoul(s + 1, &s, 0); + if ( val < INITIAL_NR_GRANT_FRAMES ) + return -EINVAL; + max_grant_frames_v2 = val; + + return *s ? -EINVAL : 0; +} + +custom_param("gnttab_max_frames", set_max_grant_frames); /* The maximum number of grant mappings is defined as a multiplier of the * maximum number of grant table entries. This defines the multiplier used. @@ -274,24 +300,33 @@ static inline void gnttab_flush_tlb(const struct domain *d) } static inline unsigned int -num_act_frames_from_sha_frames(const unsigned int num) +num_act_frames_from_sha_frames(const unsigned int num, unsigned int version) { /* * How many frames are needed for the active grant table, * given the size of the shared grant table? */ - unsigned int sha_per_page = PAGE_SIZE / sizeof(grant_entry_v1_t); + unsigned int sha_per_page; + sha_per_page = (version == 2) ? SHGNT_PER_PAGE_V2 : SHGNT_PER_PAGE_V1; return DIV_ROUND_UP(num * sha_per_page, ACGNT_PER_PAGE); } -#define max_nr_active_grant_frames \ - num_act_frames_from_sha_frames(max_grant_frames) - static inline unsigned int nr_active_grant_frames(struct grant_table *gt) { - return num_act_frames_from_sha_frames(nr_grant_frames(gt)); + return num_act_frames_from_sha_frames(nr_grant_frames(gt), gt->gt_version); +} + +unsigned int max_grant_frames(void) +{ + return max(max_grant_frames_v1, max_grant_frames_v2); +} + +static inline unsigned int +max_grant_frames_v(struct grant_table *t) +{ + return (t->gt_version == 2) ? max_grant_frames_v2 : max_grant_frames_v1; } static inline struct active_grant_entry * @@ -1663,7 +1698,7 @@ gnttab_grow_table(struct domain *d, unsigned int req_nr_frames) struct grant_table *gt = d->grant_table; unsigned int i, j; - ASSERT(req_nr_frames <= max_grant_frames); + ASSERT(req_nr_frames <= max_grant_frames_v(gt)); gdprintk(XENLOG_INFO, "Expanding dom (%d) grant table from (%d) to (%d) frames.\n", @@ -1671,7 +1706,8 @@ gnttab_grow_table(struct domain *d, unsigned int req_nr_frames) /* Active */ for ( i = nr_active_grant_frames(gt); - i < num_act_frames_from_sha_frames(req_nr_frames); i++ ) + i < num_act_frames_from_sha_frames(req_nr_frames, gt->gt_version); + i++ ) { if ( (gt->active[i] = alloc_xenheap_page()) == NULL ) goto active_alloc_failed; @@ -1710,7 +1746,8 @@ shared_alloc_failed: } active_alloc_failed: for ( i = nr_active_grant_frames(gt); - i < num_act_frames_from_sha_frames(req_nr_frames); i++ ) + i < num_act_frames_from_sha_frames(req_nr_frames, gt->gt_version); + i++ ) { free_xenheap_page(gt->active[i]); gt->active[i] = NULL; @@ -1735,15 +1772,6 @@ gnttab_setup_table( if ( unlikely(copy_from_guest(&op, uop, 1)) ) return -EFAULT; - if ( unlikely(op.nr_frames > max_grant_frames) ) - { - gdprintk(XENLOG_INFO, "Xen only supports up to %d grant-table frames" - " per domain.\n", - max_grant_frames); - op.status = GNTST_general_error; - goto out; - } - if ( !guest_handle_okay(op.frame_list, op.nr_frames) ) return -EFAULT; @@ -1763,6 +1791,15 @@ gnttab_setup_table( gt = d->grant_table; grant_write_lock(gt); + if ( unlikely(op.nr_frames > max_grant_frames_v(gt)) ) + { + gdprintk(XENLOG_INFO, "Xen only supports up to %d grant-table frames" + " per domain.\n", + max_grant_frames_v(gt)); + op.status = GNTST_general_error; + goto unlock; + } + if ( gt->gt_version == 0 ) gt->gt_version = 1; @@ -1773,7 +1810,7 @@ gnttab_setup_table( { gdprintk(XENLOG_INFO, "Expand grant table to %u failed. Current: %u Max: %u\n", - op.nr_frames, nr_grant_frames(gt), max_grant_frames); + op.nr_frames, nr_grant_frames(gt), max_grant_frames_v(gt)); op.status = GNTST_general_error; goto unlock; } @@ -1831,7 +1868,7 @@ gnttab_query_size( grant_read_lock(d->grant_table); op.nr_frames = nr_grant_frames(d->grant_table); - op.max_nr_frames = max_grant_frames; + op.max_nr_frames = max_grant_frames_v(d->grant_table); op.status = GNTST_okay; grant_read_unlock(d->grant_table); @@ -3378,7 +3415,7 @@ grant_table_create( struct domain *d) { struct grant_table *t; - unsigned int i, j; + unsigned int i, j, max_frames; if ( (t = xzalloc(struct grant_table)) == NULL ) goto no_mem_0; @@ -3389,11 +3426,12 @@ grant_table_create( t->nr_grant_frames = INITIAL_NR_GRANT_FRAMES; /* Active grant table. */ + max_frames = max(num_act_frames_from_sha_frames(max_grant_frames_v1, 1), + num_act_frames_from_sha_frames(max_grant_frames_v2, 2)); if ( (t->active = xzalloc_array(struct active_grant_entry *, - max_nr_active_grant_frames)) == NULL ) + max_frames)) == NULL ) goto no_mem_1; - for ( i = 0; - i < num_act_frames_from_sha_frames(INITIAL_NR_GRANT_FRAMES); i++ ) + for ( i = 0; i < max_frames; i++ ) { if ( (t->active[i] = alloc_xenheap_page()) == NULL ) goto no_mem_2; @@ -3408,7 +3446,7 @@ grant_table_create( goto no_mem_2; /* Shared grant table. */ - if ( (t->shared_raw = xzalloc_array(void *, max_grant_frames)) == NULL ) + if ( (t->shared_raw = xzalloc_array(void *, max_grant_frames())) == NULL ) goto no_mem_3; for ( i = 0; i < INITIAL_NR_GRANT_FRAMES; i++ ) { @@ -3419,7 +3457,7 @@ grant_table_create( /* Status pages for grant table - for version 2 */ t->status = xzalloc_array(grant_status_t *, - grant_to_status_frames(max_grant_frames)); + grant_to_status_frames(max_grant_frames_v2)); if ( t->status == NULL ) goto no_mem_4; @@ -3439,8 +3477,7 @@ grant_table_create( no_mem_3: vfree(t->maptrack); no_mem_2: - for ( i = 0; - i < num_act_frames_from_sha_frames(INITIAL_NR_GRANT_FRAMES); i++ ) + for ( i = 0; i < max_frames; i++ ) free_xenheap_page(t->active[i]); xfree(t->active); no_mem_1: @@ -3706,7 +3743,7 @@ int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, else { if ( (idx >= nr_grant_frames(d->grant_table)) && - (idx < max_grant_frames) ) + (idx < max_grant_frames_v(d->grant_table)) ) gnttab_grow_table(d, idx + 1); if ( idx < nr_grant_frames(d->grant_table) ) @@ -3794,10 +3831,12 @@ static int __init gnttab_usage_init(void) { BUILD_BUG_ON(DEFAULT_MAX_MAPTRACK_FRAMES < DEFAULT_MAX_NR_GRANT_FRAMES); - if ( !max_grant_frames ) - max_grant_frames = DEFAULT_MAX_NR_GRANT_FRAMES; + if ( !max_grant_frames_v1 ) + max_grant_frames_v1 = DEFAULT_MAX_NR_GRANT_FRAMES; + if ( !max_grant_frames_v2 ) + max_grant_frames_v2 = 2 * DEFAULT_MAX_NR_GRANT_FRAMES; - if ( !max_maptrack_frames ) + if ( max_maptrack_frames < INITIAL_NR_GRANT_FRAMES ) max_maptrack_frames = DEFAULT_MAX_MAPTRACK_FRAMES; register_keyhandler('g', gnttab_usage_print_all, diff --git a/xen/include/asm-arm/grant_table.h b/xen/include/asm-arm/grant_table.h index 581c4bf928..2b51dc954a 100644 --- a/xen/include/asm-arm/grant_table.h +++ b/xen/include/asm-arm/grant_table.h @@ -36,7 +36,8 @@ static inline void gnttab_set_frame_gfn(struct domain *d, unsigned long idx, #define gnttab_shared_gmfn(d, t, i) \ ( ((i >= nr_grant_frames(d->grant_table)) && \ - (i < max_grant_frames)) ? 0 : gfn_x(d->arch.grant_table_gfn[i])) + (i < max_grant_frames_v(d->grant_table))) \ + ? 0 : gfn_x(d->arch.grant_table_gfn[i])) #define gnttab_need_iommu_mapping(d) \ (is_domain_direct_mapped(d) && need_iommu(d)) diff --git a/xen/include/xen/grant_table.h b/xen/include/xen/grant_table.h index 06faf61a6b..e003cb39f3 100644 --- a/xen/include/xen/grant_table.h +++ b/xen/include/xen/grant_table.h @@ -29,9 +29,6 @@ #include #include -/* The maximum size of a grant table. */ -extern unsigned int __read_mostly max_grant_frames; - /* Create/destroy per-domain grant table context. */ int grant_table_create( struct domain *d); @@ -55,4 +52,6 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref, int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn, mfn_t *mfn); +unsigned int max_grant_frames(void); + #endif /* __XEN_GRANT_TABLE_H__ */