From patchwork Wed Aug 9 07:41:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9889789 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 99051603F2 for ; Wed, 9 Aug 2017 08:01:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B3AE28A09 for ; Wed, 9 Aug 2017 08:01:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7FF1828A37; Wed, 9 Aug 2017 08:01:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BF7C628A09 for ; Wed, 9 Aug 2017 08:01:21 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dfLsf-0001o2-NC; Wed, 09 Aug 2017 07:58:17 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dfLsf-0001nH-0X for xen-devel@lists.xenproject.org; Wed, 09 Aug 2017 07:58:17 +0000 Received: from [193.109.254.147] by server-6.bemta-6.messagelabs.com id 4B/6E-03937-890CA895; Wed, 09 Aug 2017 07:58:16 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrFLMWRWlGSWpSXmKPExsXS1tYhoTv9QFe kwZJd8hbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8bvg6dYC/ZHVez+9pu1gXG6UxcjF4eQwDRG iVl7N7F3MXJySAjwShxZNoMVwvaX2HP2HyOILSTQwCixfIIyiM0moC7x+GsPE4gtIqAkcW/VZ CaQQcwC25gk3l9+ATSIg0NYwF5ib7cpSA2LgKpE+4dWZhCbV8BdYuvNF8wQ8+UkTh6bDLaLU8 BDYnPHHzaIXe4S33Z1s09g5F3AyLCKUaM4tagstUjX2FAvqSgzPaMkNzEzR9fQwEwvN7W4ODE 9NScxqVgvOT93EyMwHBiAYAdj06LAQ4ySHExKorybtDsjhfiS8lMqMxKLM+KLSnNSiw8xynBw KEnwXtrfFSkkWJSanlqRlpkDDEyYtAQHj5IIbzlImre4IDG3ODMdInWKUZfj1YT/35iEWPLy8 1KlxHn3gxQJgBRllObBjYBFySVGWSlhXkago4R4ClKLcjNLUOVfMYpzMCoJ804CmcKTmVcCt+ kV0BFMQEdE+HaCHFGSiJCSamDktShXMOjf4n34/rN7Zruu78v+c3I2X9rrJdLpC0M7/wXphkT d9LklsFCwRlmHf/rTxjTrnzlCl+u6EpW4c8XOzA8u/cm/kv/hf3Xz7Xd3VeR+jzqsK/jX6M2p R5H/JCI2pXPsTf0xb9mLa59Esxn/nr79Qv7N9yuHvTuzT3548+PMIucDVjsvKLEUZyQaajEXF ScCAG1NqvWNAgAA X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-11.tower-27.messagelabs.com!1502265470!79168193!8 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 65381 invoked from network); 9 Aug 2017 07:58:14 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-11.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 9 Aug 2017 07:58:14 -0000 Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Aug 2017 00:58:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.41,346,1498546800"; d="scan'208";a="117183025" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.80]) by orsmga004.jf.intel.com with ESMTP; 09 Aug 2017 00:58:11 -0700 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Wed, 9 Aug 2017 15:41:46 +0800 Message-Id: <1502264512-4648-8-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1502264512-4648-1-git-send-email-yi.y.sun@linux.intel.com> References: <1502264512-4648-1-git-send-email-yi.y.sun@linux.intel.com> Cc: kevin.tian@intel.com, wei.liu2@citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, ian.jackson@eu.citrix.com, Yi Sun , julien.grall@arm.com, mengxu@cis.upenn.edu, jbeulich@suse.com, chao.p.peng@linux.intel.com, roger.pau@citrix.com Subject: [Xen-devel] [PATCH v1 07/13] x86: implement set value flow for MBA X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch implements set value flow for MBA including its callback function and domctl interface. It also changes the memebers in 'cos_write_info' to transfer the feature array, feature properties array and value array. Then, we can write all features values on the cos id into MSRs. Signed-off-by: Yi Sun --- v1: - rename 'check_change_val' to 'check_val'. (suggested by Chao Peng) - rename 'cat_check_change_val' to 'cat_check_cbm'. (suggested by Chao Peng) - rename 'mba_check_change_val' to 'mba_check_thrtl'. (suggested by Chao Peng) --- xen/arch/x86/domctl.c | 6 ++ xen/arch/x86/psr.c | 182 ++++++++++++++++++++++++++++++++------------ xen/include/public/domctl.h | 1 + 3 files changed, 141 insertions(+), 48 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index fa5e6d4..0aa9f34 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1468,6 +1468,12 @@ long arch_do_domctl( PSR_VAL_TYPE_L2); break; + case XEN_DOMCTL_PSR_MBA_OP_SET_THRTL: + ret = psr_set_val(d, domctl->u.psr_alloc_op.target, + domctl->u.psr_alloc_op.data, + PSR_VAL_TYPE_MBA); + break; + case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM: ret = psr_get_val(d, domctl->u.psr_alloc_op.target, &val32, PSR_VAL_TYPE_L3); diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 9455e67..bced251 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -138,6 +138,12 @@ static const struct feat_props { /* write_msr is used to write out feature MSR register. */ void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type); + + /* + * check_val is used to check if input val fulfills SDM requirement. + * Change it to valid value if SDM allows. + */ + bool (*check_val)(const struct feat_node *feat, unsigned long *val); } *feat_props[FEAT_TYPE_NUM]; /* @@ -275,29 +281,6 @@ static enum psr_feat_type psr_val_type_to_feat_type(enum psr_val_type type) return feat_type; } -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm) -{ - unsigned int first_bit, zero_bit; - - /* Set bits should only in the range of [0, cbm_len]. */ - if ( cbm & (~0ul << cbm_len) ) - return false; - - /* At least one bit need to be set. */ - if ( cbm == 0 ) - return false; - - first_bit = find_first_bit(&cbm, cbm_len); - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit); - - /* Set bits should be contiguous. */ - if ( zero_bit < cbm_len && - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len ) - return false; - - return true; -} - /* Implementation of allocation features' functions. */ static int cat_init_feature(const struct cpuid_leaf *regs, struct feat_node *feat, @@ -430,6 +413,30 @@ static bool cat_get_feat_info(const struct feat_node *feat, return true; } +static bool cat_check_cbm(const struct feat_node *feat, unsigned long *cbm) +{ + unsigned int first_bit, zero_bit; + unsigned int cbm_len = feat->cat_info.cbm_len; + + /* Set bits should only in the range of [0, cbm_len]. */ + if ( *cbm & (~0ul << cbm_len) ) + return false; + + /* At least one bit need to be set. */ + if ( *cbm == 0 ) + return false; + + first_bit = find_first_bit(cbm, cbm_len); + zero_bit = find_next_zero_bit(cbm, cbm_len, first_bit); + + /* Set bits should be contiguous. */ + if ( zero_bit < cbm_len && + find_next_bit(cbm, cbm_len, zero_bit) < cbm_len ) + return false; + + return true; +} + /* L3 CAT props */ static void l3_cat_write_msr(unsigned int cos, uint32_t val, enum psr_val_type type) @@ -443,6 +450,7 @@ static const struct feat_props l3_cat_props = { .alt_type = PSR_VAL_TYPE_UNKNOWN, .get_feat_info = cat_get_feat_info, .write_msr = l3_cat_write_msr, + .check_val = cat_check_cbm, }; /* L3 CDP props */ @@ -473,6 +481,7 @@ static const struct feat_props l3_cdp_props = { .alt_type = PSR_VAL_TYPE_L3, .get_feat_info = l3_cdp_get_feat_info, .write_msr = l3_cdp_write_msr, + .check_val = cat_check_cbm, }; /* L2 CAT props */ @@ -488,6 +497,7 @@ static const struct feat_props l2_cat_props = { .alt_type = PSR_VAL_TYPE_UNKNOWN, .get_feat_info = cat_get_feat_info, .write_msr = l2_cat_write_msr, + .check_val = cat_check_cbm, }; /* MBA props */ @@ -507,6 +517,43 @@ static bool mba_get_feat_info(const struct feat_node *feat, static void mba_write_msr(unsigned int cos, uint32_t val, enum psr_val_type type) { + wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val); +} + +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long *thrtl) +{ + if ( *thrtl > feat->mba_info.thrtl_max ) + return false; + + /* + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"): + * 1. Linear mode: In the linear mode the input precision is defined + * as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the + * input precision is 10%. Values not an even multiple of the + * precision (e.g., 12%) will be rounded down (e.g., to 10% delay + * applied). + * 2. Non-linear mode: Input delay values are powers-of-two from zero + * to the MBA_MAX value from CPUID. In this case any values not a + * power of two will be rounded down the next nearest power of two. + */ + if ( feat->mba_info.linear ) + { + unsigned int mod; + + if ( feat->mba_info.thrtl_max >= 100 ) + return false; + + mod = *thrtl % (100 - feat->mba_info.thrtl_max); + *thrtl -= mod; + } + else + { + /* Not power of 2. */ + if ( *thrtl & (*thrtl - 1) ) + *thrtl = *thrtl & (1 << (flsl(*thrtl) - 1)); + } + + return true; } static const struct feat_props mba_props = { @@ -515,6 +562,7 @@ static const struct feat_props mba_props = { .alt_type = PSR_VAL_TYPE_UNKNOWN, .get_feat_info = mba_get_feat_info, .write_msr = mba_write_msr, + .check_val = mba_check_thrtl, }; static void __init parse_psr_bool(char *s, char *value, char *feature, @@ -935,6 +983,7 @@ static int insert_val_into_array(uint32_t val[], const struct feat_node *feat; const struct feat_props *props; unsigned int i; + unsigned long check_val = new_val; int ret; ASSERT(feat_type < FEAT_TYPE_NUM); @@ -959,9 +1008,11 @@ static int insert_val_into_array(uint32_t val[], if ( array_len < props->cos_num ) return -ENOSPC; - if ( !psr_check_cbm(feat->cat_info.cbm_len, new_val) ) + if ( !props->check_val(feat, &check_val) ) return -EINVAL; + new_val = check_val; + /* * Value setting position is same as feature array. * For CDP, user may set both DATA and CODE to same value. For such case, @@ -1187,28 +1238,74 @@ static unsigned int get_socket_cpu(unsigned int socket) return nr_cpu_ids; } +/* + * Because multiple features may co-exist, we need handle all features to write + * values of them into a COS register with new COS ID. E.g: + * 1. L3 CAT and MBA co-exist. + * 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff, + * the MBA Thrtle of Dom1 is 0xa. + * 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is + * used by Dom2 too, we have to pick a new COS ID 3. The original values of + * Dom1 on COS ID 3 may be below: + * --------- + * | COS 3 | + * --------- + * L3 CAT | 0x7ff | + * --------- + * MBA | 0x0 | + * --------- + * 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA + * Thrtl is set. So, the values on COS ID 3 should be below. + * --------- + * | COS 3 | + * --------- + * L3 CAT | 0x1ff | + * --------- + * MBA | 0x14 | + * --------- + * + * So, we should write all features values into their MSRs. That requires the + * feature array, feature properties array and value array are input. + */ struct cos_write_info { unsigned int cos; - struct feat_node *feature; + struct feat_node **features; const uint32_t *val; - const struct feat_props *props; + unsigned int array_len; + const struct feat_props **props; }; static void do_write_psr_msrs(void *data) { const struct cos_write_info *info = data; - struct feat_node *feat = info->feature; - const struct feat_props *props = info->props; - unsigned int i, cos = info->cos, cos_num = props->cos_num; + unsigned int i, j, index = 0, array_len = info->array_len, cos = info->cos; + const uint32_t *val_array = info->val; - for ( i = 0; i < cos_num; i++ ) + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ ) { - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] ) + struct feat_node *feat = info->features[i]; + const struct feat_props *props = info->props[i]; + unsigned int cos_num; + + if ( !feat || !props ) + continue; + + cos_num = props->cos_num; + if ( array_len < cos_num ) + return; + + for ( j = 0; j < cos_num; j++ ) { - feat->cos_reg_val[cos * cos_num + i] = info->val[i]; - props->write_msr(cos, info->val[i], props->type[i]); + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] ) + { + feat->cos_reg_val[cos * cos_num + j] = val_array[index + j]; + props->write_msr(cos, val_array[index + j], props->type[j]); + } } + + array_len -= cos_num; + index += cos_num; } } @@ -1216,30 +1313,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos, const uint32_t val[], unsigned int array_len, enum psr_feat_type feat_type) { - int ret; struct psr_socket_info *info = get_socket_info(socket); struct cos_write_info data = { .cos = cos, - .feature = info->features[feat_type], - .props = feat_props[feat_type], + .features = info->features, + .val = val, + .array_len = array_len, + .props = feat_props, }; if ( cos > info->features[feat_type]->cos_max ) return -EINVAL; - /* Skip to the feature's value head. */ - ret = skip_prior_features(&array_len, feat_type); - if ( ret < 0 ) - return ret; - - val += ret; - - if ( array_len < feat_props[feat_type]->cos_num ) - return -ENOSPC; - - data.val = val; - if ( socket == cpu_to_socket(smp_processor_id()) ) do_write_psr_msrs(&data); else diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index a577a3e..8826cfb 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1144,6 +1144,7 @@ struct xen_domctl_psr_alloc_op { #define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5 #define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6 #define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7 +#define XEN_DOMCTL_PSR_MBA_OP_SET_THRTL 8 #define XEN_DOMCTL_PSR_MBA_OP_GET_THRTL 9 uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */ uint32_t target; /* IN */