From patchwork Sat Sep 23 09:48:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9967449 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A78D4602D8 for ; Sat, 23 Sep 2017 10:10:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9680D28F6A for ; Sat, 23 Sep 2017 10:10:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B61129895; Sat, 23 Sep 2017 10:10:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C3C9428F6A for ; Sat, 23 Sep 2017 10:10:49 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dvhML-0004ga-4N; Sat, 23 Sep 2017 10:08:29 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dvhMJ-0004eh-J2 for xen-devel@lists.xenproject.org; Sat, 23 Sep 2017 10:08:27 +0000 Received: from [85.158.137.68] by server-6.bemta-3.messagelabs.com id CD/E4-02225-A9236C95; Sat, 23 Sep 2017 10:08:26 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrAIsWRWlGSWpSXmKPExsXS1tYhrzvT6Fi kQct0TYvvWyYzOTB6HP5whSWAMYo1My8pvyKBNWPJooWsBS9TK3rfvGdqYGz27mLk5BASqJDY vbiRHcSWEOCVOLJsBiuE7S+xtec1WxcjF1BNA6PE6gPH2EASbALqEo+/9jCB2CICShL3Vk1mA iliFvjNKHFqyyGwhLCAvcSvFX9Yuhg5OFgEVCV23VEECfMKuEt0zP3PBrFATuLksclgyzgFPC Sab5xggTjIXeLvzafMEPWCEidnPgEbwwy0d/08IZAws4C8RPPW2cwTGAVmIamahVA1C0nVAkb mVYwaxalFZalFukameklFmekZJbmJmTm6hgbGermpxcWJ6ak5iUnFesn5uZsYgaFZz8DAuIOx 9YTfIUZJDiYlUd4arWORQnxJ+SmVGYnFGfFFpTmpxYcYZTg4lCR47xgA5QSLUtNTK9Iyc4BRA pOW4OBREuFNB0nzFhck5hZnpkOkTjHqcnTcvPuHSYglLz8vVUqcd68hUJEASFFGaR7cCFjEXm KUlRLmZWRgYBDiKUgtys0sQZV/xSjOwagkzFsGsoonM68EbtMroCOYgI4oX30E5IiSRISUVAN jcfsecbajp/OkgqrPHUnJWNasuERv75tbvN+On9obtHSncnTns21bjkemH5srELXC5DbDbof5 hl7HON/Ov1y4dJZtqFuI1OtJXR5NWWr7+tZyLrnMfb6mVGxea+Ij2eJgNT6++COJczj0H+4WV ttiIyJ6Ydfjj29ORv9jEmPwTGzsD5ziKPdNiaU4I9FQi7moOBEAXL04YdMCAAA= X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-5.tower-31.messagelabs.com!1506161302!112929073!2 X-Originating-IP: [134.134.136.31] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 28410 invoked from network); 23 Sep 2017 10:08:25 -0000 Received: from mga06.intel.com (HELO mga06.intel.com) (134.134.136.31) by server-5.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 23 Sep 2017 10:08:25 -0000 Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP; 23 Sep 2017 03:08:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.42,429,1500966000"; d="scan'208";a="138601224" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.75]) by orsmga002.jf.intel.com with ESMTP; 23 Sep 2017 03:08:21 -0700 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Sat, 23 Sep 2017 17:48:16 +0800 Message-Id: <1506160104-5890-8-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1506160104-5890-1-git-send-email-yi.y.sun@linux.intel.com> References: <1506160104-5890-1-git-send-email-yi.y.sun@linux.intel.com> MIME-Version: 1.0 Cc: Wei Liu , Yi Sun , Andrew Cooper , Jan Beulich , Chao Peng , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Subject: [Xen-devel] [PATCH v4 07/15] x86: implement set value flow for MBA X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch implements set value flow for MBA including its callback function and domctl interface. It also changes the memebers in 'cos_write_info' to transfer the feature array, feature properties array and value array. Then, we can write all features values on the cos id into MSRs. Because multiple features may co-exist, we need handle all features to write values of them into a COS register with new COS ID. E.g: 1. L3 CAT and MBA co-exist. 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff, the MBA Thrtle of Dom1 is 0xa. 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on COS ID 3 are all default values as below: --------- | COS 3 | --------- L3 CAT | 0x7ff | --------- MBA | 0x0 | --------- 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA Thrtl is set. So, the values on COS ID 3 should be below. --------- | COS 3 | --------- L3 CAT | 0x1ff | --------- MBA | 0x14 | --------- So, we should write all features values into their MSRs. That requires the feature array, feature properties array and value array are input. Signed-off-by: Yi Sun --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v4: - remove 'ALLOC_' from macro names. (suggested by Roger Pau Monné) - join two checks into a single if. (suggested by Roger Pau Monné) - remove redundant local variable 'array_len'. (suggested by Roger Pau Monné) v3: - modify commit message to make it clear. (suggested by Roger Pau Monné) - modify functionality of 'check_val' to make it simple to only check value. Change the last parameter type from 'unsigned long *' to 'unsigned long'. (suggested by Roger Pau Monné) - call rdmsrl to get value just written into MSR for MBA. Because HW can automatically change input value to what it wants. (suggested by Roger Pau Monné) - change type of 'write_msr' to 'uint32_t' to return the value actually written into MSR. Then, change 'do_write_psr_msrs' to set the returned value into 'cos_reg_val[]' - move the declaration of 'j' into loop in 'do_write_psr_msrs'. (suggested by Roger Pau Monné) - change 'mba_info' to 'mba'. (suggested by Roger Pau Monné) - change 'cat_info' to 'cat'. (suggested by Roger Pau Monné) - rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP' from name. (suggested by Roger Pau Monné) - change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'. (suggested by Roger Pau Monné) v2: - remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has been checked in 'mba_init_feature'. (suggested by Chao Peng) - for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If it is 0, we do not need to change it. (suggested by Chao Peng) - move comments to explain changes of 'cos_write_info' from psr.c to commit message. (suggested by Chao Peng) --- xen/arch/x86/domctl.c | 6 ++ xen/arch/x86/psr.c | 144 ++++++++++++++++++++++++++------------------ xen/include/public/domctl.h | 1 + 3 files changed, 94 insertions(+), 57 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 22650d7..66c9cab 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1468,6 +1468,12 @@ long arch_do_domctl( PSR_TYPE_L2_CBM); break; + case XEN_DOMCTL_PSR_SET_MBA_THRTL: + ret = psr_set_val(d, domctl->u.psr_alloc.target, + domctl->u.psr_alloc.data, + PSR_TYPE_MBA_THRTL); + break; + case XEN_DOMCTL_PSR_GET_L3_CBM: ret = psr_get_val(d, domctl->u.psr_alloc.target, &val32, PSR_TYPE_L3_CBM); diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 1d0a215..c6d4e31 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -137,7 +137,10 @@ static const struct feat_props { uint32_t data[], unsigned int array_len); /* write_msr is used to write out feature MSR register. */ - void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type); + uint32_t (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type); + + /* check_val is used to check if input val fulfills SDM requirement. */ + bool (*check_val)(const struct feat_node *feat, unsigned long val); } *feat_props[FEAT_TYPE_NUM]; /* @@ -274,29 +277,6 @@ static enum psr_feat_type psr_type_to_feat_type(enum psr_type type) return feat_type; } -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm) -{ - unsigned int first_bit, zero_bit; - - /* Set bits should only in the range of [0, cbm_len]. */ - if ( cbm & (~0ul << cbm_len) ) - return false; - - /* At least one bit need to be set. */ - if ( cbm == 0 ) - return false; - - first_bit = find_first_bit(&cbm, cbm_len); - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit); - - /* Set bits should be contiguous. */ - if ( zero_bit < cbm_len && - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len ) - return false; - - return true; -} - /* Implementation of allocation features' functions. */ static bool cat_init_feature(const struct cpuid_leaf *regs, struct feat_node *feat, @@ -426,11 +406,36 @@ static bool cat_get_feat_info(const struct feat_node *feat, return true; } +static bool cat_check_cbm(const struct feat_node *feat, unsigned long cbm) +{ + unsigned int first_bit, zero_bit; + unsigned int cbm_len = feat->cat.cbm_len; + + /* + * Set bits should only in the range of [0, cbm_len]. + * And, at least one bit need to be set. + */ + if ( cbm & (~0ul << cbm_len) || cbm == 0 ) + return false; + + first_bit = find_first_bit(&cbm, cbm_len); + zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit); + + /* Set bits should be contiguous. */ + if ( zero_bit < cbm_len && + find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len ) + return false; + + return true; +} + /* L3 CAT props */ -static void l3_cat_write_msr(unsigned int cos, uint32_t val, - enum psr_type type) +static uint32_t l3_cat_write_msr(unsigned int cos, uint32_t val, + enum psr_type type) { wrmsrl(MSR_IA32_PSR_L3_MASK(cos), val); + + return val; } static const struct feat_props l3_cat_props = { @@ -439,6 +444,7 @@ static const struct feat_props l3_cat_props = { .alt_type = PSR_TYPE_UNKNOWN, .get_feat_info = cat_get_feat_info, .write_msr = l3_cat_write_msr, + .check_val = cat_check_cbm, }; /* L3 CDP props */ @@ -453,13 +459,15 @@ static bool l3_cdp_get_feat_info(const struct feat_node *feat, return true; } -static void l3_cdp_write_msr(unsigned int cos, uint32_t val, - enum psr_type type) +static uint32_t l3_cdp_write_msr(unsigned int cos, uint32_t val, + enum psr_type type) { wrmsrl(((type == PSR_TYPE_L3_DATA) ? MSR_IA32_PSR_L3_MASK_DATA(cos) : MSR_IA32_PSR_L3_MASK_CODE(cos)), val); + + return val; } static const struct feat_props l3_cdp_props = { @@ -469,13 +477,16 @@ static const struct feat_props l3_cdp_props = { .alt_type = PSR_TYPE_L3_CBM, .get_feat_info = l3_cdp_get_feat_info, .write_msr = l3_cdp_write_msr, + .check_val = cat_check_cbm, }; /* L2 CAT props */ -static void l2_cat_write_msr(unsigned int cos, uint32_t val, - enum psr_type type) +static uint32_t l2_cat_write_msr(unsigned int cos, uint32_t val, + enum psr_type type) { wrmsrl(MSR_IA32_PSR_L2_MASK(cos), val); + + return val; } static const struct feat_props l2_cat_props = { @@ -484,6 +495,7 @@ static const struct feat_props l2_cat_props = { .alt_type = PSR_TYPE_UNKNOWN, .get_feat_info = cat_get_feat_info, .write_msr = l2_cat_write_msr, + .check_val = cat_check_cbm, }; /* MBA props */ @@ -502,9 +514,23 @@ static bool mba_get_feat_info(const struct feat_node *feat, return true; } -static void mba_write_msr(unsigned int cos, uint32_t val, - enum psr_type type) +static uint32_t mba_write_msr(unsigned int cos, uint32_t val, + enum psr_type type) +{ + wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val); + + /* Read actual value set by hardware. */ + rdmsrl(MSR_IA32_PSR_MBA_MASK(cos), val); + + return val; +} + +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long thrtl) { + if ( thrtl > feat->mba.thrtl_max ) + return false; + + return true; } static const struct feat_props mba_props = { @@ -513,6 +539,7 @@ static const struct feat_props mba_props = { .alt_type = PSR_TYPE_UNKNOWN, .get_feat_info = mba_get_feat_info, .write_msr = mba_write_msr, + .check_val = mba_check_thrtl, }; static bool __init parse_psr_bool(const char *s, const char *delim, @@ -978,7 +1005,7 @@ static int insert_val_into_array(uint32_t val[], if ( array_len < props->cos_num ) return -ENOSPC; - if ( !psr_check_cbm(feat->cat.cbm_len, new_val) ) + if ( !props->check_val(feat, new_val) ) return -EINVAL; /* @@ -1210,25 +1237,39 @@ static unsigned int get_socket_cpu(unsigned int socket) struct cos_write_info { unsigned int cos; - struct feat_node *feature; + struct feat_node **features; const uint32_t *val; - const struct feat_props *props; + unsigned int array_len; + const struct feat_props **props; }; static void do_write_psr_msrs(void *data) { const struct cos_write_info *info = data; - struct feat_node *feat = info->feature; - const struct feat_props *props = info->props; - unsigned int i, cos = info->cos, cos_num = props->cos_num; + unsigned int i, index = 0, cos = info->cos; + const uint32_t *val_array = info->val; - for ( i = 0; i < cos_num; i++ ) + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ ) { - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] ) + struct feat_node *feat = info->features[i]; + const struct feat_props *props = info->props[i]; + unsigned int cos_num, j; + + if ( !feat || !props ) + continue; + + cos_num = props->cos_num; + if ( info->array_len < index + cos_num ) + return; + + for ( j = 0; j < cos_num; j++ ) { - feat->cos_reg_val[cos * cos_num + i] = info->val[i]; - props->write_msr(cos, info->val[i], props->type[i]); + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] ) + feat->cos_reg_val[cos * cos_num + j] = + props->write_msr(cos, val_array[index + j], props->type[j]); } + + index += cos_num; } } @@ -1236,30 +1277,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos, const uint32_t val[], unsigned int array_len, enum psr_feat_type feat_type) { - int ret; struct psr_socket_info *info = get_socket_info(socket); struct cos_write_info data = { .cos = cos, - .feature = info->features[feat_type], - .props = feat_props[feat_type], + .features = info->features, + .val = val, + .array_len = array_len, + .props = feat_props, }; if ( cos > info->features[feat_type]->cos_max ) return -EINVAL; - /* Skip to the feature's value head. */ - ret = skip_prior_features(&array_len, feat_type); - if ( ret < 0 ) - return ret; - - val += ret; - - if ( array_len < feat_props[feat_type]->cos_num ) - return -ENOSPC; - - data.val = val; - if ( socket == cpu_to_socket(smp_processor_id()) ) do_write_psr_msrs(&data); else diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index ec6d2de..714b3e8 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1156,6 +1156,7 @@ struct xen_domctl_psr_alloc { #define XEN_DOMCTL_PSR_GET_L3_DATA 5 #define XEN_DOMCTL_PSR_SET_L2_CBM 6 #define XEN_DOMCTL_PSR_GET_L2_CBM 7 +#define XEN_DOMCTL_PSR_SET_MBA_THRTL 8 #define XEN_DOMCTL_PSR_GET_MBA_THRTL 9 uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */ uint32_t target; /* IN */