From patchwork Wed Dec 14 04:07:48 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9474027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A40B160823 for ; Wed, 14 Dec 2016 11:12:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A98A2871A for ; Wed, 14 Dec 2016 11:12:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8F08728720; Wed, 14 Dec 2016 11:12:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00, DATE_IN_PAST_06_12, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9C4F52871A for ; Wed, 14 Dec 2016 11:12:22 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cH7Rg-00051Z-7G; Wed, 14 Dec 2016 11:10:00 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cH7Re-00050T-OD for xen-devel@lists.xenproject.org; Wed, 14 Dec 2016 11:09:58 +0000 Received: from [85.158.143.35] by server-3.bemta-6.messagelabs.com id 86/AE-09053-68821585; Wed, 14 Dec 2016 11:09:58 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHLMWRWlGSWpSXmKPExsXS1tYhoduqERh h8L5Z3OL7lslMDowehz9cYQlgjGLNzEvKr0hgzdjc+ZOlYH5kxe4Zx5kbGH/YdjFycggJVEhs nbOKCcSWEOCVOLJsBiuE7Scxf9tmti5GLqCaBkaJ06v+gyXYBNQlHn/tAWsQEVCSuLdqMhNIE bNAN5PEyodbWUASwgI+Ek//fmUDsVkEVCXuHVvCDGLzCrhLLLr3lgVig5zEyWOTgYZycHAKeE j825IIcZC7xMXed+wTGHkXMDKsYtQoTi0qSy3SNTLWSyrKTM8oyU3MzNE1NDDTy00tLk5MT81 JTCrWS87P3cQIDAcGINjB+Gd+4CFGSQ4mJVFerqsBEUJ8SfkplRmJxRnxRaU5qcWHGGU4OJQk eGepB0YICRalpqdWpGXmAAMTJi3BwaMkwhukApTmLS5IzC3OTIdInWJUlBLnXQnSJwCSyCjNg 2uDRcMlRlkpYV5GoEOEeApSi3IzS1DlXzGKczAqCfNuAJnCk5lXAjf9FdBiJqDFokv8QRaXJC KkpBoYG9Kmn7uu557ixbdU9JTWsZI3fEt9N1xy28pS/f3Pji7hDzkHb9z/f3Zeu5qbs7HDub9 r82LUOP8efySxY63ScpnXlueyc0uvr0z13nVsPb/rtq1Saau8Hqw493iy/LFZXSvar+q/+y+Q b/d1SUPph7aGpbvDM7os1mvvOPG6wvO4adSNRZ5Oy5RYijMSDbWYi4oTASU5Hp2BAgAA X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-12.tower-21.messagelabs.com!1481713769!48240089!9 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18773 invoked from network); 14 Dec 2016 11:09:56 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-12.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 14 Dec 2016 11:09:56 -0000 Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP; 14 Dec 2016 03:09:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,346,1477983600"; d="scan'208";a="39863222" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.63]) by orsmga004.jf.intel.com with ESMTP; 14 Dec 2016 03:09:54 -0800 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Wed, 14 Dec 2016 12:07:48 +0800 Message-Id: <1481688484-5093-9-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1481688484-5093-1-git-send-email-yi.y.sun@linux.intel.com> References: <1481688484-5093-1-git-send-email-yi.y.sun@linux.intel.com> Cc: wei.liu2@citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, he.chen@linux.intel.com, ian.jackson@eu.citrix.com, Yi Sun , mengxu@cis.upenn.edu, jbeulich@suse.com, chao.p.peng@linux.intel.com Subject: [Xen-devel] [PATCH v4 08/24] x86: refactor psr: set value: implement framework. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP As set value flow is the most complicated one in psr, it will be divided to some patches to make things clearer. This patch implements the set value framework to show a whole picture firstly. It also changes domctl interface to make it more general. To make the set value flow be general and can support multiple features at same time, it includes below steps: 1. Get COS ID of current domain using. 2. Assemble a value array to store all features current value in it and replace the current value of the feature which is being set to the new input value. 3. Find if there is already a COS ID on which all features' values are same as the array. Then, we can reuse this COS ID. 4. If fail to find, we need allocate a new COS ID. Only COS ID which ref is 0 or 1 can be allocated. 5. Write all features MSRs according to the COS ID. 6. Update ref according to COS ID. 7. Save the COS ID into current domain's psr_cos_ids[socket] so that we can know which COS the domain is using on the socket. So, some functions are abstracted and the callback functions will be implemented in next patches. Here is an example to understand the process. The CPU supports two featuers, e.g. L3 CAT and L2 CAT. user wants to set L3 CAT of Dom1 to 0x1ff. 1. Get the old_cos of Dom1 which is 0. L3 CAT is the first element of feature list. The COS registers values are below at this time. ------------------------------- | COS 0 | COS 1 | COS 2 | ... | ------------------------------- L3 CAT | 0x7ff | ... | ... | ... | ------------------------------- L2 CAT | 0xff | ... | ... | ... | ------------------------------- 2. Assemble The value array to be: val[0]: 0x1ff val[1]: 0xff 3. It cannot find a matching COS. 4. Allocate COS 1 to store the value set. 5. Write the COS 1 registers. The COS registers values are changed to below now. ------------------------------- | COS 0 | COS 1 | COS 2 | ... | ------------------------------- L3 CAT | 0x7ff | 0x1ff | ... | ... | ------------------------------- L2 CAT | 0xff | 0xff | ... | ... | ------------------------------- 6. The ref[1] is increased to 1 because Dom1 is using it now. 7. Save 1 to Dom1's psr_cos_ids[socket]. Then, user wants to set L3 CAT of Dom2 to 0x1ff too. The old_cos of Dom2 is 0 too. Repeat above flow. The val array assembled is: val[0]: 0x1ff val[1]: 0xff So, it can find a matching COS, COS 1. Then, it can reuse COS 1 for Dom2. The ref[1] is increased to 2 now because both Dom1 and Dom2 are using this COS ID. Set 1 to Dom2's psr_cos_ids[socket]. Signed-off-by: Yi Sun --- xen/arch/x86/domctl.c | 18 ++--- xen/arch/x86/psr.c | 183 +++++++++++++++++++++++++++++++++++++++++++++- xen/include/asm-x86/psr.h | 4 +- 3 files changed, 192 insertions(+), 13 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 3d7fc34..8e5502a 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1386,21 +1386,21 @@ long arch_do_domctl( switch ( domctl->u.psr_cat_op.cmd ) { case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM: - ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target, - domctl->u.psr_cat_op.data, - PSR_CBM_TYPE_L3); + ret = psr_set_val(d, domctl->u.psr_cat_op.target, + domctl->u.psr_cat_op.data, + PSR_CBM_TYPE_L3); break; case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE: - ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target, - domctl->u.psr_cat_op.data, - PSR_CBM_TYPE_L3_CODE); + ret = psr_set_val(d, domctl->u.psr_cat_op.target, + domctl->u.psr_cat_op.data, + PSR_CBM_TYPE_L3_CODE); break; case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA: - ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target, - domctl->u.psr_cat_op.data, - PSR_CBM_TYPE_L3_DATA); + ret = psr_set_val(d, domctl->u.psr_cat_op.target, + domctl->u.psr_cat_op.data, + PSR_CBM_TYPE_L3_DATA); break; case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM: diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 1ea694e..838f2d1 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -513,18 +513,197 @@ int psr_get_val(struct domain *d, unsigned int socket, return -ENOENT; } -int psr_set_l3_cbm(struct domain *d, unsigned int socket, - uint64_t cbm, enum cbm_type type) +/* Set value functions */ +static unsigned int get_cos_num(const struct psr_socket_info *info) { return 0; } +static int get_old_set_new(uint64_t *val, + uint32_t array_len, + const struct psr_socket_info *info, + unsigned int old_cos, + enum cbm_type type, + uint64_t m) +{ + return 0; +} + +static int find_cos(const uint64_t *val, uint32_t array_len, + enum cbm_type type, + const struct psr_socket_info *info) +{ + return 0; +} + +static int alloc_new_cos(const struct psr_socket_info *info, + const uint64_t *val, uint32_t array_len, + unsigned int old_cos, + enum cbm_type type) +{ + return 0; +} + +static int write_psr_msr(unsigned int socket, unsigned int cos, + const uint64_t *val) +{ + return 0; +} + +int psr_set_val(struct domain *d, unsigned int socket, + uint64_t val, enum cbm_type type) +{ + unsigned int old_cos; + int cos, ret; + unsigned int *ref; + uint64_t *val_array; + struct psr_socket_info *info = get_socket_info(socket); + uint32_t array_len; + + if ( IS_ERR(info) ) + return PTR_ERR(info); + + /* + * Step 0: + * old_cos means the COS ID current domain is using. By default, it is 0. + * + * For every COS ID, there is a reference count to record how many domains + * are using the COS register corresponding to this COS ID. + * - If ref[old_cos] is 0, that means this COS is not used by any domain. + * - If ref[old_cos] is 1, that means this COS is only used by current + * domain. + * - If ref[old_cos] is more than 1, that mean multiple domains are using + * this COS. + */ + old_cos = d->arch.psr_cos_ids[socket]; + if ( old_cos > MAX_COS_REG_CNT ) + return -EOVERFLOW; + + ref = info->cos_ref; + + /* + * Step 1: + * Assemle a value array to store all featues cos_reg_val[old_cos]. + * And, set the input val into array according to the feature's + * position in array. + */ + array_len = get_cos_num((const struct psr_socket_info *)info); + val_array = xzalloc_array(uint64_t, array_len); + if ( !val_array ) + return -ENOMEM; + + if ( (ret = get_old_set_new(val_array, array_len, + (const struct psr_socket_info *)info, + old_cos, type, val)) != 0 ) + { + xfree(val_array); + return ret; + } + + /* + * Lock here to make sure the ref is not changed during find and + * write process. + */ + spin_lock(&info->ref_lock); + + /* + * Step 2: + * Try to find if there is already a COS ID on which all features' values + * are same as the array. Then, we can reuse this COS ID. + */ + cos = find_cos((const uint64_t *)val_array, array_len, type, + (const struct psr_socket_info *)info); + if ( cos >= 0 ) + { + if ( cos == old_cos ) + { + spin_unlock(&info->ref_lock); + xfree(val_array); + return 0; + } + } + else + { + /* + * Step 3: + * If fail to find, we need allocate a new COS ID. + * If multiple domains are using same COS ID, its ref is more + * than 1. That means we cannot free this COS to make current domain + * use it. Because other domains are using the value saved in the COS. + * Unless the ref is changed to 1 (mean only current domain is using + * it), we cannot allocate the COS ID to current domain. + * So, only the COS ID which ref is 1 or 0 can be allocated. + */ + cos = alloc_new_cos((const struct psr_socket_info *)info, + (const uint64_t *)val_array, array_len, + old_cos, type); + if ( cos < 0 ) + { + spin_unlock(&info->ref_lock); + xfree(val_array); + return cos; + } + + /* + * Step 4: + * Write all features MSRs according to the COS ID. + */ + ret = write_psr_msr(socket, cos, (const uint64_t *)val_array); + if ( ret ) + { + spin_unlock(&info->ref_lock); + xfree(val_array); + return ret; + } + } + + /* + * Step 5: + * Update ref according to COS ID. + */ + ref[cos]++; + ref[old_cos]--; + spin_unlock(&info->ref_lock); + + /* + * Step 6: + * Save the COS ID into current domain's psr_cos_ids[] so that we can know + * which COS the domain is using on the socket. One domain can only use + * one COS ID at same time. + */ + d->arch.psr_cos_ids[socket] = cos; + xfree(val_array); + + return 0; +} + /* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */ static void psr_free_cos(struct domain *d) { + unsigned int socket; + unsigned int cos; + struct psr_socket_info *info; + if( !d->arch.psr_cos_ids ) return; + /* Domain is free so its cos_ref should be decreased. */ + for( socket = 0; socket < nr_sockets; socket++ ) + { + /* cos 0 is default one which does not need be handled. */ + if ( (cos = d->arch.psr_cos_ids[socket]) == 0 ) + continue; + + /* + * If domain uses other cos ids, all corresponding refs must have been + * increased 1 for this domain. So, we need decrease them. + */ + info = socket_info + socket; + spin_lock(&info->ref_lock); + info->cos_ref[cos]--; + spin_unlock(&info->ref_lock); + } + xfree(d->arch.psr_cos_ids); d->arch.psr_cos_ids = NULL; } diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h index 8e3c90e..d245de3 100644 --- a/xen/include/asm-x86/psr.h +++ b/xen/include/asm-x86/psr.h @@ -72,8 +72,8 @@ int psr_get_info(unsigned int socket, enum cbm_type type, uint32_t dat[], uint32_t array_len); int psr_get_val(struct domain *d, unsigned int socket, uint64_t *val, enum cbm_type type); -int psr_set_l3_cbm(struct domain *d, unsigned int socket, - uint64_t cbm, enum cbm_type type); +int psr_set_val(struct domain *d, unsigned int socket, + uint64_t val, enum cbm_type type); int psr_domain_init(struct domain *d); void psr_domain_free(struct domain *d);