From patchwork Wed Dec 14 04:07:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9474013 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 34DEF60823 for ; Wed, 14 Dec 2016 11:12:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A0292871A for ; Wed, 14 Dec 2016 11:12:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D2022871F; Wed, 14 Dec 2016 11:12:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00, DATE_IN_PAST_06_12, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9C1212871A for ; Wed, 14 Dec 2016 11:12:08 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cH7RS-0004pR-3H; Wed, 14 Dec 2016 11:09:46 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cH7RR-0004oo-3D for xen-devel@lists.xenproject.org; Wed, 14 Dec 2016 11:09:45 +0000 Received: from [85.158.143.35] by server-11.bemta-6.messagelabs.com id 3D/B8-25337-87821585; Wed, 14 Dec 2016 11:09:44 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHLMWRWlGSWpSXmKPExsXS1tYhoVuuERh hsPm6scX3LZOZHBg9Dn+4whLAGMWamZeUX5HAmrFzwg7Ggg3GFT837mRrYPyp1sXIySEkUCFx s2UxI4gtIcArcWTZDFYI20/i/fKvQDYXUE0Do8Suhe/AitgE1CUef+1hArFFBJQk7q2azARSx CzQzSSx8uFWFpCEMFD3nynT2EFsFgFViXMLZrCB2LwC7hLnuv6wQ2yQkzh5bDLQBg4OTgEPiX 9bEiEOcpe42PuOfQIj7wJGhlWMGsWpRWWpRbpGxnpJRZnpGSW5iZk5uoYGZnq5qcXFiempOYl JxXrJ+bmbGIHhwAAEOxj/zA88xCjJwaQkyst1NSBCiC8pP6UyI7E4I76oNCe1+BCjDAeHkgTv LPXACCHBotT01Iq0zBxgYMKkJTh4lER4g1SA0rzFBYm5xZnpEKlTjIpS4rwrQfoEQBIZpXlwb bBouMQoKyXMywh0iBBPQWpRbmYJqvwrRnEORiVh3g0gU3gy80rgpr8CWswEtFh0iT/I4pJEhJ RUA2PbrN5bzWp/8hUinaS4yib/lJ8c/3Q5m6fCfcXWmXXW2uLLZufFb8qoO6kgGhbU8tzmtEt bfJrwh37GrdcXPTunZF7dm8/JYf+a1U5fZE/2j6CzhfdSz07r1slckhda9fHI8mn/OQ+ZzE9Z PM/+5GXDPSlTz+0X8lZb+P5K6udJbxnXPjBSU1JiKc5INNRiLipOBAB19W+EgQIAAA== X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-12.tower-21.messagelabs.com!1481713769!48240089!5 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 9.1.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15365 invoked from network); 14 Dec 2016 11:09:43 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-12.tower-21.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 14 Dec 2016 11:09:43 -0000 Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP; 14 Dec 2016 03:09:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,346,1477983600"; d="scan'208";a="39863174" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.63]) by orsmga004.jf.intel.com with ESMTP; 14 Dec 2016 03:09:40 -0800 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Wed, 14 Dec 2016 12:07:44 +0800 Message-Id: <1481688484-5093-5-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1481688484-5093-1-git-send-email-yi.y.sun@linux.intel.com> References: <1481688484-5093-1-git-send-email-yi.y.sun@linux.intel.com> Cc: wei.liu2@citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, he.chen@linux.intel.com, ian.jackson@eu.citrix.com, Yi Sun , mengxu@cis.upenn.edu, jbeulich@suse.com, chao.p.peng@linux.intel.com Subject: [Xen-devel] [PATCH v4 04/24] x86: refactor psr: implement CPU init and free flow. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch implements the CPU init and free flow including L3 CAT initialization and feature list free. Per this patch, you can see how callback functions work and how the feature list is handled. Signed-off-by: Yi Sun --- xen/arch/x86/psr.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 176 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 49a4598..fa9bc32 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -35,6 +35,9 @@ #define PSR_CAT (1<<1) #define PSR_CDP (1<<2) +#define CAT_CBM_LEN_MASK 0x1f +#define CAT_COS_MAX_MASK 0xffff + /* * Per SDM 17.17.3.3 'Cache Allocation Technology: Cache Mask Configuration', * the MSRs range from 0C90H through 0D0FH (inclusive), enables support for @@ -141,11 +144,79 @@ struct psr_assoc { struct psr_cmt *__read_mostly psr_cmt; +static struct psr_socket_info *__read_mostly socket_info; + static unsigned int opt_psr; static unsigned int __initdata opt_rmid_max = 255; +static unsigned int __read_mostly opt_cos_max = MAX_COS_REG_CNT; static uint64_t rmid_mask; static DEFINE_PER_CPU(struct psr_assoc, psr_assoc); +/* Declare feature list entry. */ +static struct feat_node *feat_l3_cat; + +/* Common functions. */ +static void free_feature(struct psr_socket_info *info) +{ + struct feat_node *feat_tmp; + + if ( !info ) + return; + + list_for_each_entry(feat_tmp, &info->feat_list, list) + { + clear_bit(feat_tmp->feature, &info->feat_mask); + list_del(&feat_tmp->list); + xfree(feat_tmp); + } + + /* Free feature which are not added into feat_list. */ + if ( feat_l3_cat ) + { + xfree(feat_l3_cat); + feat_l3_cat = NULL; + } +} + +/* L3 CAT callback functions implementation. */ +static void l3_cat_init_feature(unsigned int eax, unsigned int ebx, + unsigned int ecx, unsigned int edx, + struct feat_node *feat, + struct psr_socket_info *info) +{ + struct psr_cat_hw_info l3_cat; + unsigned int socket; + + /* No valid value so do not enable feature. */ + if ( !eax || !edx ) + return; + + l3_cat.cbm_len = (eax & CAT_CBM_LEN_MASK) + 1; + l3_cat.cos_max = min(opt_cos_max, edx & CAT_COS_MAX_MASK); + + /* cos=0 is reserved as default cbm(all ones). */ + feat->cos_reg_val[0] = (1ull << l3_cat.cbm_len) - 1; + + feat->feature = PSR_SOCKET_L3_CAT; + __set_bit(PSR_SOCKET_L3_CAT, &info->feat_mask); + + feat->info.l3_cat_info = l3_cat; + + info->nr_feat++; + + /* Add this feature into list. */ + list_add_tail(&feat->list, &info->feat_list); + + socket = cpu_to_socket(smp_processor_id()); + printk(XENLOG_INFO "L3 CAT: enabled on socket %u, cos_max:%u, cbm_len:%u\n", + socket, feat->info.l3_cat_info.cos_max, + feat->info.l3_cat_info.cbm_len); +} + +struct feat_ops l3_cat_ops = { + .init_feature = l3_cat_init_feature, +}; + static void __init parse_psr_bool(char *s, char *value, char *feature, unsigned int mask) { @@ -185,6 +256,9 @@ static void __init parse_psr_param(char *s) if ( val_str && !strcmp(s, "rmid_max") ) opt_rmid_max = simple_strtoul(val_str, NULL, 0); + if ( val_str && !strcmp(s, "cos_max") ) + opt_cos_max = simple_strtoul(val_str, NULL, 0); + s = ss + 1; } while ( ss ); } @@ -340,18 +414,113 @@ void psr_domain_free(struct domain *d) psr_free_rmid(d); } -static int psr_cpu_prepare(unsigned int cpu) +static int cpu_prepare_work(unsigned int cpu) { + if ( !socket_info ) + return 0; + + /* Malloc memory for the global feature head here. */ + if ( feat_l3_cat == NULL && + (feat_l3_cat = xzalloc(struct feat_node)) == NULL ) + return -ENOMEM; + return 0; } +static void cpu_init_work(void) +{ + unsigned int eax, ebx, ecx, edx; + struct psr_socket_info *info; + unsigned int socket; + unsigned int cpu = smp_processor_id(); + const struct cpuinfo_x86 *c = cpu_data + cpu; + struct feat_node *feat_tmp; + + if ( !cpu_has(c, X86_FEATURE_PQE) || c->cpuid_level < PSR_CPUID_LEVEL_CAT ) + return; + + socket = cpu_to_socket(cpu); + info = socket_info + socket; + if ( info->feat_mask ) + return; + + spin_lock_init(&info->ref_lock); + + cpuid_count(PSR_CPUID_LEVEL_CAT, 0, &eax, &ebx, &ecx, &edx); + if ( ebx & PSR_RESOURCE_TYPE_L3 ) + { + cpuid_count(PSR_CPUID_LEVEL_CAT, 1, &eax, &ebx, &ecx, &edx); + + feat_tmp = feat_l3_cat; + feat_l3_cat = NULL; + feat_tmp->ops = l3_cat_ops; + + feat_tmp->ops.init_feature(eax, ebx, ecx, edx, feat_tmp, info); + } +} + +static void cpu_fini_work(unsigned int cpu) +{ + unsigned int socket = cpu_to_socket(cpu); + + if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) ) + { + struct psr_socket_info *info = socket_info + socket; + + free_feature(info); + } +} + +static void __init init_psr(void) +{ + unsigned int i; + + if ( opt_cos_max < 1 ) + { + printk(XENLOG_INFO "CAT: disabled, cos_max is too small\n"); + return; + } + + socket_info = xzalloc_array(struct psr_socket_info, nr_sockets); + + if ( !socket_info ) + { + printk(XENLOG_INFO "Fail to alloc socket_info!\n"); + return; + } + + for ( i = 0; i < nr_sockets; i++ ) + INIT_LIST_HEAD(&socket_info[i].feat_list); +} + +static void __init psr_free(void) +{ + unsigned int i; + + for ( i = 0; i < nr_sockets; i++ ) + free_feature(&socket_info[i]); + + xfree(socket_info); + socket_info = NULL; +} + +static int psr_cpu_prepare(unsigned int cpu) +{ + return cpu_prepare_work(cpu); +} + static void psr_cpu_init(void) { + if ( socket_info ) + cpu_init_work(); + psr_assoc_init(); } static void psr_cpu_fini(unsigned int cpu) { + if ( socket_info ) + cpu_fini_work(cpu); return; } @@ -393,10 +562,14 @@ static int __init psr_presmp_init(void) if ( (opt_psr & PSR_CMT) && opt_rmid_max ) init_psr_cmt(opt_rmid_max); - psr_cpu_prepare(0); + if ( opt_psr & PSR_CAT ) + init_psr(); + + if ( psr_cpu_prepare(0) ) + psr_free(); psr_cpu_init(); - if ( psr_cmt_enabled() ) + if ( psr_cmt_enabled() || socket_info ) register_cpu_notifier(&cpu_nfb); return 0;