From patchwork Thu Apr 20 05:38:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9689483 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A4279601D4 for ; Thu, 20 Apr 2017 05:49:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93D2C2844C for ; Thu, 20 Apr 2017 05:49:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86E492845C; Thu, 20 Apr 2017 05:49:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BDA9528437 for ; Thu, 20 Apr 2017 05:49:00 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d14vZ-0005k7-T1; Thu, 20 Apr 2017 05:46:49 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1d14vY-0005jy-J8 for xen-devel@lists.xenproject.org; Thu, 20 Apr 2017 05:46:48 +0000 Received: from [85.158.137.68] by server-4.bemta-3.messagelabs.com id BD/B6-02192-74B48F85; Thu, 20 Apr 2017 05:46:47 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHLMWRWlGSWpSXmKPExsXS1tYhoevu/SP C4OASRovvWyYzOTB6HP5whSWAMYo1My8pvyKBNePL01tMBUddK/r3PmBuYJxk3MXIxSEkMI1R Ys7uB2xdjJwcEgK8EkeWzWCFsP0kjq4/wA5R1MAosf74fLAEm4C6xOOvPUwgtoiAksS9VZOBb A4OZgFLiQdTs0DCwgJ6Esd61oHNZBFQlfh4fRJYK6+Au8SHWZuYIebLSZw8NhkszingJbG4dR EjiC0k4CmxbsZspgmMvAsYGVYxahSnFpWlFukamuglFWWmZ5TkJmbm6BoaGOvlphYXJ6an5iQ mFesl5+duYgSGAwMQ7GBcsd3zEKMkB5OSKO9H1x8RQnxJ+SmVGYnFGfFFpTmpxYcYZTg4lCR4 4z2BcoJFqempFWmZOcDAhElLcPAoifBu9ABK8xYXJOYWZ6ZDpE4xKkqJ804H6RMASWSU5sG1w aLhEqOslDAvI9AhQjwFqUW5mSWo8q8YxTkYlYR5+0Cm8GTmlcBNfwW0mAlocUTAF5DFJYkIKa kGxgDTM0yb9L4FhqyUUajqrb9ff8T87qLpMoeDG1q5Qy3nX604+C79mKi7HvuMoAs1fh5h1+Y 2uT67EOfB+UrvH08jy/VFHFPSVpw6EJr+osYoRSRJjvHBRmHRXXG35zYX3LyS5nGJ8V/A5CUX bK9tnpPtaP5NwY8jVi73UAeDwQf+XbYdG/RWK7EUZyQaajEXFScCAIxXqW+BAgAA X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-12.tower-31.messagelabs.com!1492667204!79693991!1 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 9.4.12; banners=-,-,- X-VirusChecked: Checked Received: (qmail 63070 invoked from network); 20 Apr 2017 05:46:46 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-12.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Apr 2017 05:46:46 -0000 Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Apr 2017 22:46:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.37,224,1488873600"; d="scan'208";a="90026156" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.39]) by orsmga005.jf.intel.com with ESMTP; 19 Apr 2017 22:46:41 -0700 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Thu, 20 Apr 2017 13:38:24 +0800 Message-Id: <1492666704-6626-1-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1491054836-30488-10-git-send-email-yi.y.sun@linux.intel.com> References: <1491054836-30488-10-git-send-email-yi.y.sun@linux.intel.com> Cc: Yi Sun , jbeulich@suse.com Subject: [Xen-devel] [PATCH] dom_ids array implementation. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi, Jan, Please help to review this patch. Thank you! Signed-off-by: Yi Sun --- xen/arch/x86/psr.c | 135 ++++++++++++++++++++++------------------------------- 1 file changed, 55 insertions(+), 80 deletions(-) diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index a85ea99..7bc212f 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -125,6 +125,8 @@ struct feat_node { uint32_t cos_reg_val[MAX_COS_REG_CNT]; }; +#define PSR_DOM_IDS_NUM ((DOMID_IDLE + 1) / sizeof(uint32_t)) + /* * PSR features are managed per socket. Below structure defines the members * used to manage these features. @@ -134,9 +136,11 @@ struct feat_node { * COS ID. Every entry of cos_ref corresponds to one COS ID. */ struct psr_socket_info { - struct feat_node *features[PSR_SOCKET_MAX_FEAT]; spinlock_t ref_lock; + spinlock_t dom_ids_lock; + struct feat_node *features[PSR_SOCKET_MAX_FEAT]; unsigned int cos_ref[MAX_COS_REG_CNT]; + uint32_t dom_ids[PSR_DOM_IDS_NUM]; }; struct psr_assoc { @@ -194,26 +198,11 @@ static void free_socket_resources(unsigned int socket) { unsigned int i; struct psr_socket_info *info = socket_info + socket; - struct domain *d; + unsigned long flag; if ( !info ) return; - /* Restore domain cos id to 0 when socket is offline. */ - for_each_domain ( d ) - { - unsigned int cos = d->arch.psr_cos_ids[socket]; - if ( cos == 0 ) - continue; - - spin_lock(&info->ref_lock); - ASSERT(!cos || info->cos_ref[cos]); - info->cos_ref[cos]--; - spin_unlock(&info->ref_lock); - - d->arch.psr_cos_ids[socket] = 0; - } - /* * Free resources of features. The global feature object, e.g. feat_l3_cat, * may not be freed here if it is not added into array. It is simply being @@ -221,12 +210,17 @@ static void free_socket_resources(unsigned int socket) */ for ( i = 0; i < PSR_SOCKET_MAX_FEAT; i++ ) { - if ( !info->features[i] ) - continue; - xfree(info->features[i]); info->features[i] = NULL; } + + spin_lock(&info->ref_lock); + memset(info->cos_ref, 0, MAX_COS_REG_CNT * sizeof(unsigned int)); + spin_unlock(&info->ref_lock); + + spin_lock_irqsave(&info->dom_ids_lock, flag); + memset(info->dom_ids, 0, PSR_DOM_IDS_NUM * sizeof(uint32_t)); + spin_unlock_irqrestore(&info->dom_ids_lock, flag); } static bool feat_init_done(const struct psr_socket_info *info) @@ -682,9 +676,37 @@ void psr_ctxt_switch_to(struct domain *d) psr_assoc_rmid(®, d->arch.psr_rmid); if ( psra->cos_mask ) - psr_assoc_cos(®, d->arch.psr_cos_ids ? - d->arch.psr_cos_ids[cpu_to_socket(smp_processor_id())] : - 0, psra->cos_mask); + { + unsigned int socket = cpu_to_socket(smp_processor_id()); + struct psr_socket_info *info = socket_info + socket; + + if ( test_bit(d->domain_id, info->dom_ids) ) + goto set_assoc; + + spin_lock(&info->dom_ids_lock); + + int old_bit = test_and_set_bit(d->domain_id, info->dom_ids); + + /* + * If old_bit is 0, that means this is the first time the domain is + * switched to this socket or domain's COS ID has not been set since + * the socket is online. So, the domain's COS ID on this socket should + * be default value, 0. If not, that means this socket has been offline + * and the domain's COS ID has been set when the socket was online. So, + * this COS ID is invalid and we have to restore it to 0. + */ + if ( d->arch.psr_cos_ids && + old_bit == 0 && + d->arch.psr_cos_ids[socket] != 0 ) + d->arch.psr_cos_ids[socket] = 0; + + spin_unlock(&info->dom_ids_lock); + + set_assoc: + psr_assoc_cos(®, + d->arch.psr_cos_ids ? d->arch.psr_cos_ids[socket] : 0, + psra->cos_mask); + } if ( reg != psra->val ) { @@ -1146,40 +1168,6 @@ static int write_psr_msr(unsigned int socket, unsigned int cos, return 0; } -static void restore_default_val(unsigned int socket, unsigned int cos, - enum psr_feat_type feat_type) -{ - unsigned int i, j; - uint32_t default_val; - const struct psr_socket_info *info = get_socket_info(socket); - - for ( i = 0; i < PSR_SOCKET_MAX_FEAT; i++ ) - { - const struct feat_node *feat = info->features[i]; - /* - * There are four judgements: - * 1. Input 'feat_type' is valid so we have to get feature according to - * it. If current feature type (i) does not match 'feat_type', we - * need skip it, so continue to check next feature. - * 2. Input 'feat_type' is 'PSR_SOCKET_MAX_FEAT' which means we should - * handle all features in this case. So, go to next loop. - * 3. Do not need restore the COS value back to default if cos_num is 1, - * e.g. L3 CAT. Because next value setting will overwrite it. - * 4. 'feat' we got is NULL, continue. - */ - if ( ( feat_type != PSR_SOCKET_MAX_FEAT && feat_type != i ) || - !feat || feat->props->cos_num == 1 ) - continue; - - for ( j = 0; j < feat->props->cos_num; j++ ) - { - feat->props->get_val(feat, 0, feat->props->type[j], &default_val); - write_psr_msr(socket, cos, default_val, - feat->props->type[j], i); - } - } -} - /* The whole set process is protected by domctl_lock. */ int psr_set_val(struct domain *d, unsigned int socket, uint32_t val, enum cbm_type type) @@ -1191,6 +1179,7 @@ int psr_set_val(struct domain *d, unsigned int socket, struct psr_socket_info *info = get_socket_info(socket); unsigned int array_len; enum psr_feat_type feat_type; + unsigned long flag; if ( IS_ERR(info) ) return PTR_ERR(info); @@ -1286,22 +1275,6 @@ int psr_set_val(struct domain *d, unsigned int socket, ASSERT(!cos || ref[cos]); ASSERT(!old_cos || ref[old_cos]); ref[old_cos]--; - - /* - * Step 6: - * For features, e.g. CDP, which cos_num is more than 1, we have to - * restore the old_cos value back to default when ref[old_cos] is 0. - * Otherwise, user will see wrong values when this COS ID is reused. E.g. - * user wants to set DATA to 0x3ff for a new domain. He hopes to see the - * DATA is set to 0x3ff and CODE should be the default value, 0x7ff. But - * if the COS ID picked for this action is the one that has been used by - * other domain and the CODE has been set to 0x1ff. Then, user will see - * DATA: 0x3ff, CODE: 0x1ff. So, we have to restore COS values for features - * using multiple COSs. - */ - if ( old_cos && !ref[old_cos] ) - restore_default_val(socket, old_cos, feat_type); - spin_unlock(&info->ref_lock); /* @@ -1310,7 +1283,10 @@ int psr_set_val(struct domain *d, unsigned int socket, * which COS the domain is using on the socket. One domain can only use * one COS ID at same time on each socket. */ + spin_lock_irqsave(&info->dom_ids_lock, flag); d->arch.psr_cos_ids[socket] = cos; + test_and_set_bit(d->domain_id, info->dom_ids); + spin_unlock_irqrestore(&info->dom_ids_lock, flag); xfree(val_array); return ret; @@ -1336,6 +1312,7 @@ static void psr_free_cos(struct domain *d) for ( socket = 0; socket < nr_sockets; socket++ ) { struct psr_socket_info *info; + unsigned long flag; /* cos 0 is default one which does not need be handled. */ cos = d->arch.psr_cos_ids[socket]; @@ -1346,14 +1323,11 @@ static void psr_free_cos(struct domain *d) spin_lock(&info->ref_lock); ASSERT(!cos || info->cos_ref[cos]); info->cos_ref[cos]--; - /* - * The 'cos_ref[cos]' of 'd' is 0 now so we need restore corresponding - * COS registers to default value. Because this case happens when a - * domain is destroied, we need restore all features. - */ - if ( !info->cos_ref[cos] ) - restore_default_val(socket, cos, PSR_SOCKET_MAX_FEAT); spin_unlock(&info->ref_lock); + + spin_lock_irqsave(&info->dom_ids_lock, flag); + clear_bit(d->domain_id, info->dom_ids); + spin_unlock_irqrestore(&info->dom_ids_lock, flag); } xfree(d->arch.psr_cos_ids); @@ -1453,6 +1427,7 @@ static void psr_cpu_init(void) goto assoc_init; spin_lock_init(&info->ref_lock); + spin_lock_init(&info->dom_ids_lock); cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s); if ( regs.b & PSR_RESOURCE_TYPE_L3 )