From patchwork Wed Oct 23 13:29:25 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 3088701 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 9B2AA9F2B8 for ; Wed, 23 Oct 2013 13:33:50 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7721E203C4 for ; Wed, 23 Oct 2013 13:33:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 33E75203AF for ; Wed, 23 Oct 2013 13:33:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754530Ab3JWNdV (ORCPT ); Wed, 23 Oct 2013 09:33:21 -0400 Received: from e23smtp04.au.ibm.com ([202.81.31.146]:54694 "EHLO e23smtp04.au.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753882Ab3JWN37 (ORCPT ); Wed, 23 Oct 2013 09:29:59 -0400 Received: from /spool/local by e23smtp04.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 Oct 2013 23:29:58 +1000 Received: from d23dlp01.au.ibm.com (202.81.31.203) by e23smtp04.au.ibm.com (202.81.31.210) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 Oct 2013 23:29:56 +1000 Received: from d23relay04.au.ibm.com (d23relay04.au.ibm.com [9.190.234.120]) by d23dlp01.au.ibm.com (Postfix) with ESMTP id D88AF2CE8051; Thu, 24 Oct 2013 00:29:55 +1100 (EST) Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r9NDCgU055050440; Thu, 24 Oct 2013 00:12:42 +1100 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id r9NDTtqG008531; Thu, 24 Oct 2013 00:29:55 +1100 Received: from localhost (ericxiao.cn.ibm.com [9.111.29.64]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id r9NDTspb008510; Thu, 24 Oct 2013 00:29:54 +1100 From: Xiao Guangrong To: gleb@redhat.com Cc: avi.kivity@gmail.com, mtosatti@redhat.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Xiao Guangrong Subject: [PATCH v3 07/15] KVM: MMU: introduce nulls desc Date: Wed, 23 Oct 2013 21:29:25 +0800 Message-Id: <1382534973-13197-8-git-send-email-xiaoguangrong@linux.vnet.ibm.com> X-Mailer: git-send-email 1.8.1.4 In-Reply-To: <1382534973-13197-1-git-send-email-xiaoguangrong@linux.vnet.ibm.com> References: <1382534973-13197-1-git-send-email-xiaoguangrong@linux.vnet.ibm.com> X-TM-AS-MML: No X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13102313-9264-0000-0000-000004C927F5 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It likes nulls list and we use the pte-list as the nulls which can help us to detect whether the "desc" is moved to anther rmap then we can re-walk the rmap if that happened kvm->slots_lock is held when we do lockless walking that prevents rmap is reused (free rmap need to hold that lock) so that we can not see the same nulls used on different rmaps Signed-off-by: Xiao Guangrong --- arch/x86/kvm/mmu.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index 5cce039..4687329 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -913,6 +913,24 @@ static int mapping_level(struct kvm_vcpu *vcpu, gfn_t large_gfn) return level - 1; } +static void desc_mark_nulls(unsigned long *pte_list, struct pte_list_desc *desc) +{ + unsigned long marker; + + marker = (unsigned long)pte_list | 1UL; + desc->more = (struct pte_list_desc *)marker; +} + +static bool desc_is_a_nulls(struct pte_list_desc *desc) +{ + return (unsigned long)desc & 1; +} + +static unsigned long *desc_get_nulls_value(struct pte_list_desc *desc) +{ + return (unsigned long *)((unsigned long)desc & ~1); +} + static int __find_first_free(struct pte_list_desc *desc) { int i; @@ -951,7 +969,7 @@ static int count_spte_number(struct pte_list_desc *desc) first_free = __find_first_free(desc); - for (desc_num = 0; desc->more; desc = desc->more) + for (desc_num = 0; !desc_is_a_nulls(desc->more); desc = desc->more) desc_num++; return first_free + desc_num * PTE_LIST_EXT; @@ -985,6 +1003,7 @@ static int pte_list_add(struct kvm_vcpu *vcpu, u64 *spte, desc = mmu_alloc_pte_list_desc(vcpu); desc->sptes[0] = (u64 *)*pte_list; desc->sptes[1] = spte; + desc_mark_nulls(pte_list, desc); *pte_list = (unsigned long)desc | 1; return 1; } @@ -1030,7 +1049,7 @@ pte_list_desc_remove_entry(unsigned long *pte_list, /* * Only one entry existing but still use a desc to store it? */ - WARN_ON(!next_desc); + WARN_ON(desc_is_a_nulls(next_desc)); mmu_free_pte_list_desc(first_desc); *pte_list = (unsigned long)next_desc | 1ul; @@ -1041,7 +1060,7 @@ pte_list_desc_remove_entry(unsigned long *pte_list, * Only one entry in this desc, move the entry to the head * then the desc can be freed. */ - if (!first_desc->sptes[1] && !first_desc->more) { + if (!first_desc->sptes[1] && desc_is_a_nulls(first_desc->more)) { *pte_list = (unsigned long)first_desc->sptes[0]; mmu_free_pte_list_desc(first_desc); } @@ -1070,7 +1089,7 @@ static void pte_list_remove(u64 *spte, unsigned long *pte_list) rmap_printk("pte_list_remove: %p many->many\n", spte); desc = (struct pte_list_desc *)(*pte_list & ~1ul); - while (desc) { + while (!desc_is_a_nulls(desc)) { for (i = 0; i < PTE_LIST_EXT && desc->sptes[i]; ++i) if (desc->sptes[i] == spte) { pte_list_desc_remove_entry(pte_list, @@ -1097,11 +1116,13 @@ static void pte_list_walk(unsigned long *pte_list, pte_list_walk_fn fn) return fn((u64 *)*pte_list); desc = (struct pte_list_desc *)(*pte_list & ~1ul); - while (desc) { + while (!desc_is_a_nulls(desc)) { for (i = 0; i < PTE_LIST_EXT && desc->sptes[i]; ++i) fn(desc->sptes[i]); desc = desc->more; } + + WARN_ON(desc_get_nulls_value(desc) != pte_list); } static unsigned long *__gfn_to_rmap(gfn_t gfn, int level, @@ -1184,6 +1205,7 @@ static u64 *rmap_get_first(unsigned long rmap, struct rmap_iterator *iter) iter->desc = (struct pte_list_desc *)(rmap & ~1ul); iter->pos = 0; + WARN_ON(desc_is_a_nulls(iter->desc)); return iter->desc->sptes[iter->pos]; } @@ -1204,7 +1226,8 @@ static u64 *rmap_get_next(struct rmap_iterator *iter) return sptep; } - iter->desc = iter->desc->more; + iter->desc = desc_is_a_nulls(iter->desc->more) ? + NULL : iter->desc->more; if (iter->desc) { iter->pos = 0;