From patchwork Fri Jan 24 15:31:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11350589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E02C924 for ; Fri, 24 Jan 2020 15:32:53 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5AA2720838 for ; Fri, 24 Jan 2020 15:32:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="LxYPdoxC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AA2720838 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iv0w7-00070u-LN; Fri, 24 Jan 2020 15:31:55 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iv0w6-00070Q-O2 for xen-devel@lists.xenproject.org; Fri, 24 Jan 2020 15:31:54 +0000 X-Inumbo-ID: a5f5290c-3ebe-11ea-8032-12813bfff9fa Received: from smtp-fw-9101.amazon.com (unknown [207.171.184.25]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id a5f5290c-3ebe-11ea-8032-12813bfff9fa; Fri, 24 Jan 2020 15:31:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1579879915; x=1611415915; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0O9XWa6PMSsPO7G+77ZbS/9zJksGKayRB8cqOasAY18=; b=LxYPdoxCbBJ3kqj7dokL08dQTIWGJp8Ub1M1EIqaOiibVb2F7yZpTwl9 GL7+5hdZWLhDIqYjT9S6c0QT/RMzcqbgXRUW5QdKNsIRDDQdSYdsLhNva 6FgBLQgoLjlk8bqYarXYA1zbIDPjBgCmX3K3ns4wLG2kAthyxjoPTUTQq A=; IronPort-SDR: AOKaeCaOnPK00bWEz1iV4yK9cn+3o3TTicpOQ7M+6hlwBfMXX7Q4a2w7zdV0+pGp0qkR2x8F8V YaBCX1OwveYA== X-IronPort-AV: E=Sophos;i="5.70,358,1574121600"; d="scan'208";a="12427568" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-e7be2041.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 24 Jan 2020 15:31:41 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2a-e7be2041.us-west-2.amazon.com (Postfix) with ESMTPS id B8BCBA21D2; Fri, 24 Jan 2020 15:31:39 +0000 (UTC) Received: from EX13D32EUC004.ant.amazon.com (10.43.164.121) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1236.3; Fri, 24 Jan 2020 15:31:24 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D32EUC004.ant.amazon.com (10.43.164.121) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 24 Jan 2020 15:31:23 +0000 Received: from u2f063a87eabd5f.cbg10.amazon.com (10.125.106.135) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 24 Jan 2020 15:31:20 +0000 From: Paul Durrant To: Date: Fri, 24 Jan 2020 15:31:01 +0000 Message-ID: <20200124153103.18321-6-pdurrant@amazon.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200124153103.18321-1-pdurrant@amazon.com> References: <20200124153103.18321-1-pdurrant@amazon.com> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v4 5/7] mm: make MEMF_no_refcount pages safe to assign X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Paul Durrant , Ian Jackson , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Currently it is unsafe to assign a domheap page allocated with MEMF_no_refcount to a domain because the domain't 'tot_pages' will not be incremented, but will be decrement when the page is freed (since free_domheap_pages() has no way of telling that the increment was skipped). This patch allocates a new 'count_info' bit for a PGC_no_refcount flag which is then used to mark domheap pages allocated with MEMF_no_refcount. This then allows free_domheap_pages() to skip decrementing tot_pages when appropriate and hence makes the pages safe to assign. NOTE: The patch sets MEMF_no_refcount directly in alloc_domheap_pages() rather than in assign_pages() because the latter is called with MEMF_no_refcount by memory_exchange() as an optimization, to avoid too many calls to domain_adjust_tot_pages() (which acquires and releases the global 'heap_lock'). Signed-off-by: Paul Durrant --- Cc: Andrew Cooper Cc: George Dunlap Cc: Ian Jackson Cc: Jan Beulich Cc: Julien Grall Cc: Konrad Rzeszutek Wilk Cc: Stefano Stabellini Cc: Wei Liu Cc: Volodymyr Babchuk Cc: "Roger Pau Monné" v4: - New in v4 --- xen/common/page_alloc.c | 39 ++++++++++++++++++++++++++++----------- xen/include/asm-arm/mm.h | 5 ++++- xen/include/asm-x86/mm.h | 7 +++++-- 3 files changed, 37 insertions(+), 14 deletions(-) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 919a270587..6103f14a4e 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -460,6 +460,9 @@ unsigned long domain_adjust_tot_pages(struct domain *d, long pages) { long dom_before, dom_after, dom_claimed, sys_before, sys_after; + if ( !pages ) + goto out; + ASSERT(spin_is_locked(&d->page_alloc_lock)); d->tot_pages += pages; @@ -1389,6 +1392,8 @@ static void free_heap_pages( for ( i = 0; i < (1 << order); i++ ) { + ASSERT(!(pg[i].count_info & PGC_no_refcount)); + /* * Cannot assume that count_info == 0, as there are some corner cases * where it isn't the case and yet it isn't a bug: @@ -2314,11 +2319,6 @@ struct page_info *alloc_domheap_pages( if ( memflags & MEMF_no_owner ) memflags |= MEMF_no_refcount; - else if ( (memflags & MEMF_no_refcount) && d ) - { - ASSERT(!(memflags & MEMF_no_refcount)); - return NULL; - } if ( !dma_bitsize ) memflags &= ~MEMF_no_dma; @@ -2331,11 +2331,20 @@ struct page_info *alloc_domheap_pages( memflags, d)) == NULL)) ) return NULL; - if ( d && !(memflags & MEMF_no_owner) && - assign_pages(d, pg, order, memflags) ) + if ( d && !(memflags & MEMF_no_owner) ) { - free_heap_pages(pg, order, memflags & MEMF_no_scrub); - return NULL; + if ( assign_pages(d, pg, order, memflags) ) + { + free_heap_pages(pg, order, memflags & MEMF_no_scrub); + return NULL; + } + if ( memflags & MEMF_no_refcount ) + { + unsigned long i; + + for ( i = 0; i < (1 << order); i++ ) + pg[i].count_info |= PGC_no_refcount; + } } return pg; @@ -2368,24 +2377,32 @@ void free_domheap_pages(struct page_info *pg, unsigned int order) if ( likely(d) && likely(d != dom_cow) ) { + long pages = 0; + /* NB. May recursively lock from relinquish_memory(). */ spin_lock_recursive(&d->page_alloc_lock); for ( i = 0; i < (1 << order); i++ ) { + unsigned long count_info = pg[i].count_info; + if ( pg[i].u.inuse.type_info & PGT_count_mask ) { printk(XENLOG_ERR "pg[%u] MFN %"PRI_mfn" c=%#lx o=%u v=%#lx t=%#x\n", i, mfn_x(page_to_mfn(pg + i)), - pg[i].count_info, pg[i].v.free.order, + count_info, pg[i].v.free.order, pg[i].u.free.val, pg[i].tlbflush_timestamp); BUG(); } arch_free_heap_page(d, &pg[i]); + if ( count_info & PGC_no_refcount ) + pg[i].count_info &= ~PGC_no_refcount; + else + pages--; } - drop_dom_ref = !domain_adjust_tot_pages(d, -(1 << order)); + drop_dom_ref = !domain_adjust_tot_pages(d, pages); spin_unlock_recursive(&d->page_alloc_lock); diff --git a/xen/include/asm-arm/mm.h b/xen/include/asm-arm/mm.h index 333efd3a60..1076cc9713 100644 --- a/xen/include/asm-arm/mm.h +++ b/xen/include/asm-arm/mm.h @@ -119,9 +119,12 @@ struct page_info #define PGC_state_offlined PG_mask(2, 9) #define PGC_state_free PG_mask(3, 9) #define page_state_is(pg, st) (((pg)->count_info&PGC_state) == PGC_state_##st) +/* Page is not reference counted */ +#define _PGC_no_refcount PG_shift(10) +#define PGC_no_refcount PG_mask(1, 10) /* Count of references to this frame. */ -#define PGC_count_width PG_shift(9) +#define PGC_count_width PG_shift(10) #define PGC_count_mask ((1UL<count_info&PGC_state) == PGC_state_##st) +/* Page is not reference counted */ +#define _PGC_no_refcount PG_shift(10) +#define PGC_no_refcount PG_mask(1, 10) - /* Count of references to this frame. */ -#define PGC_count_width PG_shift(9) +/* Count of references to this frame. */ +#define PGC_count_width PG_shift(10) #define PGC_count_mask ((1UL<