From patchwork Thu May 16 21:37:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 10947047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E9A86C5 for ; Thu, 16 May 2019 21:40:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D7A6212D9 for ; Thu, 16 May 2019 21:40:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 010F028AEC; Thu, 16 May 2019 21:40:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 83A4E212D9 for ; Thu, 16 May 2019 21:40:15 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4l-0000Df-Ix; Thu, 16 May 2019 21:38:07 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4j-0000Da-TR for xen-devel@lists.xenproject.org; Thu, 16 May 2019 21:38:05 +0000 X-Inumbo-ID: e2ddeed2-7822-11e9-8980-bc764e045a96 Received: from mail-it1-f194.google.com (unknown [209.85.166.194]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e2ddeed2-7822-11e9-8980-bc764e045a96; Thu, 16 May 2019 21:38:04 +0000 (UTC) Received: by mail-it1-f194.google.com with SMTP id m3so4879143itl.1 for ; Thu, 16 May 2019 14:38:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=KL58/IRwGmE6I3Cm69naAG6NV+s6OX9WmMv6CAnobso=; b=i9eDZcOo81ua66OFMhHQxdUWRpy0iGqjrNxmtQy7BqkeJg8+pvoE04rPSJb0cVx7dH wi2b7AIHfGlgiVd7ns+N/io72KRqBRhVQIBqMR+7kQgZyNi1vTn7gAtbvV6PBtjZFUK2 vMpg/DV2KFD+QwV5ARgqJoV095v0YD2zfdW45iSjGdqa3Y4cfG2C5iGQC+GDl86LH4mv Xp3o8ctkn6gt0EVu+gfIqGxlw3Fse0ownKEFxXOxj+28Xq05uR9/hfruNpi2HMO/Ywo0 7FZh+HZABH3PS3ZJ5H2gKbiQ0QAgWEuZOGBth0fLN08PIRcPkduvrRAbNdXhzY8Sx9jt qJcQ== X-Gm-Message-State: APjAAAW5rJhSa31HF7eAPci4jQsZxODqGO9J3Jhj5TCHUP5DCRfuiGHM GLhQ9M6ddvCk8cc+At1Gxma2bW1M X-Google-Smtp-Source: APXvYqxTnIAO3yMvQc2JHXuTSk3z/isPJdv3U206RzA8VSetWgyMA3w3kxKIBIvD8BplCIWSmMDuGA== X-Received: by 2002:a05:660c:652:: with SMTP id y18mr15427333itk.93.1558042683567; Thu, 16 May 2019 14:38:03 -0700 (PDT) Received: from localhost.localdomain (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id j81sm2509701itj.26.2019.05.16.14.38.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 May 2019 14:38:02 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Thu, 16 May 2019 15:37:49 -0600 Message-Id: <20190516213752.1701-1-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v5 1/4] x86/mem_sharing: reorder when pages are unlocked and released X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel , Wei Liu , George Dunlap , Andrew Cooper , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Calling _put_page_type while also holding the page_lock for that page can cause a deadlock. The comment being dropped is incorrect since it's now out-of-date. Signed-off-by: Tamas K Lengyel Cc: Jan Beulich Cc: Andrew Cooper Cc: George Dunlap Cc: Wei Liu Cc: Roger Pau Monne --- This series is based on Andrew Cooper's x86-next branch v5: BUG_ON early before releasing references --- xen/arch/x86/mm/mem_sharing.c | 41 ++++++++++------------------------- 1 file changed, 11 insertions(+), 30 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index f16a3f5324..13b2f009d4 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -648,10 +648,6 @@ static int page_make_private(struct domain *d, struct page_info *page) return -EBUSY; } - /* We can only change the type if count is one */ - /* Because we are locking pages individually, we need to drop - * the lock here, while the page is typed. We cannot risk the - * race of page_unlock and then put_page_type. */ expected_type = (PGT_shared_page | PGT_validated | PGT_locked | 2); if ( page->u.inuse.type_info != expected_type ) { @@ -660,12 +656,11 @@ static int page_make_private(struct domain *d, struct page_info *page) return -EEXIST; } + mem_sharing_page_unlock(page); + /* Drop the final typecount */ put_page_and_type(page); - /* Now that we've dropped the type, we can unlock */ - mem_sharing_page_unlock(page); - /* Change the owner */ ASSERT(page_get_owner(page) == dom_cow); page_set_owner(page, d); @@ -900,6 +895,7 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, p2m_type_t smfn_type, cmfn_type; struct two_gfns tg; struct rmap_iterator ri; + unsigned long put_count = 0; get_two_gfns(sd, sgfn, &smfn_type, NULL, &smfn, cd, cgfn, &cmfn_type, NULL, &cmfn, 0, &tg); @@ -964,15 +960,6 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, goto err_out; } - /* Acquire an extra reference, for the freeing below to be safe. */ - if ( !get_page(cpage, dom_cow) ) - { - ret = -EOVERFLOW; - mem_sharing_page_unlock(secondpg); - mem_sharing_page_unlock(firstpg); - goto err_out; - } - /* Merge the lists together */ rmap_seed_iterator(cpage, &ri); while ( (gfn = rmap_iterate(cpage, &ri)) != NULL) @@ -984,13 +971,14 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, * Don't change the type of rmap for the client page. */ rmap_del(gfn, cpage, 0); rmap_add(gfn, spage); - put_page_and_type(cpage); + put_count++; d = get_domain_by_id(gfn->domain); BUG_ON(!d); BUG_ON(set_shared_p2m_entry(d, gfn->gfn, smfn)); put_domain(d); } ASSERT(list_empty(&cpage->sharing->gfns)); + BUG_ON(!put_count); /* Clear the rest of the shared state */ page_sharing_dispose(cpage); @@ -1002,7 +990,9 @@ static int share_pages(struct domain *sd, gfn_t sgfn, shr_handle_t sh, /* Free the client page */ if(test_and_clear_bit(_PGC_allocated, &cpage->count_info)) put_page(cpage); - put_page(cpage); + + while ( put_count-- ) + put_page_and_type(cpage); /* We managed to free a domain page. */ atomic_dec(&nr_shared_mfns); @@ -1167,20 +1157,11 @@ int __mem_sharing_unshare_page(struct domain *d, { if ( !last_gfn ) mem_sharing_gfn_destroy(page, d, gfn_info); - put_page_and_type(page); mem_sharing_page_unlock(page); - if ( last_gfn ) - { - if ( !get_page(page, dom_cow) ) - { - put_gfn(d, gfn); - domain_crash(d); - return -EOVERFLOW; - } - if ( test_and_clear_bit(_PGC_allocated, &page->count_info) ) - put_page(page); + if ( last_gfn && + test_and_clear_bit(_PGC_allocated, &page->count_info) ) put_page(page); - } + put_page_and_type(page); put_gfn(d, gfn); return 0; From patchwork Thu May 16 21:37:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 10947049 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9C9A96C5 for ; Thu, 16 May 2019 21:40:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BE2E212D9 for ; Thu, 16 May 2019 21:40:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 803D528AEC; Thu, 16 May 2019 21:40:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 00387212D9 for ; Thu, 16 May 2019 21:40:16 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4v-0000E8-72; Thu, 16 May 2019 21:38:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4u-0000Dv-Sh for xen-devel@lists.xenproject.org; Thu, 16 May 2019 21:38:16 +0000 X-Inumbo-ID: e747af14-7822-11e9-88f9-a3df1cc8208b Received: from mail-io1-f67.google.com (unknown [209.85.166.67]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e747af14-7822-11e9-88f9-a3df1cc8208b; Thu, 16 May 2019 21:38:11 +0000 (UTC) Received: by mail-io1-f67.google.com with SMTP id p2so3861134iol.2 for ; Thu, 16 May 2019 14:38:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CfagiBBbRAAgKvQm+vYn5nRDMWuN0zXgC5JSNfoZP04=; b=JN2ynqqp3PlpWc777qaV4BBu9/HujHG/vDAgtsPTZV2dLRKIw0ArH/EwIYQLRWxUK8 bzJVwN/8Nw4zMuNj+mlJb0zb+M8yumriIXVVT026b6qkIha2u9NypXX7JHCQoQemkTdv DcqDL5KuEufzqgXu5IKMwgNfCC0k9/UJ3i+ylkdaqAbtAAE4cH0GVBGxRYQXjpUxzONR j7RepBLTBeOZZtl6kl4hwSXm2+Tbm0rRMFbjQyUoo9N6L1r/ZxxjjHR8p7SHdYTD1x9Z JU6uJvBtlIgsTfu1kNlmzW7Efw6m/sKuK8caliCtQHOLKrJ9eiMJ4jr4SyIYn4ALrH9w lZew== X-Gm-Message-State: APjAAAWd1LlsmRC5A61ocYPJKY3cpp7mOOEmlCX2gPUCJD0dSiL4gBKK /utlzy/Y+mGM4k85zu2sNr0g+Usq X-Google-Smtp-Source: APXvYqyg8NZn9p7M7YKkHODLeSgoIsa9p+59N4+KuqH4a9Onmuk7N4gBhEPGyRMtLg1inD8p3xLojQ== X-Received: by 2002:a5e:a919:: with SMTP id c25mr5029534iod.166.1558042690986; Thu, 16 May 2019 14:38:10 -0700 (PDT) Received: from localhost.localdomain (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id j81sm2509701itj.26.2019.05.16.14.38.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 May 2019 14:38:10 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Thu, 16 May 2019 15:37:50 -0600 Message-Id: <20190516213752.1701-2-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190516213752.1701-1-tamas@tklengyel.com> References: <20190516213752.1701-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v5 2/4] x86/mem_sharing: copy a page_lock version to be internal to memshr X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel , Wei Liu , George Dunlap , Andrew Cooper , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Patch cf4b30dca0a "Add debug code to detect illegal page_lock and put_page_type ordering" added extra sanity checking to page_lock/page_unlock for debug builds with the assumption that no hypervisor path ever locks two pages at once. This assumption doesn't hold during memory sharing so we copy a version of page_lock/unlock to be used exclusively in the memory sharing subsystem without the sanity checks. Signed-off-by: Tamas K Lengyel Cc: George Dunlap Cc: Jan Beulich Cc: Andrew Cooper Cc: Wei Liu Cc: Roger Pau Monne Acked-by: Jan Beulich Acked-by: Jan Beulich --- v5: comments and TODOs --- xen/arch/x86/mm/mem_sharing.c | 54 ++++++++++++++++++++++++++++++++--- xen/include/asm-x86/mm.h | 15 ++-------- 2 files changed, 53 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/mm/mem_sharing.c b/xen/arch/x86/mm/mem_sharing.c index 13b2f009d4..f2354d2d6f 100644 --- a/xen/arch/x86/mm/mem_sharing.c +++ b/xen/arch/x86/mm/mem_sharing.c @@ -112,13 +112,59 @@ static inline void page_sharing_dispose(struct page_info *page) #endif /* MEM_SHARING_AUDIT */ -static inline int mem_sharing_page_lock(struct page_info *pg) +/* + * Private implementations of page_lock/unlock to bypass PV-only + * sanity checks not applicable to mem-sharing. + * + * _page_lock is used in memory sharing to protect addition (share) and removal + * (unshare) of (gfn,domain) tupples to a list of gfn's that the shared page is + * currently backing. + * Nesting may happen when sharing (and locking) two pages. + * Deadlock is avoided by locking pages in increasing order. + * All memory sharing code paths take the p2m lock of the affected gfn before + * taking the lock for the underlying page. We enforce ordering between page_lock + * and p2m_lock using an mm-locks.h construct. + * + * TODO: Investigate if PGT_validated is necessary. + */ +static inline bool _page_lock(struct page_info *page) { - int rc; + unsigned long x, nx; + + do { + while ( (x = page->u.inuse.type_info) & PGT_locked ) + cpu_relax(); + nx = x + (1 | PGT_locked); + if ( !(x & PGT_validated) || + !(x & PGT_count_mask) || + !(nx & PGT_count_mask) ) + return false; + } while ( cmpxchg(&page->u.inuse.type_info, x, nx) != x ); + + return true; +} + +static inline void _page_unlock(struct page_info *page) +{ + unsigned long x, nx, y = page->u.inuse.type_info; + + do { + x = y; + ASSERT((x & PGT_count_mask) && (x & PGT_locked)); + + nx = x - (1 | PGT_locked); + /* We must not drop the last reference here. */ + ASSERT(nx & PGT_count_mask); + } while ( (y = cmpxchg(&page->u.inuse.type_info, x, nx)) != x ); +} + +static inline bool mem_sharing_page_lock(struct page_info *pg) +{ + bool rc; pg_lock_data_t *pld = &(this_cpu(__pld)); page_sharing_mm_pre_lock(); - rc = page_lock(pg); + rc = _page_lock(pg); if ( rc ) { preempt_disable(); @@ -135,7 +181,7 @@ static inline void mem_sharing_page_unlock(struct page_info *pg) page_sharing_mm_unlock(pld->mm_unlock_level, &pld->recurse_count); preempt_enable(); - page_unlock(pg); + _page_unlock(pg); } static inline shr_handle_t get_next_handle(void) diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 6faa563167..24c4205ba7 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -356,24 +356,15 @@ struct platform_bad_page { const struct platform_bad_page *get_platform_badpages(unsigned int *array_size); /* Per page locks: - * page_lock() is used for two purposes: pte serialization, and memory sharing. + * page_lock() is used for pte serialization. * * All users of page lock for pte serialization live in mm.c, use it * to lock a page table page during pte updates, do not take other locks within * the critical section delimited by page_lock/unlock, and perform no * nesting. * - * All users of page lock for memory sharing live in mm/mem_sharing.c. Page_lock - * is used in memory sharing to protect addition (share) and removal (unshare) - * of (gfn,domain) tupples to a list of gfn's that the shared page is currently - * backing. Nesting may happen when sharing (and locking) two pages -- deadlock - * is avoided by locking pages in increasing order. - * All memory sharing code paths take the p2m lock of the affected gfn before - * taking the lock for the underlying page. We enforce ordering between page_lock - * and p2m_lock using an mm-locks.h construct. - * - * These two users (pte serialization and memory sharing) do not collide, since - * sharing is only supported for hvm guests, which do not perform pv pte updates. + * The use of PGT_locked in mem_sharing does not collide, since mem_sharing is + * only supported for hvm guests, which do not perform pv pte updates. */ int page_lock(struct page_info *page); void page_unlock(struct page_info *page); From patchwork Thu May 16 21:37:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 10947043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B50A1515 for ; Thu, 16 May 2019 21:39:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8ABE028B87 for ; Thu, 16 May 2019 21:39:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D83528CE4; Thu, 16 May 2019 21:39:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 730E228B87 for ; Thu, 16 May 2019 21:39:49 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4u-0000Dw-T1; Thu, 16 May 2019 21:38:16 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4t-0000Dq-GM for xen-devel@lists.xenproject.org; Thu, 16 May 2019 21:38:15 +0000 X-Inumbo-ID: e909b1c5-7822-11e9-8980-bc764e045a96 Received: from mail-it1-f194.google.com (unknown [209.85.166.194]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e909b1c5-7822-11e9-8980-bc764e045a96; Thu, 16 May 2019 21:38:14 +0000 (UTC) Received: by mail-it1-f194.google.com with SMTP id g71so8723660ita.5 for ; Thu, 16 May 2019 14:38:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=29kguvPIqID2QYtSaxUUcg0G2bgfjX7xxfa5dHdjQg4=; b=NzDCQI2+bRE6MNm3jeR5mpH7wjRwZd1wac0ib1PLjGNZb+fsQYk6Hj5joPrGOBweXD ShRAErnoyrwoFepUn9wYxnbnm51fxG72MjbyCgRif3NMX0cTpo4dxfz0vsAfgb/E3FkJ 1fsnTAaBLfiRSwFHAOb6Exu6eHsfGwWJl1L9CwSpfF+rQiRERfQjyePFvyYHIPJJx0ff QMCKNOrSoY6sRc03VOwtg8pM0F4L5gii91qlvn5GAmpqAUYChQucSbNHG+qv8TruIrJT UAHqAKmKcJdoPmWjwtZop52OTOs1T5obQfXRiQzTMRQWECWEc+dmRVKADI+mQalhkggP Gr6Q== X-Gm-Message-State: APjAAAX+hZMCPmE/OhuaFlCj50B813VhuFi70fDw6tQPe/QQwlVAKO5z bKE538ifbyPeOkd77dlYcCeeC6p+ X-Google-Smtp-Source: APXvYqzw4jWk8Q/Ok1a9cvBB/ke3ya1MIPLp8yBQr3u3B2LMkHlc9YLkDglMnR5zEPwo7vmJMuppVA== X-Received: by 2002:a02:3506:: with SMTP id k6mr3785074jaa.41.1558042693969; Thu, 16 May 2019 14:38:13 -0700 (PDT) Received: from localhost.localdomain (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id j81sm2509701itj.26.2019.05.16.14.38.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 May 2019 14:38:13 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Thu, 16 May 2019 15:37:51 -0600 Message-Id: <20190516213752.1701-3-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190516213752.1701-1-tamas@tklengyel.com> References: <20190516213752.1701-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v5 3/4] x86/mem_sharing: enable mem_share audit mode only in debug builds X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Tamas K Lengyel , Wei Liu , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Improves performance for release builds. Signed-off-by: Tamas K Lengyel Cc: Jan Beulich Cc: Andrew Cooper Cc: Wei Liu Cc: Roger Pau Monne --- xen/include/asm-x86/mem_sharing.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/xen/include/asm-x86/mem_sharing.h b/xen/include/asm-x86/mem_sharing.h index 9f9f7e93e3..afd0c17292 100644 --- a/xen/include/asm-x86/mem_sharing.h +++ b/xen/include/asm-x86/mem_sharing.h @@ -25,7 +25,11 @@ #include /* Auditing of memory sharing code? */ +#ifndef NDEBUG #define MEM_SHARING_AUDIT 1 +#else +#define MEM_SHARING_AUDIT 0 +#endif typedef uint64_t shr_handle_t; From patchwork Thu May 16 21:37:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamas K Lengyel X-Patchwork-Id: 10947045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5508C6C5 for ; Thu, 16 May 2019 21:39:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 442CD28B87 for ; Thu, 16 May 2019 21:39:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 385B528CE2; Thu, 16 May 2019 21:39:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3474428C8D for ; Thu, 16 May 2019 21:39:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4z-0000ES-GT; Thu, 16 May 2019 21:38:21 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hRO4y-0000EL-0u for xen-devel@lists.xenproject.org; Thu, 16 May 2019 21:38:20 +0000 X-Inumbo-ID: eacae5ac-7822-11e9-b24f-c7afc3af2d50 Received: from mail-it1-f193.google.com (unknown [209.85.166.193]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id eacae5ac-7822-11e9-b24f-c7afc3af2d50; Thu, 16 May 2019 21:38:17 +0000 (UTC) Received: by mail-it1-f193.google.com with SMTP id i63so8734644ita.3 for ; Thu, 16 May 2019 14:38:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AiQltoiasBAzM/aFYcnJZWxsWSV6I94tgOzlsqPHQiU=; b=EuEmZcHzVequAPnLymeLZkSj5MUqdsc3jpyeIxaFBiW6ND/b69N62gxkEMzQwPS4t5 T7MrehJZdGZ4ZkTvLc0wCA1C5NGKSlfBBx/yddswXTWmOKTWU5NOFHRbvRCfj+X7gEbH AoakhIcO/0jYtzNHL0XHWjpc6sY9QLS99TQy3fd72m9FR+Jn6edxRRf3NEtzV2zRok8q 0gQt5ZfkUmvHHFYPuNhatM7IffD2GRd0mxRtmlho0RsoiA/RZXGBhDszyeoJCPJU2uoa 9aszPlAVM7tC9ILZ019eExkIJFhn46Ea3y+jA6VEtq5F/uiSJ5ENmIMtWZtr/ADdh9MB Fabw== X-Gm-Message-State: APjAAAV2R5TEbBSrqdi+wtUuOm01aTNIVAhGV6Bqroor5Gb4iiPv4qAO 2YYGe/PmBVMS9U6loixZJIjg9wV3 X-Google-Smtp-Source: APXvYqwmJW3Ce3UrbrG7Hp5+jvjWIcimPjjj8z5aIyIKJjvtIBs9qZDNVDbaWBcYzdLzELTuIipdnA== X-Received: by 2002:a24:bd5:: with SMTP id 204mr16017517itd.134.1558042696860; Thu, 16 May 2019 14:38:16 -0700 (PDT) Received: from localhost.localdomain (c-71-205-12-124.hsd1.co.comcast.net. [71.205.12.124]) by smtp.gmail.com with ESMTPSA id j81sm2509701itj.26.2019.05.16.14.38.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 May 2019 14:38:16 -0700 (PDT) From: Tamas K Lengyel To: xen-devel@lists.xenproject.org Date: Thu, 16 May 2019 15:37:52 -0600 Message-Id: <20190516213752.1701-4-tamas@tklengyel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190516213752.1701-1-tamas@tklengyel.com> References: <20190516213752.1701-1-tamas@tklengyel.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v5 4/4] x86/mem_sharing: compile mem_sharing subsystem only when kconfig is enabled X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tamas K Lengyel , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Julien Grall , Stefano Stabellini , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Disable it by default as it is only an experimental subsystem. Signed-off-by: Tamas K Lengyel Cc: Jan Beulich Cc: Andrew Cooper Cc: Wei Liu Cc: Roger Pau Monne Cc: George Dunlap Cc: Ian Jackson Cc: Julien Grall Cc: Konrad Rzeszutek Wilk Cc: Stefano Stabellini Cc: Tim Deegan Cc: George Dunlap ---- v4: add ASSERT_UNREACHABLE to inlined functions where applicable & other fixups Acked-by: Jan Beulich Acked-by: Razvan Cojocaru Acked-by: Daniel De Graaf Acked-by: Razvan Cojocaru Acked-by: Jan Beulich Acked-by: Daniel De Graaf --- xen/arch/x86/Kconfig | 6 +++++- xen/arch/x86/domain.c | 2 ++ xen/arch/x86/domctl.c | 2 ++ xen/arch/x86/mm/Makefile | 2 +- xen/arch/x86/x86_64/compat/mm.c | 2 ++ xen/arch/x86/x86_64/mm.c | 2 ++ xen/common/Kconfig | 3 --- xen/common/domain.c | 2 +- xen/common/grant_table.c | 2 +- xen/common/memory.c | 2 +- xen/common/vm_event.c | 6 +++--- xen/include/asm-x86/mem_sharing.h | 28 ++++++++++++++++++++++++++++ xen/include/asm-x86/mm.h | 3 +++ xen/include/xen/sched.h | 2 +- xen/include/xsm/dummy.h | 2 +- xen/include/xsm/xsm.h | 4 ++-- xen/xsm/dummy.c | 2 +- xen/xsm/flask/hooks.c | 4 ++-- 18 files changed, 58 insertions(+), 18 deletions(-) diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 4b8b07b549..600ca5c12e 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -17,7 +17,6 @@ config X86 select HAS_KEXEC select MEM_ACCESS_ALWAYS_ON select HAS_MEM_PAGING - select HAS_MEM_SHARING select HAS_NS16550 select HAS_PASSTHROUGH select HAS_PCI @@ -198,6 +197,11 @@ config PV_SHIM_EXCLUSIVE firmware, and will not function correctly in other scenarios. If unsure, say N. + +config MEM_SHARING + bool "Xen memory sharing support" if EXPERT = "y" + depends on HVM + endmenu source "common/Kconfig" diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index d2d9f2fc3c..474df8433b 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -2046,6 +2046,7 @@ int domain_relinquish_resources(struct domain *d) d->arch.auto_unmask = 0; } +#ifdef CONFIG_MEM_SHARING PROGRESS(shared): if ( is_hvm_domain(d) ) @@ -2056,6 +2057,7 @@ int domain_relinquish_resources(struct domain *d) if ( ret ) return ret; } +#endif spin_lock(&d->page_alloc_lock); page_list_splice(&d->arch.relmem_list, &d->page_list); diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 9bf2d0820f..bc9e024ccc 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1231,9 +1231,11 @@ long arch_do_domctl( break; } +#ifdef CONFIG_MEM_SHARING case XEN_DOMCTL_mem_sharing_op: ret = mem_sharing_domctl(d, &domctl->u.mem_sharing_op); break; +#endif #if P2M_AUDIT && defined(CONFIG_HVM) case XEN_DOMCTL_audit_p2m: diff --git a/xen/arch/x86/mm/Makefile b/xen/arch/x86/mm/Makefile index 5a17646f98..5010a29d6c 100644 --- a/xen/arch/x86/mm/Makefile +++ b/xen/arch/x86/mm/Makefile @@ -6,7 +6,7 @@ obj-$(CONFIG_HVM) += guest_walk_2.o guest_walk_3.o guest_walk_4.o obj-$(CONFIG_SHADOW_PAGING) += guest_walk_2.o guest_walk_3.o guest_walk_4.o obj-$(CONFIG_MEM_ACCESS) += mem_access.o obj-y += mem_paging.o -obj-y += mem_sharing.o +obj-$(CONFIG_MEM_SHARING) += mem_sharing.o obj-y += p2m.o p2m-pt.o obj-$(CONFIG_HVM) += p2m-ept.o p2m-pod.o obj-y += paging.o diff --git a/xen/arch/x86/x86_64/compat/mm.c b/xen/arch/x86/x86_64/compat/mm.c index 32410ed273..d4c6be3032 100644 --- a/xen/arch/x86/x86_64/compat/mm.c +++ b/xen/arch/x86/x86_64/compat/mm.c @@ -152,8 +152,10 @@ int compat_arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) case XENMEM_paging_op: return mem_paging_memop(guest_handle_cast(arg, xen_mem_paging_op_t)); +#ifdef CONFIG_MEM_SHARING case XENMEM_sharing_op: return mem_sharing_memop(guest_handle_cast(arg, xen_mem_sharing_op_t)); +#endif default: rc = -ENOSYS; diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index d8f558bc3a..51d1d511f2 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -993,8 +993,10 @@ long subarch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg) case XENMEM_paging_op: return mem_paging_memop(guest_handle_cast(arg, xen_mem_paging_op_t)); +#ifdef CONFIG_MEM_SHARING case XENMEM_sharing_op: return mem_sharing_memop(guest_handle_cast(arg, xen_mem_sharing_op_t)); +#endif default: rc = -ENOSYS; diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 7a12346f19..6e28fc48af 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -45,9 +45,6 @@ config MEM_ACCESS config HAS_MEM_PAGING bool -config HAS_MEM_SHARING - bool - config HAS_PDX bool diff --git a/xen/common/domain.c b/xen/common/domain.c index 90c66079f9..0815cf8f07 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -923,7 +923,7 @@ static void complete_domain_destroy(struct rcu_head *head) xfree(d->vm_event_paging); #endif xfree(d->vm_event_monitor); -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING xfree(d->vm_event_share); #endif diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c index 80728ea57d..6c40dccae9 100644 --- a/xen/common/grant_table.c +++ b/xen/common/grant_table.c @@ -3760,7 +3760,7 @@ void grant_table_init_vcpu(struct vcpu *v) v->maptrack_tail = MAPTRACK_TAIL; } -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING int mem_sharing_gref_to_gfn(struct grant_table *gt, grant_ref_t ref, gfn_t *gfn, uint16_t *status) { diff --git a/xen/common/memory.c b/xen/common/memory.c index 520d6f4803..69851f4344 100644 --- a/xen/common/memory.c +++ b/xen/common/memory.c @@ -1671,7 +1671,7 @@ int check_get_page_from_gfn(struct domain *d, gfn_t gfn, bool readonly, return -EAGAIN; } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING if ( (q & P2M_UNSHARE) && p2m_is_shared(p2mt) ) { if ( page ) diff --git a/xen/common/vm_event.c b/xen/common/vm_event.c index 6e68be47bc..163a671cea 100644 --- a/xen/common/vm_event.c +++ b/xen/common/vm_event.c @@ -544,7 +544,7 @@ static void monitor_notification(struct vcpu *v, unsigned int port) vm_event_resume(v->domain, v->domain->vm_event_monitor); } -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING /* Registered with Xen-bound event channel for incoming notifications. */ static void mem_sharing_notification(struct vcpu *v, unsigned int port) { @@ -574,7 +574,7 @@ void vm_event_cleanup(struct domain *d) destroy_waitqueue_head(&d->vm_event_monitor->wq); (void)vm_event_disable(d, &d->vm_event_monitor); } -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING if ( vm_event_check_ring(d->vm_event_share) ) { destroy_waitqueue_head(&d->vm_event_share->wq); @@ -720,7 +720,7 @@ int vm_event_domctl(struct domain *d, struct xen_domctl_vm_event_op *vec, } break; -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING case XEN_DOMCTL_VM_EVENT_OP_SHARING: { rc = -EINVAL; diff --git a/xen/include/asm-x86/mem_sharing.h b/xen/include/asm-x86/mem_sharing.h index afd0c17292..db22468744 100644 --- a/xen/include/asm-x86/mem_sharing.h +++ b/xen/include/asm-x86/mem_sharing.h @@ -24,6 +24,8 @@ #include #include +#ifdef CONFIG_MEM_SHARING + /* Auditing of memory sharing code? */ #ifndef NDEBUG #define MEM_SHARING_AUDIT 1 @@ -99,4 +101,30 @@ int mem_sharing_domctl(struct domain *d, */ int relinquish_shared_pages(struct domain *d); +#else + +static inline unsigned int mem_sharing_get_nr_saved_mfns(void) +{ + return 0; +} +static inline unsigned int mem_sharing_get_nr_shared_mfns(void) +{ + return 0; +} +static inline int mem_sharing_unshare_page(struct domain *d, + unsigned long gfn, + uint16_t flags) +{ + ASSERT_UNREACHABLE(); + return -EOPNOTSUPP; +} +static inline int mem_sharing_notify_enomem(struct domain *d, unsigned long gfn, + bool allow_sleep) +{ + ASSERT_UNREACHABLE(); + return -EOPNOTSUPP; +} + +#endif + #endif /* __MEM_SHARING_H__ */ diff --git a/xen/include/asm-x86/mm.h b/xen/include/asm-x86/mm.h index 24c4205ba7..d8ab53d688 100644 --- a/xen/include/asm-x86/mm.h +++ b/xen/include/asm-x86/mm.h @@ -127,6 +127,8 @@ struct page_info /* For non-pinnable single-page shadows, a higher entry that points * at us. */ paddr_t up; + +#ifdef CONFIG_MEM_SHARING /* For shared/sharable pages, we use a doubly-linked list * of all the {pfn,domain} pairs that map this page. We also include * an opaque handle, which is effectively a version, so that clients @@ -134,6 +136,7 @@ struct page_info * This list is allocated and freed when a page is shared/unshared. */ struct page_sharing_info *sharing; +#endif }; /* Reference count and various PGC_xxx flags and fields. */ diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 2201faca6b..410c74489f 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -459,7 +459,7 @@ struct domain /* Various vm_events */ /* Memory sharing support */ -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING struct vm_event_domain *vm_event_share; #endif /* Memory paging support */ diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h index 01d2814fed..ef52bb1764 100644 --- a/xen/include/xsm/dummy.h +++ b/xen/include/xsm/dummy.h @@ -597,7 +597,7 @@ static XSM_INLINE int xsm_mem_paging(XSM_DEFAULT_ARG struct domain *d) } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING static XSM_INLINE int xsm_mem_sharing(XSM_DEFAULT_ARG struct domain *d) { XSM_ASSERT_ACTION(XSM_DM_PRIV); diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h index 9a90355056..d61ba77374 100644 --- a/xen/include/xsm/xsm.h +++ b/xen/include/xsm/xsm.h @@ -150,7 +150,7 @@ struct xsm_operations { int (*mem_paging) (struct domain *d); #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING int (*mem_sharing) (struct domain *d); #endif @@ -597,7 +597,7 @@ static inline int xsm_mem_paging (xsm_default_t def, struct domain *d) } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING static inline int xsm_mem_sharing (xsm_default_t def, struct domain *d) { return xsm_ops->mem_sharing(d); diff --git a/xen/xsm/dummy.c b/xen/xsm/dummy.c index c9a566f2b5..5705e52791 100644 --- a/xen/xsm/dummy.c +++ b/xen/xsm/dummy.c @@ -128,7 +128,7 @@ void __init xsm_fixup_ops (struct xsm_operations *ops) set_to_dummy_if_null(ops, mem_paging); #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING set_to_dummy_if_null(ops, mem_sharing); #endif diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index a7d690ac3c..791c1f66af 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -1262,7 +1262,7 @@ static int flask_mem_paging(struct domain *d) } #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING static int flask_mem_sharing(struct domain *d) { return current_has_perm(d, SECCLASS_DOMAIN2, DOMAIN2__MEM_SHARING); @@ -1829,7 +1829,7 @@ static struct xsm_operations flask_ops = { .mem_paging = flask_mem_paging, #endif -#ifdef CONFIG_HAS_MEM_SHARING +#ifdef CONFIG_MEM_SHARING .mem_sharing = flask_mem_sharing, #endif