From patchwork Thu Oct 21 15:41:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 12575629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1699FC4332F for ; Thu, 21 Oct 2021 15:41:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F1D47611C7 for ; Thu, 21 Oct 2021 15:41:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232051AbhJUPoD (ORCPT ); Thu, 21 Oct 2021 11:44:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229550AbhJUPnq (ORCPT ); Thu, 21 Oct 2021 11:43:46 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54189C061243; Thu, 21 Oct 2021 08:41:30 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d5so1022790pfu.1; Thu, 21 Oct 2021 08:41:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cUs+nJBemKyqHW1/+tBNnIBzfXId6r+3EciJyXUHzJs=; b=mb0JzLn6mSIZIpkjv2tkYN3XoDwClOvTasP/YSQ5DSjukGGRylBXYo7vOaPWHhniUm /nl8o14LxOnWMymuDnq+kkYlmRpcZrzVeZThes4LI+n6tlqlPQeHvlUxM/+xHl0OBV62 3JKfF7STiZwdRKtuuQwCFyoBu8VC8sJinNoTmws3JyElVs6ys2pLgkaufurRUIP10Cf6 Pu0zh3ZnjjN7UfSsPF9PQf7ei3jHwsY4ER6K7zDlC2eRBIQkniR8uwSYY2PvCnEi0Y5d RWT1Bkuf4naKZkQk+O+GW6pfSxIvMh4xZvLwgGCgk/Ii96vKhGYRkIVnaepKtOSOFSXX Vzww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cUs+nJBemKyqHW1/+tBNnIBzfXId6r+3EciJyXUHzJs=; b=rPvRA6rgDyVAPCTcvv37iK7BFaGV6YvT8vDF6M5PU5M6GLEM0SqQ8iS+351aM6aAWO oephZ3ifjwCTmNCbpNu59TfZQA+7VRMfCJ6CvMTVk0bAmJTy+C+gZjpKQjixZyg5Nvnq 5oB43ffWvo9lKFfGLQJUNTZcrONCPSZ9HDL1Ve/zR8Y4qRcYCDp9jXeUqiv/SyDX36zP Ysdp+DXbsXyt0rpuTXBJfCCz7uA308vJl46+fqI9FKXRFwz6csB83Hd5K58htqfhHq2p BfuNO3l0y2+ApmbYVK81F/nBsEdZ8O/sZ2X72r1x9lqjo3SrrdupkhQRhur9H8xlsqRp t3JA== X-Gm-Message-State: AOAM530OBSYbcb7h4j4kf5ilnZJFTtloi3VcsHYb98DP0SV5tP0gaoGw tMOnchcPzr55fzB/tk5y3l8= X-Google-Smtp-Source: ABdhPJyOBIXB9R2rYwI+Yj9DVi0eXOZaWxDY9mU8rpQ+yFLMs5jEPhcbNIajbwq+oramkw81DaqX5g== X-Received: by 2002:a05:6a00:1801:b0:44c:aab8:a5ba with SMTP id y1-20020a056a00180100b0044caab8a5bamr6444801pfa.32.1634830889653; Thu, 21 Oct 2021 08:41:29 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:f:a76d:53a5:b89f:c2a0]) by smtp.gmail.com with ESMTPSA id p9sm6384130pfn.7.2021.10.21.08.41.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Oct 2021 08:41:29 -0700 (PDT) From: Tianyu Lan To: kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, davem@davemloft.net, kuba@kernel.org, gregkh@linuxfoundation.org, arnd@arndb.de, brijesh.singh@amd.com, jroedel@suse.de, Tianyu.Lan@microsoft.com, thomas.lendacky@amd.com, rientjes@google.com, pgonda@google.com, akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, rppt@kernel.org, saravanand@fb.com, aneesh.kumar@linux.ibm.com, hannes@cmpxchg.org, tj@kernel.org, michael.h.kelley@microsoft.com Cc: linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, vkuznets@redhat.com, konrad.wilk@oracle.com, hch@lst.de, robin.murphy@arm.com, joro@8bytes.org, parri.andrea@gmail.com, dave.hansen@intel.com Subject: [PATCH V8 9/9] Drivers: hv : vmbus: Initialize VMbus ring buffer for Isolation VM Date: Thu, 21 Oct 2021 11:41:09 -0400 Message-Id: <20211021154110.3734294-10-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211021154110.3734294-1-ltykernel@gmail.com> References: <20211021154110.3734294-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tianyu Lan VMbus ring buffer are shared with host and it's need to be accessed via extra address space of Isolation VM with AMD SNP support. This patch is to map the ring buffer address in extra address space via vmap_pfn(). Hyperv set memory host visibility hvcall smears data in the ring buffer and so reset the ring buffer memory to zero after mapping. Signed-off-by: Tianyu Lan --- Change since v4: * Use PFN_DOWN instead of HVPFN_DOWN in the hv_ringbuffer_init() Change since v3: * Remove hv_ringbuffer_post_init(), merge map operation for Isolation VM into hv_ringbuffer_init() * Call hv_ringbuffer_init() after __vmbus_establish_gpadl(). --- drivers/hv/Kconfig | 1 + drivers/hv/channel.c | 19 +++++++------- drivers/hv/ring_buffer.c | 55 ++++++++++++++++++++++++++++++---------- 3 files changed, 53 insertions(+), 22 deletions(-) diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig index d1123ceb38f3..dd12af20e467 100644 --- a/drivers/hv/Kconfig +++ b/drivers/hv/Kconfig @@ -8,6 +8,7 @@ config HYPERV || (ARM64 && !CPU_BIG_ENDIAN)) select PARAVIRT select X86_HV_CALLBACK_VECTOR if X86 + select VMAP_PFN help Select this option to run Linux as a Hyper-V client operating system. diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c index b37ff4a39224..dc5c35210c16 100644 --- a/drivers/hv/channel.c +++ b/drivers/hv/channel.c @@ -683,15 +683,6 @@ static int __vmbus_open(struct vmbus_channel *newchannel, if (!newchannel->max_pkt_size) newchannel->max_pkt_size = VMBUS_DEFAULT_MAX_PKT_SIZE; - err = hv_ringbuffer_init(&newchannel->outbound, page, send_pages, 0); - if (err) - goto error_clean_ring; - - err = hv_ringbuffer_init(&newchannel->inbound, &page[send_pages], - recv_pages, newchannel->max_pkt_size); - if (err) - goto error_clean_ring; - /* Establish the gpadl for the ring buffer */ newchannel->ringbuffer_gpadlhandle.gpadl_handle = 0; @@ -703,6 +694,16 @@ static int __vmbus_open(struct vmbus_channel *newchannel, if (err) goto error_clean_ring; + err = hv_ringbuffer_init(&newchannel->outbound, + page, send_pages, 0); + if (err) + goto error_free_gpadl; + + err = hv_ringbuffer_init(&newchannel->inbound, &page[send_pages], + recv_pages, newchannel->max_pkt_size); + if (err) + goto error_free_gpadl; + /* Create and init the channel open message */ open_info = kzalloc(sizeof(*open_info) + sizeof(struct vmbus_channel_open_channel), diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c index 314015d9e912..931802ae985c 100644 --- a/drivers/hv/ring_buffer.c +++ b/drivers/hv/ring_buffer.c @@ -17,6 +17,8 @@ #include #include #include +#include +#include #include "hyperv_vmbus.h" @@ -183,8 +185,10 @@ void hv_ringbuffer_pre_init(struct vmbus_channel *channel) int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info, struct page *pages, u32 page_cnt, u32 max_pkt_size) { - int i; struct page **pages_wraparound; + unsigned long *pfns_wraparound; + u64 pfn; + int i; BUILD_BUG_ON((sizeof(struct hv_ring_buffer) != PAGE_SIZE)); @@ -192,23 +196,48 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info, * First page holds struct hv_ring_buffer, do wraparound mapping for * the rest. */ - pages_wraparound = kcalloc(page_cnt * 2 - 1, sizeof(struct page *), - GFP_KERNEL); - if (!pages_wraparound) - return -ENOMEM; + if (hv_isolation_type_snp()) { + pfn = page_to_pfn(pages) + + PFN_DOWN(ms_hyperv.shared_gpa_boundary); + + pfns_wraparound = kcalloc(page_cnt * 2 - 1, + sizeof(unsigned long), GFP_KERNEL); + if (!pfns_wraparound) + return -ENOMEM; + + pfns_wraparound[0] = pfn; + for (i = 0; i < 2 * (page_cnt - 1); i++) + pfns_wraparound[i + 1] = pfn + i % (page_cnt - 1) + 1; - pages_wraparound[0] = pages; - for (i = 0; i < 2 * (page_cnt - 1); i++) - pages_wraparound[i + 1] = &pages[i % (page_cnt - 1) + 1]; + ring_info->ring_buffer = (struct hv_ring_buffer *) + vmap_pfn(pfns_wraparound, page_cnt * 2 - 1, + PAGE_KERNEL); + kfree(pfns_wraparound); - ring_info->ring_buffer = (struct hv_ring_buffer *) - vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, PAGE_KERNEL); + if (!ring_info->ring_buffer) + return -ENOMEM; + + /* Zero ring buffer after setting memory host visibility. */ + memset(ring_info->ring_buffer, 0x00, PAGE_SIZE * page_cnt); + } else { + pages_wraparound = kcalloc(page_cnt * 2 - 1, + sizeof(struct page *), + GFP_KERNEL); + + pages_wraparound[0] = pages; + for (i = 0; i < 2 * (page_cnt - 1); i++) + pages_wraparound[i + 1] = + &pages[i % (page_cnt - 1) + 1]; - kfree(pages_wraparound); + ring_info->ring_buffer = (struct hv_ring_buffer *) + vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, + PAGE_KERNEL); + kfree(pages_wraparound); + if (!ring_info->ring_buffer) + return -ENOMEM; + } - if (!ring_info->ring_buffer) - return -ENOMEM; ring_info->ring_buffer->read_index = ring_info->ring_buffer->write_index = 0;