From patchwork Fri May 17 14:46:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xuewei Niu X-Patchwork-Id: 13667055 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 458086A33B; Fri, 17 May 2024 14:46:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957184; cv=none; b=txnYOVBhN/9XPs0H3QJgu3K7d268h2aJMmHTa/dptREF41+g/gomNoqN5w3F3rMl1ttxzgu22ib6doesN5wVnDNcfKu5001cUj99TmycOnlYoM3e4Zy9f2YPQRJMThXnePqOcuh5qVdcqeZPdsOB9Xg9mt/f7dCXaHF+VYuO6Kc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957184; c=relaxed/simple; bh=j2/GYQH+YPgVz6Sz+FMKMs1AQ1hkT2Fqf7GxI/sZoUs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=NfdN9GVuU4pzQHtveK9uUHxXVyhtWjrZ6NRmx9WpgmmvkrylsW9ekKkRLhcGdM4vbn7qwnZ/td5S3cT+CCZqCwNadZQcJKPsa4nIkBTKyzpGpGZ6wgEjmuh85+xxuh8XpGZlkZghO+goiMubtqSegTcEl7GMPoz4nSWNgmip0n4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ES+ggZC7; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ES+ggZC7" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1ee38966529so4907825ad.1; Fri, 17 May 2024 07:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715957182; x=1716561982; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k0yGtcKEuophyxuK9PPE+gmzXonmfacqDZH3bhCr3sI=; b=ES+ggZC7qHShyfUp6U+2tRYWlvhFrLzRTNc7QMvD62gCVMHNAD8e5Smx3yYsDcPyPs kCGH9XpJkmocR+YXGBHhCLfJz5atbAkUnkvPjteMMW8sGXWxN3J7kj+LQmvrGPTAQwLL M8oe5eQESxbLZd7sBZ9U81uDGrr5yMh2Yn2pXuXIVvGycVjp0/GqTCHb2cTZvFNe24GR LJ+pkXrMC/eEqvQfu+ja5GKqhv3qQmm+9eFkVDUKa+WMAM7lsprZqHSHjGqKfE7E+P9A vYCm+W7/0SJEcJfFslkcQv5gpei0MTKJWG+KfyZLbtPKhJ+D08a3HLJGPqpTEVWnFnwJ nUbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715957182; x=1716561982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k0yGtcKEuophyxuK9PPE+gmzXonmfacqDZH3bhCr3sI=; b=c5GAbnTQ7NhRtJdNF8ViQ15RMlv6ll6BggIYcO3IPIjLFMrsORNDti9HYdM0HXur6E r3tEh7i2cShJ1tipL+a8aK41uitzwwbls6invHMqsXjmcNk5o8aZ2fn+bfgCDMPKm7HM PsNHIJYc/pPHzdZms/qZnmRk/Fe5Sp2aOeGE/0kbXpmGzsEX9jEO+bAlmLeDVD7D5YY8 lNgQZeSglKSqVGbFCbIQ7jOJcvKS1nrSmzdPxpzmxJOhTGYQiH9E14CzvYJ71aQxIpDg fqOJc9HrVvzYqlrd3b06nObFPcRwM72gAA66LO4Ic0oEdyJuggR+d/HYBH/b7+rssmxn 7V8w== X-Forwarded-Encrypted: i=1; AJvYcCUt/RrFw30zSnBlMBNLt04j2HDUA2x86ZV9B5HFC6ZlLUQ0SNgfCL8RkeEn6Oo9HY12kJERdocd6+gUft/9pPRz/MVnrZowS7TlgQwAMdvoFgVDCVrC+j8DPoA0ADiO+lTEMpXJyK45R/Kr3ZGJ2WtkP16D/rhewCjQ X-Gm-Message-State: AOJu0YwXW4sCmhWG5pg/N6gxyp2cvyX2PNj8nWIKDBFAFYxk9zRaJTvo oD8uBs6RlNVpvB62Wt6cqtfKYO4V/oV7vBa+bTQ92kF/cdSrzOzZ X-Google-Smtp-Source: AGHT+IExbl8LrLvJPyU9pKwjp5uruGM2OzUlT/MtzqGW1/+10hqCuUEFamttCWLYKNGTxD89KXVlNA== X-Received: by 2002:a17:902:ea03:b0:1f0:af34:6fa7 with SMTP id d9443c01a7336-1f0af3472d8mr66089495ad.10.1715957182587; Fri, 17 May 2024 07:46:22 -0700 (PDT) Received: from devant.hz.ali.com ([47.89.83.81]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ef0c160a1esm158504985ad.279.2024.05.17.07.46.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 07:46:22 -0700 (PDT) From: Xuewei Niu X-Google-Original-From: Xuewei Niu To: stefanha@redhat.com, sgarzare@redhat.com Cc: mst@redhat.com, davem@davemloft.net, kvm@vger.kernel.org, virtualization@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Xuewei Niu Subject: [RFC PATCH 1/5] vsock/virtio: Extend virtio-vsock spec with an "order" field Date: Fri, 17 May 2024 22:46:03 +0800 Message-Id: <20240517144607.2595798-2-niuxuewei.nxw@antgroup.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> References: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The "order" field determines the location of the device in the linked list, the device with CID 4, having a smallest order, is in the first place, and so forth. Rules: * It doesn’t have to be continuous; * It cannot exist conflicts; * It is optional for the mode of a single device, but is required for the mode of multiple devices. Signed-off-by: Xuewei Niu --- include/uapi/linux/virtio_vsock.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/uapi/linux/virtio_vsock.h b/include/uapi/linux/virtio_vsock.h index 64738838bee5..b62ec7d2ab1e 100644 --- a/include/uapi/linux/virtio_vsock.h +++ b/include/uapi/linux/virtio_vsock.h @@ -43,6 +43,7 @@ struct virtio_vsock_config { __le64 guest_cid; + __le64 order; } __attribute__((packed)); enum virtio_vsock_event_id { From patchwork Fri May 17 14:46:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuewei Niu X-Patchwork-Id: 13667056 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CCFA8479; Fri, 17 May 2024 14:46:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957188; cv=none; b=Gt2PLPpiAn0OnqQBgtJyS+NrTRotCUTraHIWIBSva7ATWDoRm1M+2iNq2WWfnT95bzQl2paeNvDOUqaP5WURPwzuA3esCQqu1X4VP/aiiipCNEIkkW9cjqUeE48kzvsYoxuYLKt6+269/8E4CzgfKul+a05KMFxWqiXfx9Mz4e0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957188; c=relaxed/simple; bh=iwIZEQ4Q5rFuaDAbh5Occ2g1dKr7xvxCd4LQ8ioI13g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iaZXm6zxHBDaj7KlHRc0U3rGLchKLYdLdIvNDyjOa9J8wRFn5mHTe60QK2OxS6v4exKZFKZibAnHG4ULbHDjj79vAm5IT76POzo3TqMKalD+fbb3U+FIOF0qTKX3hP8fCgSzH5LN7EugzZzf7rzU0gYsvAUeuiLACcVoLSTU21A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gbxQsQ8X; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gbxQsQ8X" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1eb0e08bfd2so11071885ad.1; Fri, 17 May 2024 07:46:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715957186; x=1716561986; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hhcPAkTMoXtpRsZDzlIHR80mutMmolukX3vTrgtCrMU=; b=gbxQsQ8XRqgAxXFBPcPccYrp+ryaEn0i0CpvWNF9PXIP0bz20Kl2B8TYhbNXUAi6a4 /ddI9fxhddd1ER526AJTRXq9dYTtZfpT4BlxJh9RHVm6gx1XmGymEJA7F6rFZUdIhsSy HI84WVK5L7KPpjDMFSzy2b8htOoOQMKaFex86oy8KtbAqTIvpDhWi7RlWTwjdxC/Bnw0 SBItBRmBLmT4Sqsacd9XEIoLoJ5xADaIriob4yA6xGfEyd/0QReaxeyOrbvOYJm2Sau6 CL1xZjNaFklhdWfkvzHdPP4w9GxZqyPOfyXjTyq9SXAw3YTeoGnrvrUkH7xLbpCfndGv oZGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715957186; x=1716561986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hhcPAkTMoXtpRsZDzlIHR80mutMmolukX3vTrgtCrMU=; b=pkzBjesgdaDVN4NEQAwrZzkWVyxxtF+YvLLcFWsNI+Is+vFB/D7wDjK4Fl2Whs4l9K +0ZKTX7V/PDwXsipJIsoUvIFQ3LleebQCPPk/abv7HUmP2h7zsHFKxKB43cyNu/nAkWk avpms89vJEO1/4Yr65d5tbBCeRyj4r9zllqGzGFJ8HAkVzF9rNHgkJK6x8Pt7eBISxIr nr41AfwaLnTTdHMTCz3pSjHgGUi8yZ+ZkoqQ97wokLRWY9F0P504ZJwJFG84+H9y41ES GvM3B9fEaGEbhJLZTwFLdcPvuMae0aqpvX8WH0ZBH2RbwTrhPJ8DMCDPP6C7NfAKXfRs lPPw== X-Forwarded-Encrypted: i=1; AJvYcCWQpqvlC5oi+nO4M6gYN22rmTY81iWYvHt2uk9eVmgt53KhlEQVMGnoq1sh4XkBBecyMEsNp0KmLjxZ3eVwIeflCksvSF6/QkGVnXJK37h45ooWTCR4OnCXq/BDb/m9XuGK0HIMuHXtXQWY7kVSkAfZC3ZaDlBQuUnU X-Gm-Message-State: AOJu0Ywq7kPsE8OJXg4qsUKpAVbsD32UXrqtxm77UKocfoO8ddfYpMbw ZwzrSRvXbO/On6wGLyZbuOi07/cyLe6IiF5918HrbvgawYpi5o4p X-Google-Smtp-Source: AGHT+IEsRXp1QxlbePsr0S9375oLAEJ6UIiJo9StEKrx45TszjRvZFUOBjLw7QDr6FnWZR6v2PokUg== X-Received: by 2002:a17:903:2444:b0:1e4:55d8:dfae with SMTP id d9443c01a7336-1ef43c0cec0mr270583525ad.4.1715957185505; Fri, 17 May 2024 07:46:25 -0700 (PDT) Received: from devant.hz.ali.com ([47.89.83.81]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ef0c160a1esm158504985ad.279.2024.05.17.07.46.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 07:46:25 -0700 (PDT) From: Xuewei Niu X-Google-Original-From: Xuewei Niu To: stefanha@redhat.com, sgarzare@redhat.com Cc: mst@redhat.com, davem@davemloft.net, kvm@vger.kernel.org, virtualization@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Xuewei Niu Subject: [RFC PATCH 2/5] vsock/virtio: Add support for multi-devices Date: Fri, 17 May 2024 22:46:04 +0800 Message-Id: <20240517144607.2595798-3-niuxuewei.nxw@antgroup.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> References: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The maximum number of devices is limited by `MAX_VSOCK_NUM`. Extends `vsock_transport` struct with 4 methods to support multi-devices: * `get_virtio_vsock()`: It receives a CID, and returns a struct of virtio vsock. This method is designed to select a vsock device by its CID. * `get_default_cid()`: It receives nothing, returns the default CID of the first vsock device registered to the kernel. * `get_local_cids()`: It returns a vector of vsock devices' CIDs. * `compare_order()`: It receives two different CIDs, named "left" and "right" respectively. It returns "-1" while the "left" is behind the "right". Otherwise, return "1". `get_local_cid()` is retained, but returns "-1" if the transport supports multi-devices. Replaces the single instance of `virtio_vsock` with a list, named `virtio_vsock_list`. The devices are inserted into the list when probing. The kernel will deny devices from being registered if there are conflicts existing in CIDs or orders. Signed-off-by: Xuewei Niu --- include/net/af_vsock.h | 16 ++ include/uapi/linux/vm_sockets.h | 6 + net/vmw_vsock/af_vsock.c | 82 ++++++-- net/vmw_vsock/virtio_transport.c | 246 ++++++++++++++++++------ net/vmw_vsock/virtio_transport_common.c | 10 +- 5 files changed, 293 insertions(+), 67 deletions(-) diff --git a/include/net/af_vsock.h b/include/net/af_vsock.h index 535701efc1e5..0151296a0bc5 100644 --- a/include/net/af_vsock.h +++ b/include/net/af_vsock.h @@ -174,6 +174,22 @@ struct vsock_transport { /* Addressing. */ u32 (*get_local_cid)(void); + /* Held rcu read lock by the caller. */ + struct virtio_vsock *(*get_virtio_vsock)(unsigned int cid); + unsigned int (*get_default_cid)(void); + /* Get an list containing all the CIDs of registered vsock. Return + * the length of the list. + * + * Held rcu read lock by the caller. + */ + int (*get_local_cids)(unsigned int *local_cids); + /* Compare the order of two devices. Given the guest CIDs of two + * different devices, returns -1 while the left is behind the right. + * Otherwise, return 1. + * + * Held rcu read lock by the caller. + */ + int (*compare_order)(unsigned int left, unsigned int right); /* Read a single skb */ int (*read_skb)(struct vsock_sock *, skb_read_actor_t); diff --git a/include/uapi/linux/vm_sockets.h b/include/uapi/linux/vm_sockets.h index ed07181d4eff..36ca5023293a 100644 --- a/include/uapi/linux/vm_sockets.h +++ b/include/uapi/linux/vm_sockets.h @@ -189,6 +189,12 @@ struct sockaddr_vm { sizeof(__u8)]; }; +/* The maximum number of vsock devices. Each vsock device has an exclusive + * context id. + */ + +#define MAX_VSOCK_NUM 16 + #define IOCTL_VM_SOCKETS_GET_LOCAL_CID _IO(7, 0xb9) /* MSG_ZEROCOPY notifications are encoded in the standard error format, diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index 54ba7316f808..da06ddc940cd 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -234,19 +234,45 @@ static void __vsock_remove_connected(struct vsock_sock *vsk) static struct sock *__vsock_find_bound_socket(struct sockaddr_vm *addr) { - struct vsock_sock *vsk; + struct vsock_sock *vsk, *any_vsk = NULL; + rcu_read_lock(); list_for_each_entry(vsk, vsock_bound_sockets(addr), bound_table) { + /* The highest priority: full match. */ if (vsock_addr_equals_addr(addr, &vsk->local_addr)) - return sk_vsock(vsk); + goto out; - if (addr->svm_port == vsk->local_addr.svm_port && - (vsk->local_addr.svm_cid == VMADDR_CID_ANY || - addr->svm_cid == VMADDR_CID_ANY)) - return sk_vsock(vsk); + /* Port match */ + if (addr->svm_port == vsk->local_addr.svm_port) { + /* The second priority: local cid is VMADDR_CID_ANY. */ + if (vsk->local_addr.svm_cid == VMADDR_CID_ANY) + goto out; + + /* The third priority: local cid isn't VMADDR_CID_ANY. */ + if (addr->svm_cid == VMADDR_CID_ANY) { + if (!any_vsk) { + any_vsk = vsk; + continue; + } + // Use the device with smaller order + if (vsk->transport->compare_order(any_vsk->local_addr.svm_cid, + vsk->local_addr.svm_cid) < 0) + any_vsk = vsk; + } + } + } + rcu_read_unlock(); + + if (any_vsk) { + pr_debug("matched a any_vsk at %p\n", any_vsk); + return sk_vsock(any_vsk); } return NULL; + +out: + rcu_read_unlock(); + return sk_vsock(vsk); } static struct sock *__vsock_find_connected_socket(struct sockaddr_vm *src, @@ -408,7 +434,11 @@ static bool vsock_use_local_transport(unsigned int remote_cid) return true; if (transport_g2h) { - return remote_cid == transport_g2h->get_local_cid(); + if (transport_g2h->get_virtio_vsock) + return transport_g2h->get_virtio_vsock(remote_cid) != + NULL; + else + return remote_cid == transport_g2h->get_local_cid(); } else { return remote_cid == VMADDR_CID_HOST; } @@ -516,9 +546,26 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) } EXPORT_SYMBOL_GPL(vsock_assign_transport); +bool transport_g2h_verify_cid(unsigned int cid) +{ + /* transports that support multi devices */ + rcu_read_lock(); + if (transport_g2h->get_virtio_vsock && + (cid == VMADDR_CID_ANY || transport_g2h->get_virtio_vsock(cid))) { + rcu_read_unlock(); + return true; + } + rcu_read_unlock(); + /* other transports */ + if (cid == transport_g2h->get_local_cid()) + return true; + + return false; +} + bool vsock_find_cid(unsigned int cid) { - if (transport_g2h && cid == transport_g2h->get_local_cid()) + if (transport_g2h && transport_g2h_verify_cid(cid)) return true; if (transport_h2g && cid == VMADDR_CID_HOST) @@ -697,7 +744,9 @@ static int __vsock_bind(struct sock *sk, struct sockaddr_vm *addr) /* Now bind to the provided address or select appropriate values if * none are provided (VMADDR_CID_ANY and VMADDR_PORT_ANY). Note that * like AF_INET prevents binding to a non-local IP address (in most - * cases), we only allow binding to a local CID. + * cases), we only allow binding to a local CID. In the cases of + * multi-devices, only CIDs of vsock devices registered in the kernel + * are allowed. */ if (addr->svm_cid != VMADDR_CID_ANY && !vsock_find_cid(addr->svm_cid)) return -EADDRNOTAVAIL; @@ -825,7 +874,6 @@ static void __vsock_release(struct sock *sk, int level) __vsock_release(pending, SINGLE_DEPTH_NESTING); sock_put(pending); } - release_sock(sk); sock_put(sk); } @@ -1181,7 +1229,12 @@ static int vsock_dgram_sendmsg(struct socket *sock, struct msghdr *msg, */ if (remote_addr->svm_cid == VMADDR_CID_ANY) - remote_addr->svm_cid = transport->get_local_cid(); + if (transport->get_default_cid) + remote_addr->svm_cid = + transport->get_default_cid(); + else + remote_addr->svm_cid = + transport->get_local_cid(); if (!vsock_addr_bound(remote_addr)) { err = -EINVAL; @@ -1191,7 +1244,12 @@ static int vsock_dgram_sendmsg(struct socket *sock, struct msghdr *msg, remote_addr = &vsk->remote_addr; if (remote_addr->svm_cid == VMADDR_CID_ANY) - remote_addr->svm_cid = transport->get_local_cid(); + if (transport->get_default_cid) + remote_addr->svm_cid = + transport->get_default_cid(); + else + remote_addr->svm_cid = + transport->get_local_cid(); /* XXX Should connect() or this function ensure remote_addr is * bound? diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index ee5d306a96d0..93d25aeafb83 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -22,8 +22,8 @@ #include static struct workqueue_struct *virtio_vsock_workqueue; -static struct virtio_vsock __rcu *the_virtio_vsock; -static DEFINE_MUTEX(the_virtio_vsock_mutex); /* protects the_virtio_vsock */ +static LIST_HEAD(virtio_vsock_list); /* vsock multi-devices */ +static DEFINE_MUTEX(virtio_vsock_list_mutex); /* protects virtio_vsock_list */ static struct virtio_transport virtio_transport; /* forward declaration */ struct virtio_vsock { @@ -62,6 +62,7 @@ struct virtio_vsock { struct virtio_vsock_event event_list[8]; u32 guest_cid; + u32 order; bool seqpacket_allow; /* These fields are used only in tx path in function @@ -74,24 +75,70 @@ struct virtio_vsock { */ struct scatterlist *out_sgs[MAX_SKB_FRAGS + 1]; struct scatterlist out_bufs[MAX_SKB_FRAGS + 1]; + + struct list_head node; + struct rcu_head rcu; }; static u32 virtio_transport_get_local_cid(void) +{ + return VMADDR_CID_ANY; +} + +/* Held rcu read lock by the caller. */ +static struct virtio_vsock *virtio_transport_get_virtio_vsock(unsigned int cid) { struct virtio_vsock *vsock; - u32 ret; + list_for_each_entry(vsock, &virtio_vsock_list, node) { + if (vsock->guest_cid == cid) + return vsock; + } + return NULL; +} - rcu_read_lock(); - vsock = rcu_dereference(the_virtio_vsock); - if (!vsock) { - ret = VMADDR_CID_ANY; - goto out_rcu; +static unsigned int virtio_transport_get_default_cid(void) +{ + struct virtio_vsock *vsock; + + vsock = list_first_or_null_rcu(&virtio_vsock_list, struct virtio_vsock, + node); + if (!vsock) + return VMADDR_CID_ANY; + + return vsock->guest_cid; +} + +/* Held rcu read lock by the caller. */ +static int virtio_transport_get_local_cids(u32 *cids) +{ + int count = 0; + struct virtio_vsock *vsock; + + if (!cids) + return -EFAULT; + + list_for_each_entry(vsock, &virtio_vsock_list, node) { + cids[count++] = vsock->guest_cid; } + return count; +} - ret = vsock->guest_cid; -out_rcu: - rcu_read_unlock(); - return ret; +/* Held rcu read lock by the caller. */ +static int virtio_transport_compare_order(unsigned int left, unsigned int right) +{ + struct virtio_vsock *vsock; + + if (left == right) + return 0; + + list_for_each_entry(vsock, &virtio_vsock_list, node) { + if (right == vsock->guest_cid) + return -1; + if (left == vsock->guest_cid) + return 1; + } + + return 0; } static void @@ -201,12 +248,16 @@ virtio_transport_send_pkt(struct sk_buff *skb) struct virtio_vsock_hdr *hdr; struct virtio_vsock *vsock; int len = skb->len; + unsigned int src_cid; hdr = virtio_vsock_hdr(skb); + src_cid = le64_to_cpu(hdr->src_cid); rcu_read_lock(); - vsock = rcu_dereference(the_virtio_vsock); + vsock = virtio_transport_get_virtio_vsock(src_cid); if (!vsock) { + pr_debug("pkt sending has been failed, as vsock with cid %u not found\n", + src_cid); kfree_skb(skb); len = -ENODEV; goto out_rcu; @@ -236,13 +287,17 @@ virtio_transport_cancel_pkt(struct vsock_sock *vsk) int cnt = 0, ret; rcu_read_lock(); - vsock = rcu_dereference(the_virtio_vsock); - if (!vsock) { + if (list_empty(&virtio_vsock_list)) { ret = -ENODEV; goto out_rcu; } - cnt = virtio_transport_purge_skbs(vsk, &vsock->send_pkt_queue); + list_for_each_entry(vsock, &virtio_vsock_list, node) { + cnt = virtio_transport_purge_skbs(vsk, &vsock->send_pkt_queue); + /* Skbs for the vsk belong to one virtio_vsock */ + if (cnt) + break; + } if (cnt) { struct virtqueue *rx_vq = vsock->vqs[VSOCK_VQ_RX]; @@ -385,6 +440,16 @@ static void virtio_vsock_update_guest_cid(struct virtio_vsock *vsock) vsock->guest_cid = le64_to_cpu(guest_cid); } +static void virtio_vsock_update_order(struct virtio_vsock *vsock) +{ + struct virtio_device *vdev = vsock->vdev; + __le64 order; + + vdev->config->get(vdev, offsetof(struct virtio_vsock_config, order), + &order, sizeof(order)); + vsock->order = le64_to_cpu(order); +} + /* event_lock must be held */ static void virtio_vsock_event_handle(struct virtio_vsock *vsock, struct virtio_vsock_event *event) @@ -492,13 +557,17 @@ static bool virtio_transport_msgzerocopy_allow(void) return true; } -static bool virtio_transport_seqpacket_allow(u32 remote_cid); +static bool virtio_transport_seqpacket_allow(u32 src_cid, u32 remote_cid); static struct virtio_transport virtio_transport = { .transport = { .module = THIS_MODULE, .get_local_cid = virtio_transport_get_local_cid, + .get_virtio_vsock = virtio_transport_get_virtio_vsock, + .get_default_cid = virtio_transport_get_default_cid, + .get_local_cids = virtio_transport_get_local_cids, + .compare_order = virtio_transport_compare_order, .init = virtio_transport_do_socket_init, .destruct = virtio_transport_destruct, @@ -617,7 +686,38 @@ static void virtio_transport_rx_work(struct work_struct *work) static int virtio_vsock_vqs_init(struct virtio_vsock *vsock) { struct virtio_device *vdev = vsock->vdev; - static const char * const names[] = { + struct virtio_vsock *_vsock = NULL; + int i; + unsigned int guest_cid, order; + + virtio_vsock_update_guest_cid(vsock); + virtio_vsock_update_order(vsock); + guest_cid = vsock->guest_cid; + order = vsock->order; + + i = 0; + list_for_each_entry(_vsock, &virtio_vsock_list, node) { + if (++i > MAX_VSOCK_NUM) { + pr_debug("vsock num reaches limit %d\n", MAX_VSOCK_NUM); + return -ENOMEM; + } + + /* Check for guest_cid conflicts */ + if (unlikely(guest_cid == _vsock->guest_cid)) { + pr_debug("conflict exists in vsock's guest_cid %u\n", + guest_cid); + return -EBUSY; + } + + /* Check for order conflicts */ + if (unlikely(order == _vsock->order)) { + pr_debug("conflict exists in vsock's order %u\n", + order); + return -EBUSY; + } + } + + static const char *const names[] = { "rx", "tx", "event", @@ -634,8 +734,6 @@ static int virtio_vsock_vqs_init(struct virtio_vsock *vsock) if (ret < 0) return ret; - virtio_vsock_update_guest_cid(vsock); - virtio_device_ready(vdev); return 0; @@ -716,21 +814,14 @@ static void virtio_vsock_vqs_del(struct virtio_vsock *vsock) static int virtio_vsock_probe(struct virtio_device *vdev) { - struct virtio_vsock *vsock = NULL; - int ret; - int i; + struct virtio_vsock *vsock, *_vsock, *first_vsock; + int ret, i; + unsigned int guest_cid, order; - ret = mutex_lock_interruptible(&the_virtio_vsock_mutex); + ret = mutex_lock_interruptible(&virtio_vsock_list_mutex); if (ret) return ret; - /* Only one virtio-vsock device per guest is supported */ - if (rcu_dereference_protected(the_virtio_vsock, - lockdep_is_held(&the_virtio_vsock_mutex))) { - ret = -EBUSY; - goto out; - } - vsock = kzalloc(sizeof(*vsock), GFP_KERNEL); if (!vsock) { ret = -ENOMEM; @@ -764,28 +855,56 @@ static int virtio_vsock_probe(struct virtio_device *vdev) for (i = 0; i < ARRAY_SIZE(vsock->out_sgs); i++) vsock->out_sgs[i] = &vsock->out_bufs[i]; - rcu_assign_pointer(the_virtio_vsock, vsock); + order = vsock->order; + guest_cid = vsock->guest_cid; + first_vsock = + list_first_entry(&virtio_vsock_list, struct virtio_vsock, node); + /* Insert virtio-vsock device into a proper location. */ + if (list_empty(&virtio_vsock_list) || first_vsock->order > order) { + list_add_rcu(&vsock->node, &virtio_vsock_list); + } else { + list_for_each_entry(_vsock, &virtio_vsock_list, node) { + struct virtio_vsock *next = container_of(_vsock->node.next, + struct virtio_vsock, node); + if (&next->node != &virtio_vsock_list && + next->order < order) + continue; + list_add_rcu(&vsock->node, &_vsock->node); + break; + } + } + + pr_debug("virtio_vsock@%p registered (cid = %u, order = %u)\n", vsock, guest_cid, order); + virtio_vsock_vqs_start(vsock); - mutex_unlock(&the_virtio_vsock_mutex); + mutex_unlock(&virtio_vsock_list_mutex); return 0; out: kfree(vsock); - mutex_unlock(&the_virtio_vsock_mutex); + mutex_unlock(&virtio_vsock_list_mutex); return ret; } static void virtio_vsock_remove(struct virtio_device *vdev) { - struct virtio_vsock *vsock = vdev->priv; + struct virtio_vsock *vsock, *_vsock; - mutex_lock(&the_virtio_vsock_mutex); + vsock = vdev->priv; + + mutex_lock(&virtio_vsock_list_mutex); vdev->priv = NULL; - rcu_assign_pointer(the_virtio_vsock, NULL); - synchronize_rcu(); + /* Remove virtio-vsock device from the list. */ + list_for_each_entry(_vsock, &virtio_vsock_list, node) { + if (vsock == _vsock) { + list_del_rcu(&vsock->node); + synchronize_rcu(); + break; + } + } virtio_vsock_vqs_del(vsock); @@ -797,7 +916,7 @@ static void virtio_vsock_remove(struct virtio_device *vdev) flush_work(&vsock->event_work); flush_work(&vsock->send_pkt_work); - mutex_unlock(&the_virtio_vsock_mutex); + mutex_unlock(&virtio_vsock_list_mutex); kfree(vsock); } @@ -805,43 +924,62 @@ static void virtio_vsock_remove(struct virtio_device *vdev) #ifdef CONFIG_PM_SLEEP static int virtio_vsock_freeze(struct virtio_device *vdev) { - struct virtio_vsock *vsock = vdev->priv; + struct virtio_vsock *vsock, *_vsock; - mutex_lock(&the_virtio_vsock_mutex); + vsock = vdev->priv; - rcu_assign_pointer(the_virtio_vsock, NULL); - synchronize_rcu(); + mutex_lock(&virtio_vsock_list_mutex); + + list_for_each_entry(_vsock, &virtio_vsock_list, node) { + if (vsock == _vsock) { + list_del_rcu(&vsock->node); + synchronize_rcu(); + break; + } + } virtio_vsock_vqs_del(vsock); - mutex_unlock(&the_virtio_vsock_mutex); + mutex_unlock(&virtio_vsock_list_mutex); return 0; } static int virtio_vsock_restore(struct virtio_device *vdev) { - struct virtio_vsock *vsock = vdev->priv; + struct virtio_vsock *vsock, *_vsock, *first_vsock; int ret; + unsigned int order; - mutex_lock(&the_virtio_vsock_mutex); + vsock = vdev->priv; - /* Only one virtio-vsock device per guest is supported */ - if (rcu_dereference_protected(the_virtio_vsock, - lockdep_is_held(&the_virtio_vsock_mutex))) { - ret = -EBUSY; - goto out; - } + mutex_lock(&virtio_vsock_list_mutex); ret = virtio_vsock_vqs_init(vsock); if (ret < 0) goto out; - rcu_assign_pointer(the_virtio_vsock, vsock); + order = vsock->order; + first_vsock = + list_first_entry(&virtio_vsock_list, struct virtio_vsock, node); + /* Insert virtio-vsock device into a proper location. */ + if (list_empty(&virtio_vsock_list) || first_vsock->order > order) { + list_add_rcu(&vsock->node, &virtio_vsock_list); + } else { + list_for_each_entry(_vsock, &virtio_vsock_list, node) { + struct virtio_vsock *next = container_of(_vsock->node.next, + struct virtio_vsock, node); + if (&next->node != &virtio_vsock_list && + next->order < order) + continue; + list_add_rcu(&vsock->node, &_vsock->node); + break; + } + } virtio_vsock_vqs_start(vsock); out: - mutex_unlock(&the_virtio_vsock_mutex); + mutex_unlock(&virtio_vsock_list_mutex); return ret; } #endif /* CONFIG_PM_SLEEP */ diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 16ff976a86e3..bed75a41419e 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -340,7 +340,15 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, if (unlikely(!t_ops)) return -EFAULT; - src_cid = t_ops->transport.get_local_cid(); + if (vsk->local_addr.svm_cid == VMADDR_CID_ANY) { + if (t_ops->transport.get_default_cid) + src_cid = t_ops->transport.get_default_cid(); + else + src_cid = t_ops->transport.get_local_cid(); + } else { + src_cid = vsk->local_addr.svm_cid; + } + src_port = vsk->local_addr.svm_port; if (!info->remote_cid) { dst_cid = vsk->remote_addr.svm_cid; From patchwork Fri May 17 14:46:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuewei Niu X-Patchwork-Id: 13667057 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EE4F6CDAB; Fri, 17 May 2024 14:46:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957190; cv=none; b=T9EiLJotJPS/tGzBm1eAuD/dJnQUJKwQmCa8Hq3zYbNnQaWmLiFVrgv7IzqIH7GhwJ/g+zgNFL7v2b86Vh1m0+9kxW+JKIKyGEJpk1SNXSoH3EnlC/vjw7u/x0728Fbq0ARtJThTqwN221h9bAtBbPsZou1wEFBfDOVCdJcQ2bU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957190; c=relaxed/simple; bh=ayTww+DFRdv8kU8QHxSwm1TVYpqV1rJoYpxyAD0nsTA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=A4S8MEXyobPlBbeFmDXrCXbp+iYT1WuSQVgW7mv6aDU2+Xu2jN2JUOXfdeuGXDj03TwDGElQWZl0j4g9nNF/O2Rh6JPU/nvsU8Oce3K5V+1+RV2L+I3M5PaYpc9OYG0QKFqg6LnygJWs+geYUP15V5PhUaKVW0lhFTjLHYFaW/I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZOR7UksO; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZOR7UksO" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1ee38966529so4908705ad.1; Fri, 17 May 2024 07:46:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715957188; x=1716561988; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ObdkphHypw+pCyvAkBy4EywArmTATwnandulAo4H4xI=; b=ZOR7UksOqiDVrq7ySQeiOQlGcgnFXIqujtENiBpgATOF3mLcncUokt+SK5LFyWc+dX uWT588vIdG2YZ8uYUCeYzu7CTg++2krSR2s8hkDibwsFjCngqASMuHWFaHuip5Alpgh/ gJjDxFXU8JU/mqQ/FJsmmZrRJMuby8Svk3LR/EWx+jV6yFjcvmYE8k8/Z0GLddu5WB+v l82lnTYttAaFjRLKpo5rZvF8+2bHeSXkJX/FUxgD3uVDnztfp9O8WzNKjVpWjCcX35wB hFyqKPZ0WviBN35B2wSLmiu0MUWS/B0Y78BkvBS1TwCIKSmTSNrTENTARBTXMTnifRXx 5RFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715957188; x=1716561988; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ObdkphHypw+pCyvAkBy4EywArmTATwnandulAo4H4xI=; b=SN8v1/QdoH9QRayswRMa8IJ/EFR2GvAOPRHBq7CcPCBQ5z1uB5lSJlXasN18lhc0cD KHg30tzAzXPIczsjAKkWTfpt9BkKX6Sufl/7Cdi4RT623AnlA+XsWxqT/pU/lEC+SlgN pbWBdssDdYNe71NJdblxaCojTC/UcdY6Dt0OnwZ9dtyeIplD76mCr6Ihc9uYMgWKHj2P C66lgsWuJxiR1xDDro+tIOpkZ15TIJYIXnU/6TKw8zsv20uTGCZTB1F3h9UV+Ec8YvpX XfvncXSxSsLd9QWRtIc0eMsn27uHBOJ2rJ1sqLuw8h6d2O8DQP6QBtr8wzxz3cx8sHaH TIgQ== X-Forwarded-Encrypted: i=1; AJvYcCUVXErjMer/I8Auznu2bRj4oQH1lurTHaOu7u6uA8I6eoCCwlV6t5IB9a4MryQiDq3vo6IuKsCWxXfSvWDtu2iFpBsHgO81yVsrlC+2S53vF5JPjkHOqEBL4XaL45onFfZVTk3VfGAOIpwHwwM5e5rv+WT2xpoTf0nH X-Gm-Message-State: AOJu0YwK4HI6cOxD4cR8eSbQjVbPLKKSibwp7sCztJc1F3+pKthZXXJE WQ4di+UWZUuewm4iMdYuCP6SLRc0Eo68Sm2tdpjMFmohH9Q9wFhG X-Google-Smtp-Source: AGHT+IGznRigy/pv1m+3lTVs4TUKCUcENj3vk2rDy123ij37qxwzekLbGPS3kJ0q5z1BkgDnP4tNkA== X-Received: by 2002:a17:903:18a:b0:1eb:7855:43d5 with SMTP id d9443c01a7336-1ef432a0bf9mr313781075ad.30.1715957188387; Fri, 17 May 2024 07:46:28 -0700 (PDT) Received: from devant.hz.ali.com ([47.89.83.81]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ef0c160a1esm158504985ad.279.2024.05.17.07.46.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 07:46:27 -0700 (PDT) From: Xuewei Niu X-Google-Original-From: Xuewei Niu To: stefanha@redhat.com, sgarzare@redhat.com Cc: mst@redhat.com, davem@davemloft.net, kvm@vger.kernel.org, virtualization@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Xuewei Niu Subject: [RFC PATCH 3/5] vsock/virtio: can_msgzerocopy adapts to multi-devices Date: Fri, 17 May 2024 22:46:05 +0800 Message-Id: <20240517144607.2595798-4-niuxuewei.nxw@antgroup.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> References: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Adds a new argument, named "cid", to let them know which `virtio_vsock` to be selected. Signed-off-by: Xuewei Niu --- include/linux/virtio_vsock.h | 2 +- net/vmw_vsock/virtio_transport.c | 5 ++--- net/vmw_vsock/virtio_transport_common.c | 6 +++--- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h index c82089dee0c8..21bfd5e0c2e7 100644 --- a/include/linux/virtio_vsock.h +++ b/include/linux/virtio_vsock.h @@ -168,7 +168,7 @@ struct virtio_transport { * extra checks and can perform zerocopy transmission by * default. */ - bool (*can_msgzerocopy)(int bufs_num); + bool (*can_msgzerocopy)(u32 cid, int bufs_num); }; ssize_t diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index 93d25aeafb83..998b22e5ce36 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -521,14 +521,13 @@ static void virtio_vsock_rx_done(struct virtqueue *vq) queue_work(virtio_vsock_workqueue, &vsock->rx_work); } -static bool virtio_transport_can_msgzerocopy(int bufs_num) +static bool virtio_transport_can_msgzerocopy(u32 cid, int bufs_num) { struct virtio_vsock *vsock; bool res = false; rcu_read_lock(); - - vsock = rcu_dereference(the_virtio_vsock); + vsock = virtio_transport_get_virtio_vsock(cid); if (vsock) { struct virtqueue *vq = vsock->vqs[VSOCK_VQ_TX]; diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index bed75a41419e..e7315d7b9af1 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -39,7 +39,7 @@ virtio_transport_get_ops(struct vsock_sock *vsk) static bool virtio_transport_can_zcopy(const struct virtio_transport *t_ops, struct virtio_vsock_pkt_info *info, - size_t pkt_len) + size_t pkt_len, unsigned int cid) { struct iov_iter *iov_iter; @@ -62,7 +62,7 @@ static bool virtio_transport_can_zcopy(const struct virtio_transport *t_ops, int pages_to_send = iov_iter_npages(iov_iter, MAX_SKB_FRAGS); /* +1 is for packet header. */ - return t_ops->can_msgzerocopy(pages_to_send + 1); + return t_ops->can_msgzerocopy(cid, pages_to_send + 1); } return true; @@ -375,7 +375,7 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, info->msg->msg_flags &= ~MSG_ZEROCOPY; if (info->msg->msg_flags & MSG_ZEROCOPY) - can_zcopy = virtio_transport_can_zcopy(t_ops, info, pkt_len); + can_zcopy = virtio_transport_can_zcopy(t_ops, info, pkt_len, src_cid); if (can_zcopy) max_skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, From patchwork Fri May 17 14:46:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuewei Niu X-Patchwork-Id: 13667058 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BA8C6E613; Fri, 17 May 2024 14:46:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957193; cv=none; b=rSEtJG94B4L17lfmCu665CVYkk6f0GmbrwRaBotyLM4LrbhchGYeQZPyUKlRZR3Q0TPLoYuI8PtDidUKQz6bmdwnEKzEm48eIhwelzpFgPI2RQln68aKUEPn6f+pGfKQMvqIa34EN+Y64lLqnb9f5NJnMOCK1KgntE1MMI34ha0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957193; c=relaxed/simple; bh=q3sDsBUcUDme/MtiN7St5f8cv1EJMwhSScadewV7zew=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mVxLpbrmwawFD16ZL+wIrzpsU0uU23h4yhu2jCsPIiVLN8fnQd+YM7NRzqfVc02xnZ1Rsej/fAe4tLS79qmo5cm4G7AaONKgFDJvbeF1pN3tOtRxOyANWO4UQIy1YoqPDbwSKNsRyMzoKmQ5kJ3kFB/7LhK14cqSVrGL2Eh8Ljc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QEQQYZWC; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QEQQYZWC" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1e651a9f3ffso10221635ad.1; Fri, 17 May 2024 07:46:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715957191; x=1716561991; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hJq2b9RGdc8kjbmcDqovAimStl9IOIGQIeg0f2YK8Ws=; b=QEQQYZWChPlrWoUlTcI2bWFY4hFLou1WlOtVtR5LDM0WO5NBkYDa2CnUtztjtqTsP9 ny09gOGm7VoAebnSM89IaOnBLQYym5O5msTWKB9LXfkminhl1UxRpmni8HngQF+9zH8D +WWuBNbLlpzfmzHiza1M3ol0N5pyBAkVgIuVSWxM9P6DWcWOMszj+b142JsbhYTDOzTd 3lb1KH2zaaMktZjhJVCVVP8YOrucqzfly/X5JhISbcQpbc1TiPLvnVCdlDxExA/ZgPOm QlXBsd1jOtfXAkcVq8DUnm+ykoZb+EPsny1mjwBCVpbSbf2PDTXLIioeWVUaR3jT1aPL F6rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715957191; x=1716561991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hJq2b9RGdc8kjbmcDqovAimStl9IOIGQIeg0f2YK8Ws=; b=aAfkwaUDldwstwTEjSozbU6/XsApRoyJxCd7CKgMbhLfffjTagGKCsgW9TGUxb0p7o zfuRRKOP8svu29GbyfXmgCE/PTYl3LrnNBoN4EKKewH4orL+ByLQh15XmBsV+lauqcCH b1GuF0/CQvRO4uI3fOrb/f+cYlf+qrxWW7DGrE7CNzNi4WmUD9rhONsFIUWCtivztFYg vhLmgMISKOaUGJ20jfSQJ2qskDP2Vgrdq+KD435vtWE1reYpN6HywOwno49vKpZ+uOJz Tq4Il1KatQ5o9g9B1lblfnNVG2GQdzdeuJT50ysdYSO9OyY/LXSihAge545FhcXBhQZa tDTA== X-Forwarded-Encrypted: i=1; AJvYcCXzBH5jIW/gcbIGiaLglKAq/7GSx6+THTd4bWwEFSbKFFDO0BuDBsqS1mEtc79ZfiaMGVFzkiE4a+UVfEsFYrczGd8m7j6p3Ir45a815170w64/kQW31l/6cK8UIhpjYTuJaJRODAX/Sc+xCbL/QPpE3xHApglzTL+D X-Gm-Message-State: AOJu0YwEMHAakpk+7GLAmQXchIYT5x91TgAMLO1/j2rMeVhzngZGY2cd AN8UIFSgrMCzn7As8cl4/KOr+e40wmYJ1P1KTxfrgdltdUgUsJDb X-Google-Smtp-Source: AGHT+IHL2uRGyT1nIYq0kpvkRlAo7ap2G40vKTgf52EWRitQ6LcfBBTk+NlOH/B34nRbz+e6iAEM4g== X-Received: by 2002:a17:902:d506:b0:1e4:24cc:e021 with SMTP id d9443c01a7336-1ef44050595mr282515655ad.50.1715957191409; Fri, 17 May 2024 07:46:31 -0700 (PDT) Received: from devant.hz.ali.com ([47.89.83.81]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ef0c160a1esm158504985ad.279.2024.05.17.07.46.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 07:46:30 -0700 (PDT) From: Xuewei Niu X-Google-Original-From: Xuewei Niu To: stefanha@redhat.com, sgarzare@redhat.com Cc: mst@redhat.com, davem@davemloft.net, kvm@vger.kernel.org, virtualization@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Xuewei Niu Subject: [RFC PATCH 4/5] vsock: seqpacket_allow adapts to multi-devices Date: Fri, 17 May 2024 22:46:06 +0800 Message-Id: <20240517144607.2595798-5-niuxuewei.nxw@antgroup.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> References: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Adds a new argument, named "src_cid", to let them know which `virtio_vsock` to be selected. Signed-off-by: Xuewei Niu --- include/net/af_vsock.h | 2 +- net/vmw_vsock/af_vsock.c | 15 +++++++++++++-- net/vmw_vsock/virtio_transport.c | 4 ++-- net/vmw_vsock/vsock_loopback.c | 4 ++-- 4 files changed, 18 insertions(+), 7 deletions(-) diff --git a/include/net/af_vsock.h b/include/net/af_vsock.h index 0151296a0bc5..25f7dc3d602d 100644 --- a/include/net/af_vsock.h +++ b/include/net/af_vsock.h @@ -143,7 +143,7 @@ struct vsock_transport { int flags); int (*seqpacket_enqueue)(struct vsock_sock *vsk, struct msghdr *msg, size_t len); - bool (*seqpacket_allow)(u32 remote_cid); + bool (*seqpacket_allow)(u32 src_cid, u32 remote_cid); u32 (*seqpacket_has_data)(struct vsock_sock *vsk); /* Notification. */ diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index da06ddc940cd..3b34be802bf2 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -470,10 +470,12 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) { const struct vsock_transport *new_transport; struct sock *sk = sk_vsock(vsk); - unsigned int remote_cid = vsk->remote_addr.svm_cid; + unsigned int src_cid, remote_cid; __u8 remote_flags; int ret; + remote_cid = vsk->remote_addr.svm_cid; + /* If the packet is coming with the source and destination CIDs higher * than VMADDR_CID_HOST, then a vsock channel where all the packets are * forwarded to the host should be established. Then the host will @@ -527,8 +529,17 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) return -ENODEV; if (sk->sk_type == SOCK_SEQPACKET) { + if (vsk->local_addr.svm_cid == VMADDR_CID_ANY) { + if (new_transport->get_default_cid) + src_cid = new_transport->get_default_cid(); + else + src_cid = new_transport->get_local_cid(); + } else { + src_cid = vsk->local_addr.svm_cid; + } + if (!new_transport->seqpacket_allow || - !new_transport->seqpacket_allow(remote_cid)) { + !new_transport->seqpacket_allow(src_cid, remote_cid)) { module_put(new_transport->module); return -ESOCKTNOSUPPORT; } diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index 998b22e5ce36..0bddcbd906a2 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -615,14 +615,14 @@ static struct virtio_transport virtio_transport = { .can_msgzerocopy = virtio_transport_can_msgzerocopy, }; -static bool virtio_transport_seqpacket_allow(u32 remote_cid) +static bool virtio_transport_seqpacket_allow(u32 src_cid, u32 remote_cid) { struct virtio_vsock *vsock; bool seqpacket_allow; seqpacket_allow = false; rcu_read_lock(); - vsock = rcu_dereference(the_virtio_vsock); + vsock = virtio_transport_get_virtio_vsock(src_cid); if (vsock) seqpacket_allow = vsock->seqpacket_allow; rcu_read_unlock(); diff --git a/net/vmw_vsock/vsock_loopback.c b/net/vmw_vsock/vsock_loopback.c index 6dea6119f5b2..b94358f5bb2c 100644 --- a/net/vmw_vsock/vsock_loopback.c +++ b/net/vmw_vsock/vsock_loopback.c @@ -46,7 +46,7 @@ static int vsock_loopback_cancel_pkt(struct vsock_sock *vsk) return 0; } -static bool vsock_loopback_seqpacket_allow(u32 remote_cid); +static bool vsock_loopback_seqpacket_allow(u32 src_cid, u32 remote_cid); static bool vsock_loopback_msgzerocopy_allow(void) { return true; @@ -104,7 +104,7 @@ static struct virtio_transport loopback_transport = { .send_pkt = vsock_loopback_send_pkt, }; -static bool vsock_loopback_seqpacket_allow(u32 remote_cid) +static bool vsock_loopback_seqpacket_allow(u32 src_cid, u32 remote_cid) { return true; } From patchwork Fri May 17 14:46:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xuewei Niu X-Patchwork-Id: 13667059 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB05971757; Fri, 17 May 2024 14:46:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957196; cv=none; b=oU/EF1EfBsZNuvMpDat4g7S1NjGEUeOvDDNbxIQ8OwSDFzQWRSIP7Efmiudab9YzAb+Zcwh8Zm/6EhvLPwwPoaPP6AD1DNGk2Tu7UvJqjBKU5InZA4ilZ2Y3vrWNNUEdjc80BeGjR+yO8yfIdgzKzBsMKHh+bZGBcQX0wmB0Nj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715957196; c=relaxed/simple; bh=WnxVwjgYGJtcoPZ87tCxtUxpJqNACgXajD6ezrDCG4I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Eh3ia+mZJJTMNG3+hHLTKvpUQbdfPyuufT5X0vyZSrdtkQjEuGa01fAG55onTh7SmBzRk+1zoWzrt6Za8+HO+w/uoUXGSpx8Yg59A9EQWSkUgBJQNpmvj/XQHgtvcKIklmda87vKGsI3dFPV10eRUrvyqtap3ZMx2YIw4dcEjjg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ixvJ17XS; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ixvJ17XS" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1eb0e08bfd2so11077705ad.1; Fri, 17 May 2024 07:46:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715957194; x=1716561994; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1E5gdZlICe3lq/gDwWvHR3o2K94KvkxWORr8jEuKK78=; b=ixvJ17XSuqQ4El0fKozGFftyw1b7Us6lI6k8Diupf+u7bUCx+/f5FBGwgMvgoRVqCK e0itaRRP0i6NZHZG6vsR6ERILlUJw+viW9FTlJtFVgSVlR3L6EBwJ4ZwI/xcsOg6r1f5 uZZScjg8UA9/AZBftVl+vE1GlQ00feJ2oBv0DwW3Fj6gvKgq553iVFZ6Pa9bQl7IpmBL 1oinH+cNgYADno6g0WHJ5ChpT0+55e5iczPoU8FgnIzgBwZWiShKFoHIWtwjf7L0KqeL WII3+2Vn6nS6WrIB+vHl4mo9jgVuCK8nYiQVaboNmuMoRa3jE0oXHTYEXrBpFz0yAdqv IDgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715957194; x=1716561994; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1E5gdZlICe3lq/gDwWvHR3o2K94KvkxWORr8jEuKK78=; b=bi6plwDmXjOfb9Ln8IQ0k6K5JMvK2/GwhtAgbCJ2+6Ctgqd7+ujEYbADe6PiwAb9AQ a656Zr025mOsS0l0HuMrgaoDOiFCERtsZZkGRL8v/+GQg2cIcAyaap120go0cKy4snzb 2GIlfKiul+2wTYUJqHiru4dF7uakOv+DxnibqV3ktFcJtgnyarqCp8hN2J6lZGABwjoY SgLfQJmKnc+WNoRJkK2gjwVYKZfUeA8iMc55YprPC94B8P331ixEh/3GNWyLKuHacMvo qbegto55MzRHaG8F3PecNuxM4qMfoptGClo09oTytPY4f+eEjBBee50ywiVBTpzdILBa qWyw== X-Forwarded-Encrypted: i=1; AJvYcCVWlGEET3N/PVtFvgbZkf419Rx5jPekR9UZfypb2acyK7vPQP9FArPcKNYWnacMvBKg72NuzP9pQMu3wvRRodmmTR3H6kt24vv0vZXM6Rvqvx7obHBX+2zQeJY7KdJWcMJpRHRGVkYCMYowSg2s4+ue9it3pfMGQfWS X-Gm-Message-State: AOJu0YwYnnCt/AtyxXgQ6TYn2cwLHS2/vsdUodCexQKlR5Aq23TL9qBo 9+GDGKE3OX4Mp49Fi3P/6ZovNL3iU0RkTP57DqNk1f62VLM9tVL5lMd+Jc9zuZo29szv X-Google-Smtp-Source: AGHT+IGlrB2ABZOfxviKuEq2qAGBP9HT0HFRenQskQ3kOdrAUJQmjATvt0uOz+tRWRqVGbrjhLcsCA== X-Received: by 2002:a17:902:e889:b0:1f0:9938:d260 with SMTP id d9443c01a7336-1f09938d438mr74524725ad.50.1715957194441; Fri, 17 May 2024 07:46:34 -0700 (PDT) Received: from devant.hz.ali.com ([47.89.83.81]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1ef0c160a1esm158504985ad.279.2024.05.17.07.46.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 07:46:34 -0700 (PDT) From: Xuewei Niu X-Google-Original-From: Xuewei Niu To: stefanha@redhat.com, sgarzare@redhat.com Cc: mst@redhat.com, davem@davemloft.net, kvm@vger.kernel.org, virtualization@lists.linux.dev, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Xuewei Niu Subject: [RFC PATCH 5/5] vsock: Add an ioctl request to get all CIDs Date: Fri, 17 May 2024 22:46:07 +0800 Message-Id: <20240517144607.2595798-6-niuxuewei.nxw@antgroup.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> References: <20240517144607.2595798-1-niuxuewei.nxw@antgroup.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The new request is called `IOCTL_VM_SOCKETS_GET_LOCAL_CIDS`. And the old one, `IOCTL_VM_SOCKETS_GET_LOCAL_CID` is retained. For the transport that supports multi-devices: * `IOCTL_VM_SOCKETS_GET_LOCAL_CID` returns "-1"; * `IOCTL_VM_SOCKETS_GET_LOCAL_CIDS` returns a vector of CIDS. The usage is shown as following. ``` struct vsock_local_cids local_cids; if ((ret = ioctl(fd, IOCTL_VM_SOCKETS_GET_LOCAL_CIDS, &local_cids))) { perror("failed to get cids"); exit(1); } for (i = 0; i --- include/net/af_vsock.h | 7 +++++++ include/uapi/linux/vm_sockets.h | 8 ++++++++ net/vmw_vsock/af_vsock.c | 19 +++++++++++++++++++ 3 files changed, 34 insertions(+) diff --git a/include/net/af_vsock.h b/include/net/af_vsock.h index 25f7dc3d602d..2febc816e388 100644 --- a/include/net/af_vsock.h +++ b/include/net/af_vsock.h @@ -264,4 +264,11 @@ static inline bool vsock_msgzerocopy_allow(const struct vsock_transport *t) { return t->msgzerocopy_allow && t->msgzerocopy_allow(); } + +/**** IOCTL ****/ +/* Type of return value of IOCTL_VM_SOCKETS_GET_LOCAL_CIDS. */ +struct vsock_local_cids { + int nr; + unsigned int data[MAX_VSOCK_NUM]; +}; #endif /* __AF_VSOCK_H__ */ diff --git a/include/uapi/linux/vm_sockets.h b/include/uapi/linux/vm_sockets.h index 36ca5023293a..01f73fb7af5a 100644 --- a/include/uapi/linux/vm_sockets.h +++ b/include/uapi/linux/vm_sockets.h @@ -195,8 +195,16 @@ struct sockaddr_vm { #define MAX_VSOCK_NUM 16 +/* Return actual context id if the transport not support vsock + * multi-devices. Otherwise, return `-1U`. + */ + #define IOCTL_VM_SOCKETS_GET_LOCAL_CID _IO(7, 0xb9) +/* Only available in transports that support multiple devices. */ + +#define IOCTL_VM_SOCKETS_GET_LOCAL_CIDS _IOR(7, 0xba, struct vsock_local_cids) + /* MSG_ZEROCOPY notifications are encoded in the standard error format, * sock_extended_err. See Documentation/networking/msg_zerocopy.rst in * kernel source tree for more details. diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index 3b34be802bf2..2ea2ff52f15b 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -2454,6 +2454,7 @@ static long vsock_dev_do_ioctl(struct file *filp, u32 __user *p = ptr; u32 cid = VMADDR_CID_ANY; int retval = 0; + struct vsock_local_cids local_cids; switch (cmd) { case IOCTL_VM_SOCKETS_GET_LOCAL_CID: @@ -2469,6 +2470,24 @@ static long vsock_dev_do_ioctl(struct file *filp, retval = -EFAULT; break; + case IOCTL_VM_SOCKETS_GET_LOCAL_CIDS: + if (!transport_g2h || !transport_g2h->get_local_cids) + goto fault; + + rcu_read_lock(); + local_cids.nr = transport_g2h->get_local_cids(local_cids.data); + rcu_read_unlock(); + + if (local_cids.nr < 0 || + copy_to_user(p, &local_cids, sizeof(local_cids))) + goto fault; + + break; + +fault: + retval = -EFAULT; + break; + default: retval = -ENOIOCTLCMD; }