From patchwork Wed Feb 19 14:48:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Levon X-Patchwork-Id: 13982418 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E18EAC021B1 for ; Wed, 19 Feb 2025 14:55:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tklOx-0003Sh-GE; Wed, 19 Feb 2025 09:50:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tklOu-0003PS-VG; Wed, 19 Feb 2025 09:50:12 -0500 Received: from mx0a-002c1b01.pphosted.com ([148.163.151.68]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tklOs-0007V3-2G; Wed, 19 Feb 2025 09:50:12 -0500 Received: from pps.filterd (m0127839.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51J9LnWr011251; Wed, 19 Feb 2025 06:50:07 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint20171006; bh=CbYCxq57zE19zKX5mUO4iVNHgd4VV/+dCMgQHeUY/ Js=; b=vJpQ5MxpHNRXB5yRinqVGlSGdtEN6urn7dYMklgF4x/ub/OThW87osTZ/ CSbds7aAaT3RpcXSQ+Ihrrxpym0gv8/0pWYr5RU3p+PSSeo2YNEAzO6uz+eksX2O 7QQPxP+Zn418UTPYg+ovBGLLyaSBlGjO0bwLqTO3yvTyn/nFxfnKg0I4/El4qNiB 0/nydOc9x8KBl/oSofGQhVFXaCe2wmTreG15vM0deax8NQBGwHaTpUXe88ULt2/6 v7s6eYfVn7cgu5hyK2MtmUXlRiDEUPWxWdw/7XDq63wI9HSOaMM3Lb1r2q4S9vVa SOzSmfjA/ZKKM2NdB4btl09R2VZnQ== Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2044.outbound.protection.outlook.com [104.47.58.44]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 44w4bdhxxr-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 19 Feb 2025 06:50:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FrmeCOoEU0qiDdomy9mluVjMDjlIbFcBMjLeQBNlimRVifGazUIzI6tatzfyC4e+C1rc2us7iBVca5/EZnH+NGUUqxl+m0buNGw3ayo8K6dGkHhX9g+k4/HWOSYxDWfUFmucwQgFfCtVk7axOdnnnE3qCelynQqXLDY6A8Nr3Fzhd4bAGCWXYGBXhlCJ95fVeJz0qElAG1OAiSsCd9oX1wRaCwhrONPQG3FPtV5ARH2EpBNu6wWRI3xcmZKNlItlGPCn6MoymM80KcHHeFnywv07Yfvfk7gzGj3TIlGHKYA+ic8g4exvkshKW2bpn29KhBTSxYD1XChyjNS7oA3Ihw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CbYCxq57zE19zKX5mUO4iVNHgd4VV/+dCMgQHeUY/Js=; b=EDee0+WGUDOoutrjxQ/0lDniQPzwEnKmGl3Nfz8Ptb71ZLBH+W65HDtXXEhev2djkhb9MfHv4JwWTgLzF6M1quY+yuV3mxIXd4crjXF09i7pSXaAjwsRC9zYd8w6NEbyFK8NlkwxM71rdTruSZWSz8NaiwXbki3/zGzqw9X2EvRqqrXGdAl/PhihpOohy899H0bKz5UuSbCj7hQJjXvdMQ2q5g2p7W6uef5j1G8NPpzbDHr3pNUE6vxoORnJ5RfZKaGFvwrioM3cDAYPTJparzIZuDnKd9h/iCx3K7+f6SXwEXkCOa72fYzvignTFUxJqaXM6MprrP/y4Ke6Zn4Bfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CbYCxq57zE19zKX5mUO4iVNHgd4VV/+dCMgQHeUY/Js=; b=cFDMuwhFeKzMVW5pRdF3EiPwrfW4OUJoLZmoH/UW7Bi6XO6K2t9CLKPxP76GGzAphvKnHz/CrHJEJJoEnzIKqtniezmv+QsRZ8ftNoX9f4yj5qCz99hzMTjSlh/eU9U1KE8GvfuGwjFZ6fd20raPzd/a/n6hoE4dD7qeo0qkOWXcFIHC+BahvXssZxcyaaObjkv4llgSbGKOd5yAKbZEJh3UPcYSpL5SdQy3tI39bkP/PfBfGH1le7N1TyktqfWjTlKaureesTQFCLmLID3cAy6uRYsJM35RmcG3wF8b/htCFW9ZFj4F4OasvXldWIl1obAC+SHszEM9M0sMQBK9EQ== Received: from CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) by LV8PR02MB10096.namprd02.prod.outlook.com (2603:10b6:408:181::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.14; Wed, 19 Feb 2025 14:49:54 +0000 Received: from CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51]) by CH2PR02MB6760.namprd02.prod.outlook.com ([fe80::fd77:ea65:a159:ef51%7]) with mapi id 15.20.8466.013; Wed, 19 Feb 2025 14:49:54 +0000 From: John Levon To: qemu-devel@nongnu.org Cc: Jason Herne , Thanos Makatos , Halil Pasic , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eric Farman , Tony Krowiak , Thomas Huth , qemu-s390x@nongnu.org, Matthew Rosato , John Levon , Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Stefano Garzarella , Alex Williamson , David Hildenbrand , =?utf-8?q?C=C3=A9dric_Le_Goater?= , Peter Xu , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , "Michael S. Tsirkin" , Jagannathan Raman , John Johnson , Elena Ufimtseva Subject: [PATCH v8 13/28] vfio-user: connect vfio proxy to remote server Date: Wed, 19 Feb 2025 15:48:43 +0100 Message-Id: <20250219144858.266455-14-john.levon@nutanix.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250219144858.266455-1-john.levon@nutanix.com> References: <20250219144858.266455-1-john.levon@nutanix.com> X-ClientProxiedBy: AM0PR02CA0154.eurprd02.prod.outlook.com (2603:10a6:20b:28d::21) To CH2PR02MB6760.namprd02.prod.outlook.com (2603:10b6:610:7f::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR02MB6760:EE_|LV8PR02MB10096:EE_ X-MS-Office365-Filtering-Correlation-Id: 85357136-f640-4288-7f6e-08dd50f4ac06 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|1800799024|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?q?6UHVrIQViVOCFj8x3COMERzugfd0ZwV?= =?utf-8?q?iWhyd3JA/gy1kQcL2PfeER/q2zXfi0+V5+LWxKs7AaYOcMzOdZf95sSlCnQgGlm6i?= =?utf-8?q?a9pBYHhdIFeV176t8scpynEnWijDzAzgNNnV/djGTUwf3WmbqA/2BiXBUc6yQwJyM?= =?utf-8?q?S0tO4lmYTY7zk4eTlf83EkSF+DthEaWDTNlMbzA2/JfBFzYsJUHfyl17Z/j9eJ32/?= =?utf-8?q?dsg+Ry32N5c2pFkLmpPlD/rpw3FR1uhmk/fTK8GbLn9Xt4r5/5MHmohYfcN1W1Ak4?= =?utf-8?q?OqTaTUQ+SMgar4O2nvkL8bHAPxR4ZaHedB3ukoLMFIHMr1GFH4wkdYuC3d0+m8esC?= =?utf-8?q?nIYfn0tyG3IukFQz1vIb4epJYbqR85YoiMskgh1kxvaYV2emGWQC2v2dYztIYu+zh?= =?utf-8?q?hboW9h8n6rVl9596IKegk3jfcqAzHocpMGtIxZI5aio3reaztKmwPi9fzOP+kLOcM?= =?utf-8?q?+Bm8R4TVwii6dJ9H0CIONWhlIZlyZVVFxjJ6G6o2Q2sylFMcNGAYQr6PuHnBJCmXP?= =?utf-8?q?u6hqsDHcwosGdm8Xv2oeV7RYnfkWa7sv50QOFnp0Z0I1/nJuBbhG/fOLoEYv6+ZZN?= =?utf-8?q?Y4vL1ZfN3B9remwTNYEAeKGgOo+hvtIn767M9nFacZL4mftj/eMZAEbAguOg20Qp9?= =?utf-8?q?OjN/sffPex6A9dcKVAsPg8LS/BTtKIEfHULJZMtqgsD7K8VBfTvWMfxmm//90eK8y?= =?utf-8?q?UaLrIhnNET372XcOi9Z6KaAEyDr5FLyPhcc2zjyNgET1L9q++pvvGDddzdNFf6BE5?= =?utf-8?q?QvZq4cUel8t5mJZE7UR8qTwSP126yuLvhp4cWeTTAu02tfo2Tet5d/YD7lFQNYl5Q?= =?utf-8?q?meVj8dN20p+10Upm7jC0RM8cXQQOgVWjCT0Aitpn492efXha286Vi9aN6DJNnoHP7?= =?utf-8?q?Qxv1BXk0FUY2j/XzpvOSBwUJTHH+/o+SmBo08XR51pF0QwhCPTYRo/H5ThxSc3Qv0?= =?utf-8?q?NSySjWqSxyC4A0P3TGYVYUC/lvr9+3+nItCW5vy9d5xMtD0cEQbVchA6/xTmta/aO?= =?utf-8?q?usbe/9QgLV4gUCvCoMO6LKwIjyYX92jiNEehaUt7jajJI7Yy8AVcjyK0SBK2HDHYg?= =?utf-8?q?XTWrSDCgXFgc62mEYDRlXr08tCoUGjF+i4sAMJrfIs1phyDlwYwi/9XkT87iLkMbC?= =?utf-8?q?LIe0CdnwsogK4q+Gyqd2WyfcSN58nYgIx86tGE1j2uIG8+s9znaK4XGqGe3x9X80y?= =?utf-8?q?gbnl97+TQzJPuJnuG+m0c2CvuSPTQBYLxl0+yQv3DUNVF3JL0UlcNMy3e+Hu4ve+2?= =?utf-8?q?htJ0ysEpTtTijc7EoUApFOmOpPwB6Ci0KsW4kett6XU8+pCcfTkjs3Xv3QZVPaUOX?= =?utf-8?q?8qw5IEaN+zgu?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR02MB6760.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(7416014)(1800799024)(376014); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?DDkNPdEfPnHi5Z8fbIKWxm7wHWal?= =?utf-8?q?V/jWt0FtuXSA+hkus1hH13UNAlJLlke/KyY0q2UT5xn57U8NGDkS3M0qbUvTUCQun?= =?utf-8?q?YvBTPRs5FRaDWhFAwEq6GZwQyEc6GVwcMngQi61ywfAeNX7BdX5nWSs+/wgiaBaGH?= =?utf-8?q?DaoMcG6DSb1pmGa+t/LcCLACzhdpPyHP5ZzQLl6bYpPGKlDeO1EY+VtFiZ57fLmMz?= =?utf-8?q?VKQ99Ta0sqXZfecY81TEn29NTnZv3twHZoXhGRD0JwrNnxZcSSG7CVnnFLFQea0dR?= =?utf-8?q?wKzmbfqaYFz2woEyNJ2xBb2K003s2HuomFUiS8GW/Fz2e3GjjvoLVe9mBebmNJ0zz?= =?utf-8?q?2GAVG2pbhRDSvpe3JV4vmOT4OFchea/9pDga0PxInRU2dGv15NDSjdybZzq53KtUc?= =?utf-8?q?R4VupXtVmtOWqN+x9RS90Jcmus9R3p1O85ZhiSMpMeNpj3xu9dCSgyLWbg7TQFGyt?= =?utf-8?q?3pLmuyg6uVLj4oa2MExkWGgU6BiRCRxE/nU5rCdQmQEZ+DJowrVRnrdel2V1vOp65?= =?utf-8?q?cTsQ951TADfUNBuOYWR37iIFi2DioWuuPZGkSDe3N/RiJNA6BoRi6bAglrmJWIil5?= =?utf-8?q?Qk4k7VxtHkZCpGsGEi3W1k2Ew9nUdzGFbSB7cvO62ERhS1gwjwAd71VcvQSf8iBt5?= =?utf-8?q?wHA7nbK4a7EAH+pu9EzBHeXq9XYPixBQQe2jQIzfEcPZOk7QUni6yvLDxms/U0l1H?= =?utf-8?q?WvfbAE63AMcZBoVZ0IqLEkax+OxhxqavDkILolYYHpqcfKzAeq2pVA709YEiKFD7f?= =?utf-8?q?P2bDuNptI8NiI4vJuOs0d6cx5IdTDZ/8YQCzQH9GwBdUyyIuEJ7D5NBzb/015cM4s?= =?utf-8?q?NOJ5TCVvYzpmEVLNL2u0DGVK6x97ZyreXYnOxibYDDt7p3qY2U8Z37nqZ/hVZ5K8W?= =?utf-8?q?DyNLZuXWcLERTBJhjKS8+L+Sh5imhvkIJpxK/cthQR3XfR5SSnpm5wnIhVa5M2BaV?= =?utf-8?q?i7gYsGceVD9RuU9NR/gTLmMaffuIrZQw/y8ofXksJJFJwOuEFXlu/NqCaYoqnLZ/w?= =?utf-8?q?wp40kXJ+d/yPxiYyRBD4cJb+FUogzLIl/Ct/d57NTe8sGFMS1u3s29CDZ9v2rinMc?= =?utf-8?q?6VbNFhEcp6DILbJf33+x6W73jYHV7ZbEZ4K/8B1MwbcNzRh14tTyKdnmfZay7EFD5?= =?utf-8?q?tPfLfKie2t4kGjJFV56NHwtZHidh4Ko0Uv8ZmGpekR9LKIvgweD/TQ4JD3itJCMPH?= =?utf-8?q?y3ryBGpX5pY1aY8lxFfdLotyspa5v9fn+nQLWWZ48tf8WdkSmAqLZCxxmL3Fuo3KY?= =?utf-8?q?Zr2yQI6COHvZ8ivwHJ8KReMaqPsfuDwFgcIwSkNjo3dNU/33t1Y7WVWclJtfYf3/o?= =?utf-8?q?NCCG1S4UVxX5uxZyJrlbBvCL0Vh5npFzC6AeDJkkqciB031aEdh6Ua6l7fASwdEbd?= =?utf-8?q?z/9Y5skd4DG0VA1MkObdTDARAghPf0yFMghWVBNeVMiMfW4KIIbYKTyOln2oTjJrk?= =?utf-8?q?WH6tGslTCueeU124MWcvhKB4f9zuK7sCS+/W4r1XYVbU/z0MwXZhDumT+TjC3aPBb?= =?utf-8?q?Z2oSYjSfMSaU?= X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 85357136-f640-4288-7f6e-08dd50f4ac06 X-MS-Exchange-CrossTenant-AuthSource: CH2PR02MB6760.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2025 14:49:54.6659 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: l7HVAsugf9UkMmXKgC2JYcIkBKAyy5384pVMWqtI8nvH1u7pbz5WhM0TLQnQlLczfwQqgnshr7rNX6hPXPmHTQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR02MB10096 X-Proofpoint-ORIG-GUID: Eyxtzb7HSPh2SmtgFrvH8s-oR1ALwBCL X-Proofpoint-GUID: Eyxtzb7HSPh2SmtgFrvH8s-oR1ALwBCL X-Authority-Analysis: v=2.4 cv=Fo7//3rq c=1 sm=1 tr=0 ts=67b5ef9e cx=c_pps a=di3315gfm3qlniCp1Rh91A==:117 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=T2h4t0Lz3GQA:10 a=0034W8JfsZAA:10 a=0kUYKlekyDsA:10 a=yPCof4ZbAAAA:8 a=64Cc0HZtAAAA:8 a=vhHEnE-nI2wpHcbFAcgA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=14NRyaPF5x3gF6G45PvQ:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-19_06,2025-02-19_01,2024-11-22_01 X-Proofpoint-Spam-Reason: safe Received-SPF: pass client-ip=148.163.151.68; envelope-from=john.levon@nutanix.com; helo=mx0a-002c1b01.pphosted.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.191, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Jagannathan Raman Introduce the vfio-user "proxy": this is the client code responsible for sending and receiving vfio-user messages across the control socket. The new files hw/vfio-user/common.[ch] contain some basic plumbing for managing the proxy; initialize the proxy during realization of the VFIOUserPCIDevice instance. Originally-by: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John Levon --- hw/vfio-user/common.c | 171 ++++++++++++++++++++++++++++++++++ hw/vfio-user/common.h | 78 ++++++++++++++++ hw/vfio-user/meson.build | 1 + hw/vfio-user/pci.c | 18 ++++ include/hw/vfio/vfio-common.h | 2 + 5 files changed, 270 insertions(+) create mode 100644 hw/vfio-user/common.c create mode 100644 hw/vfio-user/common.h diff --git a/hw/vfio-user/common.c b/hw/vfio-user/common.c new file mode 100644 index 0000000000..e829abccec --- /dev/null +++ b/hw/vfio-user/common.c @@ -0,0 +1,171 @@ +/* + * vfio protocol over a UNIX socket. + * + * Copyright © 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include +#include + +#include "hw/hw.h" +#include "hw/vfio/vfio-common.h" +#include "io/channel.h" +#include "io/channel-socket.h" +#include "io/channel-util.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "qemu/lockable.h" +#include "qemu/sockets.h" +#include "system/iothread.h" + +#include "common.h" + +static IOThread *vfio_user_iothread; + +static void vfio_user_shutdown(VFIOUserProxy *proxy); + + +/* + * Functions called by main, CPU, or iothread threads + */ + +static void vfio_user_shutdown(VFIOUserProxy *proxy) +{ + qio_channel_shutdown(proxy->ioc, QIO_CHANNEL_SHUTDOWN_READ, NULL); + qio_channel_set_aio_fd_handler(proxy->ioc, proxy->ctx, NULL, + proxy->ctx, NULL, NULL); +} + +/* + * Functions only called by iothread + */ + +static void vfio_user_cb(void *opaque) +{ + VFIOUserProxy *proxy = opaque; + + QEMU_LOCK_GUARD(&proxy->lock); + + proxy->state = VFIO_PROXY_CLOSED; + qemu_cond_signal(&proxy->close_cv); +} + + +/* + * Functions called by main or CPU threads + */ + +static QLIST_HEAD(, VFIOUserProxy) vfio_user_sockets = + QLIST_HEAD_INITIALIZER(vfio_user_sockets); + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp) +{ + VFIOUserProxy *proxy; + QIOChannelSocket *sioc; + QIOChannel *ioc; + char *sockname; + + if (addr->type != SOCKET_ADDRESS_TYPE_UNIX) { + error_setg(errp, "vfio_user_connect - bad address family"); + return NULL; + } + sockname = addr->u.q_unix.path; + + sioc = qio_channel_socket_new(); + ioc = QIO_CHANNEL(sioc); + if (qio_channel_socket_connect_sync(sioc, addr, errp)) { + object_unref(OBJECT(ioc)); + return NULL; + } + qio_channel_set_blocking(ioc, false, NULL); + + proxy = g_malloc0(sizeof(VFIOUserProxy)); + proxy->sockname = g_strdup_printf("unix:%s", sockname); + proxy->ioc = ioc; + proxy->flags = VFIO_PROXY_CLIENT; + proxy->state = VFIO_PROXY_CONNECTED; + + qemu_mutex_init(&proxy->lock); + qemu_cond_init(&proxy->close_cv); + + if (vfio_user_iothread == NULL) { + vfio_user_iothread = iothread_create("VFIO user", errp); + } + + proxy->ctx = iothread_get_aio_context(vfio_user_iothread); + + QTAILQ_INIT(&proxy->outgoing); + QTAILQ_INIT(&proxy->incoming); + QTAILQ_INIT(&proxy->free); + QTAILQ_INIT(&proxy->pending); + QLIST_INSERT_HEAD(&vfio_user_sockets, proxy, next); + + return proxy; +} + +void vfio_user_disconnect(VFIOUserProxy *proxy) +{ + VFIOUserMsg *r1, *r2; + + qemu_mutex_lock(&proxy->lock); + + /* our side is quitting */ + if (proxy->state == VFIO_PROXY_CONNECTED) { + vfio_user_shutdown(proxy); + if (!QTAILQ_EMPTY(&proxy->pending)) { + error_printf("vfio_user_disconnect: outstanding requests\n"); + } + } + object_unref(OBJECT(proxy->ioc)); + proxy->ioc = NULL; + + proxy->state = VFIO_PROXY_CLOSING; + QTAILQ_FOREACH_SAFE(r1, &proxy->outgoing, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->outgoing, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->incoming, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->incoming, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->pending, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->pending, r1, next); + g_free(r1); + } + QTAILQ_FOREACH_SAFE(r1, &proxy->free, next, r2) { + qemu_cond_destroy(&r1->cv); + QTAILQ_REMOVE(&proxy->free, r1, next); + g_free(r1); + } + + /* + * Make sure the iothread isn't blocking anywhere + * with a ref to this proxy by waiting for a BH + * handler to run after the proxy fd handlers were + * deleted above. + */ + aio_bh_schedule_oneshot(proxy->ctx, vfio_user_cb, proxy); + qemu_cond_wait(&proxy->close_cv, &proxy->lock); + + /* we now hold the only ref to proxy */ + qemu_mutex_unlock(&proxy->lock); + qemu_cond_destroy(&proxy->close_cv); + qemu_mutex_destroy(&proxy->lock); + + QLIST_REMOVE(proxy, next); + if (QLIST_EMPTY(&vfio_user_sockets)) { + iothread_destroy(vfio_user_iothread); + vfio_user_iothread = NULL; + } + + g_free(proxy->sockname); + g_free(proxy); +} diff --git a/hw/vfio-user/common.h b/hw/vfio-user/common.h new file mode 100644 index 0000000000..3529f03df9 --- /dev/null +++ b/hw/vfio-user/common.h @@ -0,0 +1,78 @@ +#ifndef VFIO_USER_COMMON_H +#define VFIO_USER_COMMON_H + +/* + * vfio protocol over a UNIX socket. + * + * Copyright © 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + */ + +typedef struct { + int send_fds; + int recv_fds; + int *fds; +} VFIOUserFDs; + +enum msg_type { + VFIO_MSG_NONE, + VFIO_MSG_ASYNC, + VFIO_MSG_WAIT, + VFIO_MSG_NOWAIT, + VFIO_MSG_REQ, +}; + +typedef struct VFIOUserMsg { + QTAILQ_ENTRY(VFIOUserMsg) next; + VFIOUserFDs *fds; + uint32_t rsize; + uint32_t id; + QemuCond cv; + bool complete; + enum msg_type type; +} VFIOUserMsg; + + +enum proxy_state { + VFIO_PROXY_CONNECTED = 1, + VFIO_PROXY_ERROR = 2, + VFIO_PROXY_CLOSING = 3, + VFIO_PROXY_CLOSED = 4, +}; + +typedef QTAILQ_HEAD(VFIOUserMsgQ, VFIOUserMsg) VFIOUserMsgQ; + +typedef struct VFIOUserProxy { + QLIST_ENTRY(VFIOUserProxy) next; + char *sockname; + struct QIOChannel *ioc; + void (*request)(void *opaque, VFIOUserMsg *msg); + void *req_arg; + int flags; + QemuCond close_cv; + AioContext *ctx; + QEMUBH *req_bh; + + /* + * above only changed when BQL is held + * below are protected by per-proxy lock + */ + QemuMutex lock; + VFIOUserMsgQ free; + VFIOUserMsgQ pending; + VFIOUserMsgQ incoming; + VFIOUserMsgQ outgoing; + VFIOUserMsg *last_nowait; + enum proxy_state state; +} VFIOUserProxy; + +/* VFIOProxy flags */ +#define VFIO_PROXY_CLIENT 0x1 + +VFIOUserProxy *vfio_user_connect_dev(SocketAddress *addr, Error **errp); +void vfio_user_disconnect(VFIOUserProxy *proxy); + +#endif /* VFIO_USER_COMMON_H */ diff --git a/hw/vfio-user/meson.build b/hw/vfio-user/meson.build index f1fee70c85..b89985c641 100644 --- a/hw/vfio-user/meson.build +++ b/hw/vfio-user/meson.build @@ -1,5 +1,6 @@ vfio_user_ss = ss.source_set() vfio_user_ss.add(files( + 'common.c', 'container.c', 'pci.c', )) diff --git a/hw/vfio-user/pci.c b/hw/vfio-user/pci.c index 6c85c60179..bfd9051524 100644 --- a/hw/vfio-user/pci.c +++ b/hw/vfio-user/pci.c @@ -19,6 +19,8 @@ #include "hw/qdev-properties.h" #include "hw/qdev-properties-system.h" #include "hw/vfio/pci.h" +#include "hw/vfio-user/common.h" +#include "hw/vfio-user/container.h" #include "qapi/error.h" #include "qobject/qdict.h" #include "qemu/error-report.h" @@ -59,6 +61,8 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) VFIOPCIDevice *vdev = VFIO_PCI_BASE(pdev); VFIODevice *vbasedev = &vdev->vbasedev; AddressSpace *as; + SocketAddress addr; + VFIOUserProxy *proxy; /* * TODO: make option parser understand SocketAddress @@ -71,6 +75,15 @@ static void vfio_user_pci_realize(PCIDevice *pdev, Error **errp) return; } + memset(&addr, 0, sizeof(addr)); + addr.type = SOCKET_ADDRESS_TYPE_UNIX; + addr.u.q_unix.path = udev->sock_name; + proxy = vfio_user_connect_dev(&addr, errp); + if (!proxy) { + return; + } + vbasedev->proxy = proxy; + vbasedev->name = g_strdup_printf("VFIO user <%s>", udev->sock_name); vbasedev->ops = &vfio_user_pci_ops; vbasedev->type = VFIO_DEVICE_TYPE_PCI; @@ -119,8 +132,13 @@ static void vfio_user_instance_init(Object *obj) static void vfio_user_instance_finalize(Object *obj) { VFIOPCIDevice *vdev = VFIO_PCI_BASE(obj); + VFIODevice *vbasedev = &vdev->vbasedev; vfio_pci_put_device(vdev); + + if (vbasedev->proxy != NULL) { + vfio_user_disconnect(vbasedev->proxy); + } } static const Property vfio_user_pci_dev_properties[] = { diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index ee8e7f7c0d..699e8a9376 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -79,6 +79,7 @@ typedef struct VFIOMigration { } VFIOMigration; struct VFIOGroup; +typedef struct VFIOUserProxy VFIOUserProxy; /* MMU container sub-class for legacy vfio implementation. */ typedef struct VFIOContainer { @@ -155,6 +156,7 @@ typedef struct VFIODevice { IOMMUFDBackend *iommufd; VFIOIOASHwpt *hwpt; QLIST_ENTRY(VFIODevice) hwpt_next; + VFIOUserProxy *proxy; struct vfio_region_info **regions; } VFIODevice;