From patchwork Wed Dec 15 15:35:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1B98C433EF for ; Wed, 15 Dec 2021 16:14:03 +0000 (UTC) Received: from localhost ([::1]:52914 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWQb-0005tv-Cj for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:42:49 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWK1-0005EO-60 for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:03 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:24020) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWJx-0008RT-1y for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:00 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEYwBW008125; Wed, 15 Dec 2021 15:35:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=kR7vPzM1V8PihMFmjBOTnuFSROPHpR+JbmdbUlqkwEQ=; b=soTNCbAP1LmFBz5Kv2pTmhDH/QehGOS0EsJfE+So7yTJsa0n2wMcL7xRdQmtcJTXLyb/ qpivWwgGYwwMq9W74hzqWhi0C3QnRbVjZ+9INX5FMPrw24KttlGZo8IGiVp1PoKPDE4m uSLBo+fmxIMb1/RBznVdTBK/J21DXiH6eIc2OjB9yDqfbyzwabf/lma/Fl07RW8Y+qSN R85LpIDQ4fBJugWVVYqRvmZzxEQtT1n1QJpBcZdgWH8nBihnk5iZSTPdOkJxtJDmmHsq V3bIOVvYbgXUQzLsGiXLlLpxUcTCCP8rHNMCIJxrngBP7c8lHtT/9d+n8/1WGmpVR+pL 7g== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx2nff720-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:35:53 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFVi0G127819; Wed, 15 Dec 2021 15:35:52 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49]) by userp3020.oracle.com with ESMTP id 3cvnes2c6r-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:35:52 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CERaBYGuU1dhALy07pjcS32rzUOB44E0QNmkCElKJyCN/3OBPNl5u8kbLpycK93indlmWQ+tc8hvknoR/fHofycU/IckRHqtc0HwkEdRA6HxYi4kHV5tBLtgnUeVMx7bzrgTwRbbssdNx71PM0BQ+2oV2K4GDyl949Q9UetdpF7dOCk8RqBTxeKxuKXGBoCN3Qh6hfmSfjWzGHbSAY/YpBbyEomPixL7INsnOf/Y1C/lUPbyc4rhHbQiE3AiLhMQ4ts++fmp913C/5CKPIIX0uthq+K8/vrf17iHUEmNw+lMaOIb6C0tLw061sdj40wt0MnfIUftuEWfXY19p4gwjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kR7vPzM1V8PihMFmjBOTnuFSROPHpR+JbmdbUlqkwEQ=; b=Jl+j24dF+dQV/vMgBCYKXq1coSpEHizvaefis9DQSmJRvdEZyU+Q7Tqf/tIPQLUDwtrJRGevSIuyzN5XAjD39VAA9onbH7CuknOv29joz/yUKOn39mPqBL6D1DVw2B3ncf7puDO/rV1dzGt/3GjUWRY+eapohtZ13fhKbMOKQnsqn2K8fiIQ1NCc1VOWlK5y6ysCPsRxgSaWme+ju8/QkkBSn6Rr6Sne3ftuRoe48VJIuqMlgdiVduQyEZSyfNXhQEhkG3ddOK7F+dtHZdKnP4I5VQuTrMwQBUME8uVrYlxyggg3MV1L8rhNPvvRaqpgPfB7dCDCMHEfVUuQ0EtuZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kR7vPzM1V8PihMFmjBOTnuFSROPHpR+JbmdbUlqkwEQ=; b=Lw6Pftes7MdvwuWWrWlL5Wm/1NOJKyG9vkF42CslYPgVyzTfyH21ZrsbvPIpFPpF/7oipu3foJXd1hkEC4n3lE9zwkFKRMb+mpFJFSow5WRNEP0Zk1zwq153adX3orX8Om3sTwLLKOSKVFZRw3CQfE7Q1yCGmbpyWYJT0HADs5Q= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4094.namprd10.prod.outlook.com (2603:10b6:208:11e::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.14; Wed, 15 Dec 2021 15:35:49 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:35:49 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 01/14] configure, meson: override C compiler for cmake Date: Wed, 15 Dec 2021 10:35:25 -0500 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d807fb28-db52-48ed-502d-08d9bfe091d4 X-MS-TrafficTypeDiagnostic: MN2PR10MB4094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Wu9MtQ+iunguZXDExnhPd3wgCElV+/JlVXEF8rBZL3Qj9vxq9BlJYjE6aYcle4cSqosK8gROIWAV00mj37g7wGYIAV0Is4U02DPPJdyO1KGq3s/hprFtW+j/WVCeXb9QO2zKy37m+eR91NX7QMos9vQ0y1FlbZQR5Ma4LPXxk8//+oTYZ8FFcRC567q72mLStpP7fQw8oa+jmf8roEvNAmCGKcLmgi7C+KtVLOd/fYk8fwuPNM2GHF7U6l5mja2boi1JEx4BBgSRdaoiNGQozkXrF/eZfpRWYKvh2AsgdzmAPcot4Q/hGqkA8PxumZCZxe8ZlVP39NieAhhRifYgbQoJeXhjqGRmmgWS45scj35ECj3KIrdzOYRACbSo5kG0EScAsGKQ+r4pEuTpsrVjMQ1h5EeKv0gmZ3tQk24w3wxsSgGLgo4CZSwWjfaPedCxtsRFLMShoCBOanHCw4ZuYm/Rbqr42zMVPQTXV7G58FUgVTNcevEAMrY2NrE18bLtvCAo4wxXRR59PPWDyDzFDL03mhf6GJR1Wq5mDxUMI0AFzDzMFJ9O1aETvVz4B/s0bk9PAejZmqXssRO0VooxjmKzQzOSSzkHzvjKw5BbcJP6psR945np3+Q08qu4oXHsjVbFdHVYLY4c9w1un8/LIg43TqwzSiYYSUmJSaMxUERYx7IEd9hDI6YXeDz7PfHaV5kCRd0HlgftF6Vk6Vn3Zg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(66556008)(6666004)(52116002)(4326008)(2616005)(6506007)(86362001)(8936002)(6512007)(5660300002)(6486002)(26005)(2906002)(38350700002)(6916009)(38100700002)(316002)(186003)(107886003)(66946007)(8676002)(66476007)(508600001)(36756003)(7416002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oU7K4kA8yWPGHLAVj/HFasuqcX64IzVUAWPeq4i1juFj5UAaaBwOQ3tbvLhyyNBKv9xfLEcQ5U7/v5DJbSvWywgx6D+LCEIh/IyeKsAq5tCcPQ189sfaE3ZTDRq8ES1KKQKewCI8iIZ5oVZKOofmUbJvJT0u/fU4oO33dhj5vuVCyYT/vWGaKysIDQaZlmqXAaD2j+3OeBgTQsfBBGgJVOKmBBIJi7w0NSFd71HGFBpGlh+ZlZTTUHVR8ml6m3GIU8aSUPRCP/IedCFDmlYRxoay1hVigCubpnlGHdQEgi1AUsMw7KZ7UzvrTAfSIpcZME+1PK/pzl6ocPR0Yiui6VNZNXFQ9Drh8i2Ll5zlMpQEmCgid69kAddFg3HXQ3unuLJNKL8y0aphlFhgQhIqttmnchDa/9jDSHRy7/SMcMSXjEFfntAJNJFvE+IxqypTonTbGwF7NR1k59iDm5uXvDrKAL6uFzywUCsfEV62U+89mqP9Pf0wcGgStpnJJcsXG8kBObzFKwOv0jzkt1Qw/403RaUW+gRbB1Z98NkZw/WbBt1hCDW6zVQu6AcOlsnpJbYcvoO0Bj2OXZsoN56pfW09HPCwKTntWCwIwK/zwzUU27uZSOFkwVGSd1r6xOR/pyA1jCb0nvrM51klD0VfP6yhSTjcXPs3YP9CFriEMzDxr8SU7sS1UG5bjgkPgdOY5ga7lvhjwA6+R9hBieBPTV64QQ8+X7/5/Qxc3XhT/FekeXgVu31YM45mzGcELRwznlcanmE5UHCnJtDg5/vzeIGJHaolZORD7HQNOqJk03exS5s58qpK3aZ3iq3xkKk9ZebWPAc8zOiuESEnUEZ6PHT7RcEWM+kYCA7pGJk/lgHAlcAr6izc7mXBNBoyaEh5V9GClLNIwxJth5px39+lhK5VhFvaUldtc62I6UO3vz0jmt9Ze3FjFT28j+CJVoJVgFWlqtiLLOaDmj5DpVjUEcp4CINtA2gisxnjNkJ0g1RGckbtlUJLIv4b6Ig5KuSjdIYRnZCUELtus8rZjotbz3GNyqVZKDknSCnJZwiq3VId7iUSJnsgnODBUgBQdUqANEe6PRVqp5XStImi2OqRQ+nrTQw46ndv5AdcZkss4KNJvvSx4nmxwCR+qSuMsPVlWibzoAHlN77SqqHdTHSSWAjBt36/VJgXGvAWPp9511/Y1RXBULYw/Bxx6piRcYTlzW2T8sXqlA+LII+/zMTCMD0ch3Am7jsqQkvFQqODmVwhmRAVR1Wia2T4g4LyKeB4h3iBxcUT2hj4F5HUwf6MzxtYBKEMIqyqEQDh5DITokceQoqw6nmtBZTsEWWnWf0Hi3a2ZB2Xp0zE6itTCVhdFRLU17aXe+Nh+Yoww9xGfHCvmgbZRDvH6wBDChlU4vwjT96qLumDYO8cKAYrYBbU6UB0GDT37ftPSdn+rziBlyoIKjks1R5QeO55JNnd7VPOVYIr7s/pnHimpCczL1ZGzPVaOYzyF2UFn8ULhoIXoCP06j0eoCRLn4bXabkb2DtHhCyWLO5GXSNztVefjhRpku4RVRgMV+v/CbavtHdP17FmZ+X5VssZDWRY1XzlOBAiWnuu79+aLmaPFNzhNC0RLeBQdiGC+8o3BHvw8xgXAcQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d807fb28-db52-48ed-502d-08d9bfe091d4 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:35:49.2481 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2/Hm5GtXgoAJvqWt5dYdwkt5keNH3DPC8qUCYUdDwc3pgenxxgAl0LtE4DvcXv/FH0USUCk+or3agxytHIy38w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4094 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: 6rhtBHMrMtQ4cUozpXlexyhQqETHcX25 X-Proofpoint-GUID: 6rhtBHMrMtQ4cUozpXlexyhQqETHcX25 Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The compiler path that cmake gets from meson is corrupted. It results in the following error: | -- The C compiler identification is unknown | CMake Error at CMakeLists.txt:35 (project): | The CMAKE_C_COMPILER: | /opt/rh/devtoolset-9/root/bin/cc;-m64;-mcx16 | is not a full path to an existing compiler tool. Explicitly specify the C compiler for cmake to avoid this error Signed-off-by: Jagannathan Raman Acked-by: Paolo Bonzini --- configure | 2 ++ 1 file changed, 2 insertions(+) diff --git a/configure b/configure index 48c21775f3..dd000ce299 100755 --- a/configure +++ b/configure @@ -3919,6 +3919,8 @@ if test "$skip_meson" = no; then echo "cpp_args = [$(meson_quote $CXXFLAGS $EXTRA_CXXFLAGS)]" >> $cross echo "c_link_args = [$(meson_quote $CFLAGS $LDFLAGS $EXTRA_CFLAGS $EXTRA_LDFLAGS)]" >> $cross echo "cpp_link_args = [$(meson_quote $CXXFLAGS $LDFLAGS $EXTRA_CXXFLAGS $EXTRA_LDFLAGS)]" >> $cross + echo "[cmake]" >> $cross + echo "CMAKE_C_COMPILER = [$(meson_quote $cc $CPU_CFLAGS)]" >> $cross echo "[binaries]" >> $cross echo "c = [$(meson_quote $cc $CPU_CFLAGS)]" >> $cross test -n "$cxx" && echo "cpp = [$(meson_quote $cxx $CPU_CFLAGS)]" >> $cross From patchwork Wed Dec 15 15:35:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CA871C433EF for ; Wed, 15 Dec 2021 16:12:34 +0000 (UTC) Received: from localhost ([::1]:43894 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWMr-0008AL-OO for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:38:57 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWK1-0005EN-9J for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:03 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:29980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWJy-0008Rp-QX for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:00 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEgO1x008084; Wed, 15 Dec 2021 15:35:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=8xoyHZEUyMcTWfa3wzI5ZpQ8ifq1XM2JKlVzhEWkFXg=; b=rUVWsrjORRvc8NedbzBghZe+Wq2PlWWxnFs/+5d+wS/HyIfCDbu3Fv1TzZnhautAwpSt 96pa+cSpZsxiMRPdP2SW/JXn1KAf7D9EBXep8V2wgnWSZcYTtGRRPWmTNZFfrdh4zsaw VoncVhI4z1jYL9AeB9ZZQau/MQE8SQ6O0/jhoIUrXLaz8NXXX9XdMHHoTIPIpW7YJcoI TvxmYS4Y2WiJYXg37+gSa+/NeamFr1vkzUNMMOjkpmXjJccTOTsC4nSOg60x9CWnmlXi S8zxENdyhwRf7GTcrbhTXTR5Gl0mCNZMN7vxOM2VpgymawqHnquyqZfN8mFdU59Z4N+F 2w== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx2nff729-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:35:55 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFU7pW165413; Wed, 15 Dec 2021 15:35:55 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2104.outbound.protection.outlook.com [104.47.55.104]) by aserp3030.oracle.com with ESMTP id 3cyju8grxe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:35:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xr46n2B/heXBSde0h7qi073JazabFYaEUkkyhJAw/C7QlUlzp6ZMiKtwiqcFuAW9wxi/ZW6hAInuL7+mu0Oh0eDgXTJEz++LKvIs3FY1zqVUq0NwMzW4/Qa8/x67tbP4jLm/N2Dcka1/0/nR78CsN/h/41hG8VqOWp+4v0aOyBQZF98b3LLMD6KrThBEl3+uSCmw2ozkL3OLCvEEKB6OW1RkUmSe8g86HlAOh4f7V5HqoNT/uOfp2ZmnAVQUsj04DTW9ndQLBGnlWQBEdWx/ux2vb4azg0ceZ6a53DpjMU0a9FEyazxw1WiwBqCGvLtZCdBNXBNfAbUWojsgHY/0aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8xoyHZEUyMcTWfa3wzI5ZpQ8ifq1XM2JKlVzhEWkFXg=; b=GIB0bT88CTv3UU55qHK7ewde4lBeUBYBkYDI0/RbQSXjtnv8yGwKhGVpTSJzGHR87m7WFsuk01WvmAG7h9g7S0zSPS08h8ohIdXajsD4jCPC2KTeO2j0JFBo0mD3eswkckINHeZs1J5vctNFHKN3hAH1pgZQKrLMpuV7holggNpFScchAzFKGDSFeIFZRFxFxdIVzvWapXgGWbrM4ezG9m11KGP2mVPpPt8UCztmVD3BN9xkZAcE6PWrpk1FZMyWzsEm0NZJQJXmscF/jjnBetxsuMmVAfj2HozvVfkVZVoSZsVj851ckLyPpYTM0LK8nnUv/VaRHKK9qJHxgl7P6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8xoyHZEUyMcTWfa3wzI5ZpQ8ifq1XM2JKlVzhEWkFXg=; b=eLCARr2XUBYQLD3WSFFWbZSnPwc79GEr4yK5spNrOxbfwVhtIqoFStWyuuSuB3f6mqm56ZPg3E7hHVn6i8C5hV4Q1j57fCP4ELqPo80D7/4Q6ULQcL7MCXDlv0NVss9woEd48zNBRNpv8lipWUh+oYsklpVYuUH2+W0c7zYUTQc= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4094.namprd10.prod.outlook.com (2603:10b6:208:11e::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.14; Wed, 15 Dec 2021 15:35:52 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:35:52 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 02/14] tests/avocado: Specify target VM argument to helper routines Date: Wed, 15 Dec 2021 10:35:26 -0500 Message-Id: <31a4b3bb6a48f291890641ab348e7749c70a2d06.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a8681a5e-c886-411e-e001-08d9bfe093d4 X-MS-TrafficTypeDiagnostic: MN2PR10MB4094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N3QqvARxH0gogXFi5zT7x4pk0ezOw2UkwhGE+NScsf7GtSf8y6ieWh3QMJVjm1qk7fsDD4Hz2/D0yt1PNbqyOhYejVnXgLn2yLyJVKj4M9RfTkm1c/AOOREF6r84jiU2o45UB435t9KvZ73YyESMPbN71XwWknNOAf/U+h8f2s210sAQePiC/DOy33JJPMzdBk3sQcY2KqL9EjF9pomE5dJ1XKD0gW+D6RWNRWdvAIBnhXyY4UephBndhRyT80loqGIpFifOtUyeBPtqMnFe6RGspD4Oz+RLKmtKB4XsP2Kxjj7KUlcWDAL8XbKNhqVBY3IDqvv2CS2HFBzwqp5ibReEKyi1RU1/jI+4yyyfuIVs6IP/nRx91v+2E2in0zqF32ApwLcOAvisIK7BYjCxzMm3VB5nJ9s73ohfB02/557lTah+vPTqRKKGMpRaGLDru1boJVv1IZU8H434DA7kwkL38hE94bjE2U8WeudTFi7j56myYkoMymzLqEPZkKjk6aqD7SOWXLFc1n2Ocrw0ah3J0Ib+axK3GiYSotE3NQRj90r3UNlugNQzpsrXAb4cKFKbfO1AmDzsXS2vxKLg0p5cf3X8ni3m0l1C+ztp5gp4xq8OnthBkmUr7wPmUVuY7ynb7gzA9nBo7TdDQ3zDWvmvAUh6JLcNWatcy0Lxkd18lxlxPX+uzV460XUaWeFWu03kYNDaKb+sv6Iuv/lcIQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(66556008)(6666004)(52116002)(4326008)(2616005)(6506007)(86362001)(8936002)(6512007)(5660300002)(6486002)(26005)(2906002)(38350700002)(6916009)(38100700002)(316002)(186003)(107886003)(66946007)(83380400001)(8676002)(66476007)(508600001)(36756003)(7416002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tzCreNLwK02HA2D1cvXx95NQBPBDbfm3822zEdQ9AJIboLsehGi3B9Wl+oMjD1+mHSWp57RW6TiIUjTPTgV4YrQbUCk68gASHG22h9h4DbYNNRe0BrmrPhsCYiBakGxjO+pxVNqI1hjO5Ki1N43pfLeaWr+ImmhTcESAXW+qqmhNUO2R05HTcpwdkY6cwa++VJ/4DCzkCXwnO1Ax5nYDRIeteu2YHSESpzhCProU3cdH7Lk+Z4EPfxRTI5tiSD/ApGA1qaVHc46hxpLL6QDrTeFEztalvgtA1GT2ql9fAzHqkduDPIItNQLMmD5MhwI+0G9cDRNiJd7h/YqVa9s1LhUpqHT88JzPO2Y20gw92KTUiCRHs/caErE0P12oKQomH1htgBXJJP/NWSYxHSELI8KVGDTIZy08F3PILYfOtNQhWXb8F37pmZfzLgrtSPvQw7rtiXn2nSxkZNF7F4FnKH5zXAecWotG5k0NhyNW9GrMN5ll7rLwKtjFfCY987Z1kBO8orcH8FkJm365jto25iz+7lpGa4QEac3yZxWKYby4VS5FCksw/mWkULjTJby8U2A8eFaevkOrmjdLzU8/zre7xUTzMpCl/+4uG/NaT4HNxY10tkh51E3NMLCUcYSgu+OxRqgkatObJNyOkSiu5BiALgBewoxvab3n38B5YQjV7NUIV3tYmJUyprIET0JCTzCrQWuGkudHOxkqJl3r5qqIHxSXyXNjkvUSB0FIKfEl/FHdgCCNrtvjXHTh3d9FWHeOwZq+TlBZOXHK4DEpY0hWMiIqSIM2I28rsjgpWc8TvEZy9hvZ2KwCtG74qd1nD4raa1V8iKOw1Yvos5aR2cKCISYULk85dGkZrjN1R8ZYawAXo1DDYjYbiOOEHKzN64TlkNa5yOHzvZDaWmOap1kUItvHKWx6XaD+Pjl+Onrkjn1PLgge1xOADMXMnsE5gmu9Hb0kFN1u2EQYir/hm1M64Zo27JDiH4NGsuwRhnjjN8HkjB86UYIxjzm7Irzz0mFY+hMfUh779TV/QLmKY5zvY5rcbMrnC30CsK3Cnnh4TBcBPeoRpgxAig4vOZoXp5CWQz3WSplSpkH6hGNTLFURrFdNlcfUurdZm89Tvo40ecFR1GyV4NBBeg9Tmo/hw7XQ8RNDaJYufHST/e8Ct0eysQRtxv3rCL6NX+2xAmTAnN2XOPf08Pm6WRsTk2/RwprXFpLK8JCZrkg8flZZM3F3Brr2Rad5yiuUCzmcMTQYYsC8K9Y/iBJDnb3mI4BftrJpgEDV7qPlIK3NuT9PT8BnMLpnNlSWCZ/3IUvZgSkZG9ux3fEns6oT73Qfkn9oPR1qXLx6E9jIHJ/HbRU8EvfaOMEDfPmJr5u3fMxrB5hzg1QgP+2qRQZ+VrCJVT6B5RX4XZL2t9/gcmCbNQJYkBkhHxrk/cRscYScpBP4yZSfHAnfK4gyoSAmCv6YZx3T8TToJ4avRCHj8mqncd1iQ4S93kGkUezPJkBNzZRn47AjSeWqfRrDnfg/lyIELYO6pfEUC1ZKfB6E5ktfG0AIOYDYJprdP+FAw+Bz3cKogXhrzN6/iRibAAAfXqNaZ8mPWpGunLCuRbRSwdWutV2S2RVQwtUdbF2vKixvbvuceP0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a8681a5e-c886-411e-e001-08d9bfe093d4 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:35:52.5135 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BFugJaW0RPQcegduq20keWfnjtFKJvEQSx//ND4XedB425gOk8LYTxVto61G2pgnDYAm3JZvBmHn55PsFkQ/Tg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4094 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: KHHKo9C4Qc67zIblDUZGx8E5kQ519BdI X-Proofpoint-GUID: KHHKo9C4Qc67zIblDUZGx8E5kQ519BdI Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Specify target VM for exec_command and exec_command_and_wait_for_pattern routines Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Beraldo Leal --- tests/avocado/avocado_qemu/__init__.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py index 75063c0c30..26ac782f53 100644 --- a/tests/avocado/avocado_qemu/__init__.py +++ b/tests/avocado/avocado_qemu/__init__.py @@ -198,7 +198,7 @@ def wait_for_console_pattern(test, success_message, failure_message=None, """ _console_interaction(test, success_message, failure_message, None, vm=vm) -def exec_command(test, command): +def exec_command(test, command, vm=None): """ Send a command to a console (appending CRLF characters), while logging the content. @@ -208,10 +208,11 @@ def exec_command(test, command): :param command: the command to send :type command: str """ - _console_interaction(test, None, None, command + '\r') + _console_interaction(test, None, None, command + '\r', vm=vm) def exec_command_and_wait_for_pattern(test, command, - success_message, failure_message=None): + success_message, failure_message=None, + vm=None): """ Send a command to a console (appending CRLF characters), then wait for success_message to appear on the console, while logging the. @@ -224,7 +225,8 @@ def exec_command_and_wait_for_pattern(test, command, :param success_message: if this message appears, test succeeds :param failure_message: if this message appears, test fails """ - _console_interaction(test, success_message, failure_message, command + '\r') + _console_interaction(test, success_message, failure_message, command + '\r', + vm=vm) class QemuBaseTest(avocado.Test): def _get_unique_tag_val(self, tag_name): From patchwork Wed Dec 15 15:35:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 645D6C433F5 for ; Wed, 15 Dec 2021 16:25:28 +0000 (UTC) Received: from localhost ([::1]:33754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWU9-0003i8-WF for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:46:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42894) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWK4-0005Fl-Tv for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:06 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:41334) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWK2-0008Sq-CV for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:04 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEYwBb008125; Wed, 15 Dec 2021 15:36:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=ORTJgFGThi1I5c5bQEIkd0o1YYX+KOTuAhGQ4CaypUA=; b=OwB3wNxSfMC71nyFo6SA4hstMwKKtRTW0be0byFMdDXc1KGIugtdsgqVP7owTbYVhRJ/ cGwWmwwkRcAGFB3aWkluv5H20lHG4mTnPZKZd8gcV4Uo2XJavWYEuRnanO/4egTXaH8f /a65EDDJbAEvNqwTtt44+pyyytNBCRBPACTDO/dzN7m1Bf3Mf5rkzNYmRd1dUrSTsMM2 b+vAckAKzy8Gb6JMype1KIab2+XfOwscYV8XOT3RyTBNZFQj/ELdJxrpC/PZSTQkLAwS 7eQU6du3Jfsd5/nr8ojUAt30o4h0EWKXbRFDiBksK5dYqMQmgIhypBaWc0JaEfk/rgx/ gQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx2nff72w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:00 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUTJp083868; Wed, 15 Dec 2021 15:35:59 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2042.outbound.protection.outlook.com [104.47.66.42]) by userp3030.oracle.com with ESMTP id 3cvh40b2e1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:35:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hXoxKJlVFhZX2LNvyEao0xpbEA3I4tWms3obCQKOh9DQuZUycmrvKKOVtZ4FNpu+oP4RU8boVWIeOQRwCLeM5XMSYOzyfLLhDC9JUbzH1lnfg/yvHZh8fCZQIC5j2hpqCz7vWmwuqOVZKCf+PnlVjGmegJIzHhaZP+WPEvWlMxSR1jdgi4vMkHc4NJZLAN4+a/W/pAovnKpR9rcgNQZM6mlfZeX7UdpieNGlr4dbr6VwQO5/ROcYpv5HCv2FfHSLpd9we2e/42l+tYN/oFvNj1K0oCaF8IP1AOQjirJ6IPoumOLLjfYqwVP2XAuygWTFKynuRqswiMDp48DmpLxLOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ORTJgFGThi1I5c5bQEIkd0o1YYX+KOTuAhGQ4CaypUA=; b=Ixra35p4iQC5/QlCLhrH/lYXnoWPOXBqQwQ3tHKgP7/RsODtUcwYNrATm9oOx6rQwHdxsTxALbpznA80/v2lAUoFbHpsK0LF0C3mek5nfzrf9n5L+sCDCf/oZ8DApmTIq5q4EqHXjzsUyiZ+BFiKdFGVo34OicLbFrd57g8b34xC+XKvZs6Dmfq8M8XQsa4m66WVdP52FyPe72bfg0B5p+YbyvgwGyTx482/xTr5VDyNaszyPFCHH1T3DUlFHY6ViyiW/qr1uPbSXbM5xKyBotEfnBaXMUm3Jj+6i25ypbOa0vGBKa5btk1h8I267L7yzeaIMIw3VfzfNY1KTgbFGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ORTJgFGThi1I5c5bQEIkd0o1YYX+KOTuAhGQ4CaypUA=; b=e323frkQNbnjb7g2QIDIT+2A+mv6x1QExyYJQHjCDYZ4xyi/xJbOBslOY/ffuLAnwhZpfahcgUS39halrc6pf3U9brIM3Vzs2OCrBxK7kYe57ROnkjgPZ0SCswMFc2Y5XoMnvU5VABiYCuMz8y84ZuWD2PcLjAYrJgi+jdrPLZM= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4094.namprd10.prod.outlook.com (2603:10b6:208:11e::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.14; Wed, 15 Dec 2021 15:35:56 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:35:55 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 03/14] vfio-user: build library Date: Wed, 15 Dec 2021 10:35:27 -0500 Message-Id: <2e3a3054ebc7ff9ce0d1cfeb42a73acdf84fcf9a.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7405f74c-200d-414d-6fea-08d9bfe095d5 X-MS-TrafficTypeDiagnostic: MN2PR10MB4094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5yeLLzUSMQavUJvbzqDGQYFqLlnkHb6/9oDFNsMnucPKkawwBSgvNm/I5TFiKVkOC9LIRXfTEbhYu6XkCQ5sVz9n0F5WrXIx2e+Nj7gbYRJTEVNQX7Ws7ImXAz3gRPZvWngq+1QTXOKDcfQiXUgNAhBoaLdgyqRSrls4qH3oTv1EuFtCJym2AuP1toraE7QbenoW6jFRHBps+r/PRmU6L+WRyGAIjUFvIndVM6/GhgaOL1MFGXk0NKBJDU3i8BKe8hgWhJWya2FgW0Hgaotg/8addHc1ppzKJ43KJlp/tNwkGtGUof033ij2k70qzS2kwA6zMMYhwcaGruCLZ2nhiLh60SXSAlQQW0B7uOALSXcN2pXpXPxlARfnUjWc7J6ScFwYwPTzlYbJaU0XD5OuRNm/n3qiJSJH6n8EqbklojjYBg52xFCN1z2TWhpDnq0UMr5IkPRL8jH2i2x0gRABtMDrJn1x/hRAYEDowMPA8zumgnawBV5Cyvo2RZLOhwkmrgstgViaDJvx6owtMhUmOE5LKozHvWLbcUx33SwvH++sRUAPyrXNXEc55xa9zE/4PwKzXqPUV8WXad8kz/FjOrMqP5gDgfveTxPPxPWfVok8pKUVtzaImTT2BUQCe701lO/+nfHdCvsAvz6wzoP/7KbTkTD3+e5g7kOtdgk2tNifJI/1Rhdo6b9C2RaORCmSSgTQXm0sArxXS4VsLuHj2HRU2vIiQmFoCyNmqP5yVVBT7VhJR1TFr/uuqdOoXdA/y3V4qpl+t9naS0+cWcsMgbu28utqr4AOrczr2dufIfDO9iVcUaDScBl8vNh8hnWNmFxQ6uncXCmwG4zq39eo3xkM22ZHd3mW+WqISHaP3K4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(66556008)(6666004)(52116002)(4326008)(2616005)(6506007)(86362001)(8936002)(6512007)(5660300002)(966005)(6486002)(26005)(30864003)(2906002)(38350700002)(6916009)(38100700002)(316002)(186003)(107886003)(66946007)(83380400001)(8676002)(66476007)(508600001)(36756003)(7416002)(21314003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 9fEG8N/RLiiohmsjxVhlkSAITnJtR85p9dgvO2bu/7ylCHY3sEr8hN4rkTKQF9q96KzLkXkv1XThwo2+CfkxiF5jT3DafDY/O5GNXihhtnsKVyQjd7EKiYoDvctNnsfxrIobAO5nPF2uoJxd1AZbQQJjbhgkI0griPPOz983DhfrcO5qbZbk6YdgJsdHSOYdf3+TpPWis80ygZZbcTdaO91zV09gT14P4MBls2HvW975voy4HONHi9drpmY4yYq5I1+gSQZ0fabfbSGo8umXhZehaAaumzVVfTqctJAPvzNWofjIttC1FAXEyuP95mOj7ZeDFzEdJrbXUnF+3MFJJa/ID01lFcxhFk3A3A70MUuvkorLWns48CgIPHzTNlabzgYRZoof/u0gBQnuW6RjBdm4s117hEGmWqr7pwtxiuNO0Fv7VytSQWq8RegjHV18T9IfGe3jVZP3/R6KwonSuG8QuX7910FqxYizMrnEsS62dM/8Ip1cZ+7W/BOWxpZLro4c5YEXNwXIrpviRdgjv8UVPWskf6SV/a+lqo3JcZZNPWfCPMLak5DiiXWSyEMoX4bLwofRAhQjXtYZvueJuyPMXRd01/iJdOFcwPTQ/XRG7/63Alx2JdZm67ec/oGfead9/5kSxr4MprlMzn22DYA7WdhNW45NIoGbuyb9oOc0m2BzM99O7x0UZMdUBb+KOY7gY+FxAo52vJ5GtvZj7RxrHyjwzVjc6kTzPVY+Wts6T7etuN/e4b2ZpprkA8xzR0Zea5ERwUyZhs8Ax2slab8ZxcBvxKbPYrVaswZ8tyXPSbN6pdx9/03YhTLe57w9BdhtjxLUzCh3482aP+oamF9Yx/QrIdeJt1xlVXCFqBWdophUe/pZJbsKgiDuXtgNcHtiO4TZKCxQP2xBXM2ShHnfjxG8x4Ag5tsg+gNRstdYcuvaknQsGc2uZwSvjvhe4gMfuv5OxOOEBsyq8pZnOmUiKpEj/kVHMXf8lUzibD4ZTHj67/ahmszk8Jwi+J7L0tGlvS/5rSgnUth9mkv0VEA2wryNwP9eg1VMi95+irh2jR50BYyX9mO/k/IkjVQRvwhQTDWnBK+42koQ6TnjOqeofDjMrqhpzsq6Y4a00HnIKcHmZrdZHhpmIwjYaux8FMXho3aFGnS1aDrJLanH0Xcibye7jHUro72cysccpdLfSmPnubocVOKKjOLAdzJG9bKcHx/qQQ38SFM4x+MiiaaGgfW0OfKMTPsIOetaRhxeelITwFDUxwtU9EpljMCMpdC3PdYKqz8sOMPx0p78rBas1XLQ45SZUVs3OZmjRUpoGnRYn7qBBcgB/ToCE57s1RgTChWo+oCXuCzteQzuDBoA0006XmP1cGlbQILhEA5i9G5fSD2JcbYUAhcRDpRt7q66RBb9B6BLFLh2P0PeCyReiFyvurJ4sNV2wSvEPjDKYIearluLEohv2MmNSlz2C9ONIAUfq0sWrWXTGR7F0N1SC+dLO9+QZduTQwDPw1Dk0r0zCR0dXQ6BItWFKPm1Fhd5B5xpYU1rsvJgBIBxD1vKKuylw2++PqRlNyeebNWd+6VWZAM0GHbt3QSHYqjfhPf1lzHTy89zzOq3nFxmnXoKDO41/wcY1NcW0HBJ+d8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7405f74c-200d-414d-6fea-08d9bfe095d5 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:35:55.8726 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1tTLZ4lRYS+4eF4t7BotQw5/oTANhuE2IpEdU9X6zqp/90msJsVbdPvbJlXGH6JfU06rw1LF2ZoaQ0Dwtz2hzg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4094 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: 2GSTVXy06AbjaQ8ycClTu6W6t3wBJFai X-Proofpoint-GUID: 2GSTVXy06AbjaQ8ycClTu6W6t3wBJFai Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" add the libvfio-user library as a submodule. build it as a cmake subproject. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- configure | 19 +++++++++- meson.build | 44 +++++++++++++++++++++- .gitlab-ci.d/buildtest.yml | 2 + .gitmodules | 3 ++ Kconfig.host | 4 ++ MAINTAINERS | 1 + hw/remote/Kconfig | 4 ++ hw/remote/meson.build | 2 + meson_options.txt | 2 + subprojects/libvfio-user | 1 + tests/docker/dockerfiles/centos8.docker | 2 + tests/docker/dockerfiles/ubuntu2004.docker | 2 + 12 files changed, 84 insertions(+), 2 deletions(-) create mode 160000 subprojects/libvfio-user diff --git a/configure b/configure index dd000ce299..7748802d11 100755 --- a/configure +++ b/configure @@ -366,6 +366,7 @@ ninja="" gio="$default_feature" skip_meson=no slirp_smbd="$default_feature" +vfio_user_server="disabled" # The following Meson options are handled manually (still they # are included in the automatically generated help message) @@ -1190,6 +1191,10 @@ for opt do ;; --disable-blobs) meson_option_parse --disable-install-blobs "" ;; + --enable-vfio-user-server) vfio_user_server="enabled" + ;; + --disable-vfio-user-server) vfio_user_server="disabled" + ;; --enable-tcmalloc) meson_option_parse --enable-malloc=tcmalloc tcmalloc ;; --enable-jemalloc) meson_option_parse --enable-malloc=jemalloc jemalloc @@ -1470,6 +1475,7 @@ cat << EOF rng-none dummy RNG, avoid using /dev/(u)random and getrandom() gio libgio support slirp-smbd use smbd (at path --smbd=*) in slirp networking + vfio-user-server vfio-user server support NOTE: The object files are built at the place where configure is launched EOF @@ -3250,6 +3256,17 @@ but not implemented on your system" fi fi +########################################## +# check for vfio_user_server + +case "$vfio_user_server" in + auto | enabled ) + if test "$git_submodules_action" != "ignore"; then + git_submodules="${git_submodules} subprojects/libvfio-user" + fi + ;; +esac + ########################################## # End of CC checks # After here, no more $cc or $ld runs @@ -3998,7 +4015,7 @@ if test "$skip_meson" = no; then -Db_pie=$(if test "$pie" = yes; then echo true; else echo false; fi) \ -Db_coverage=$(if test "$gcov" = yes; then echo true; else echo false; fi) \ -Db_lto=$lto -Dcfi=$cfi -Dtcg=$tcg -Dxen=$xen \ - -Dcapstone=$capstone -Dfdt=$fdt -Dslirp=$slirp \ + -Dcapstone=$capstone -Dfdt=$fdt -Dslirp=$slirp -Dvfio_user_server=$vfio_user_server \ $(test -n "${LIB_FUZZING_ENGINE+xxx}" && echo "-Dfuzzing_engine=$LIB_FUZZING_ENGINE") \ $(if test "$default_feature" = no; then echo "-Dauto_features=disabled"; fi) \ "$@" $cross_arg "$PWD" "$source_path" diff --git a/meson.build b/meson.build index 9702fdce6d..339c28ee25 100644 --- a/meson.build +++ b/meson.build @@ -252,6 +252,11 @@ if targetos != 'linux' and get_option('multiprocess').enabled() endif multiprocess_allowed = targetos == 'linux' and not get_option('multiprocess').disabled() +if targetos != 'linux' and get_option('vfio_user_server').enabled() + error('vfio-user server is supported only on Linux') +endif +vfio_user_server_allowed = targetos == 'linux' and not get_option('vfio_user_server').disabled() + libm = cc.find_library('m', required: false) threads = dependency('threads') util = cc.find_library('util', required: false) @@ -1824,7 +1829,8 @@ host_kconfig = \ (have_virtfs ? ['CONFIG_VIRTFS=y'] : []) + \ ('CONFIG_LINUX' in config_host ? ['CONFIG_LINUX=y'] : []) + \ ('CONFIG_PVRDMA' in config_host ? ['CONFIG_PVRDMA=y'] : []) + \ - (multiprocess_allowed ? ['CONFIG_MULTIPROCESS_ALLOWED=y'] : []) + (multiprocess_allowed ? ['CONFIG_MULTIPROCESS_ALLOWED=y'] : []) + \ + (vfio_user_server_allowed ? ['CONFIG_VFIO_USER_SERVER_ALLOWED=y'] : []) ignored = [ 'TARGET_XML_FILES', 'TARGET_ABI_DIR', 'TARGET_ARCH' ] @@ -2201,6 +2207,41 @@ if get_option('cfi') and slirp_opt == 'system' + ' Please configure with --enable-slirp=git') endif +vfiouser = not_found +if have_system and vfio_user_server_allowed + have_internal = fs.exists(meson.current_source_dir() / 'subprojects/libvfio-user/Makefile') + + if not have_internal + error('libvfio-user source not found - please pull git submodule') + endif + + json_c = dependency('json-c', required: false) + if not json_c.found() + json_c = dependency('libjson-c', required: false) + endif + if not json_c.found() + json_c = dependency('libjson-c-dev', required: false) + endif + + if not json_c.found() + error('Unable to find json-c package') + endif + + cmake = import('cmake') + + vfiouser_subproj = cmake.subproject('libvfio-user') + + vfiouser_sl = vfiouser_subproj.dependency('vfio-user-static') + + # Although cmake links the json-c library with vfio-user-static + # target, that info is not available to meson via cmake.subproject. + # As such, we have to separately declare the json-c dependency here. + # This appears to be a current limitation of using cmake inside meson. + # libvfio-user is planning a switch to meson in the future, which + # would address this item automatically. + vfiouser = declare_dependency(dependencies: [vfiouser_sl, json_c]) +endif + fdt = not_found fdt_opt = get_option('fdt') if have_system @@ -3301,6 +3342,7 @@ summary_info += {'target list': ' '.join(target_dirs)} if have_system summary_info += {'default devices': get_option('default_devices')} summary_info += {'out of process emulation': multiprocess_allowed} + summary_info += {'vfio-user server': vfio_user_server_allowed} endif summary(summary_info, bool_yn: true, section: 'Targets and accelerators') diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 71d0f407ad..e29f8c1f13 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -42,6 +42,7 @@ build-system-ubuntu: variables: IMAGE: ubuntu2004 CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-slirp=system + --enable-vfio-user-server TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu microblazeel-softmmu mips64el-softmmu MAKE_CHECK_ARGS: check-build @@ -142,6 +143,7 @@ build-system-centos: IMAGE: centos8 CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-fdt=system --enable-modules --enable-trace-backends=dtrace + --enable-vfio-user-server TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu MAKE_CHECK_ARGS: check-build diff --git a/.gitmodules b/.gitmodules index 08b1b48a09..cfeea7cf20 100644 --- a/.gitmodules +++ b/.gitmodules @@ -64,3 +64,6 @@ [submodule "roms/vbootrom"] path = roms/vbootrom url = https://gitlab.com/qemu-project/vbootrom.git +[submodule "subprojects/libvfio-user"] + path = subprojects/libvfio-user + url = https://github.com/nutanix/libvfio-user.git diff --git a/Kconfig.host b/Kconfig.host index 60b9c07b5e..f2da8bcf8a 100644 --- a/Kconfig.host +++ b/Kconfig.host @@ -45,3 +45,7 @@ config MULTIPROCESS_ALLOWED config FUZZ bool select SPARSE_MEM + +config VFIO_USER_SERVER_ALLOWED + bool + imply VFIO_USER_SERVER diff --git a/MAINTAINERS b/MAINTAINERS index a8fa61a20d..e0daf349ae 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3464,6 +3464,7 @@ F: hw/remote/proxy-memory-listener.c F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h +F: subprojects/libvfio-user EBPF: M: Jason Wang diff --git a/hw/remote/Kconfig b/hw/remote/Kconfig index 08c16e235f..2d6b4f4cf4 100644 --- a/hw/remote/Kconfig +++ b/hw/remote/Kconfig @@ -2,3 +2,7 @@ config MULTIPROCESS bool depends on PCI && PCI_EXPRESS && KVM select REMOTE_PCIHOST + +config VFIO_USER_SERVER + bool + depends on MULTIPROCESS diff --git a/hw/remote/meson.build b/hw/remote/meson.build index e6a5574242..dfea6b533b 100644 --- a/hw/remote/meson.build +++ b/hw/remote/meson.build @@ -7,6 +7,8 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c')) +remote_ss.add(when: 'CONFIG_VFIO_USER_SERVER', if_true: vfiouser) + specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('memory.c')) specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy-memory-listener.c')) diff --git a/meson_options.txt b/meson_options.txt index e740dce2a5..3c2d73481a 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -66,6 +66,8 @@ option('cfi_debug', type: 'boolean', value: 'false', description: 'Verbose errors in case of CFI violation') option('multiprocess', type: 'feature', value: 'auto', description: 'Out of process device emulation support') +option('vfio_user_server', type: 'feature', value: 'auto', + description: 'vfio-user server support') option('attr', type : 'feature', value : 'auto', description: 'attr/xattr support') diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user new file mode 160000 index 0000000000..7056525da5 --- /dev/null +++ b/subprojects/libvfio-user @@ -0,0 +1 @@ +Subproject commit 7056525da5399d00831e90bed4aedb4b8442c9b2 diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker index 46398c61ee..646abcda1f 100644 --- a/tests/docker/dockerfiles/centos8.docker +++ b/tests/docker/dockerfiles/centos8.docker @@ -12,6 +12,7 @@ ENV PACKAGES \ capstone-devel \ ccache \ clang \ + cmake \ ctags \ cyrus-sasl-devel \ daxctl-devel \ @@ -32,6 +33,7 @@ ENV PACKAGES \ gtk3-devel \ hostname \ jemalloc-devel \ + json-c-devel \ libaio-devel \ libasan \ libattr-devel \ diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker index 39de63d012..ca4dff0e6b 100644 --- a/tests/docker/dockerfiles/ubuntu2004.docker +++ b/tests/docker/dockerfiles/ubuntu2004.docker @@ -6,6 +6,7 @@ ENV PACKAGES \ ca-certificates \ ccache \ clang \ + cmake \ dbus \ debianutils \ diffutils \ @@ -44,6 +45,7 @@ ENV PACKAGES \ libiscsi-dev \ libjemalloc-dev \ libjpeg-turbo8-dev \ + libjson-c-dev \ liblttng-ust-dev \ liblzo2-dev \ libncursesw5-dev \ From patchwork Wed Dec 15 15:35:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F153DC433F5 for ; Wed, 15 Dec 2021 16:14:17 +0000 (UTC) Received: from localhost ([::1]:54454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWRC-0006y9-KO for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:43:26 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43376) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKm-0006uY-68 for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:48 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:38468) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKj-000090-TC for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:47 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEWhpQ023135; Wed, 15 Dec 2021 15:36:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=7Nm+AnbH7D0QdDWIFxF9ZgOS5s4qHtaae24twTaC60o=; b=iGwNlTYwqDzSQ/j7aJh8O0hc/7a83+HLznYXXBwE+O4W+BxlDdiS3cgVHCvmwu+rL1r5 HtQuinX5TN/Mq9QoIAxmKBgUzubXf3fFT2eunRbagxaEb3qdUSE0xH9P/LBDTnQebIBO pNQaHscdc80d64m3R360rEXXmDU7QSgdsCBgx3eEvqKqYxGLPCmSQRWLwvJfVWolRM8D tx3VrFdaKOU82HPPYJ+g1+CBOCPpgqo3WEby0w2gyW04OmdZuMUXsAZcc82iRtxbzRTd qT2hs2/BiCfy7Weq5pDch/5+C2UWa8ZSQdB0l7AE7DOkIu1apbyxoQ1Y7jfCxAYbqWYA tw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3py779m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:42 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFU80J165507; Wed, 15 Dec 2021 15:36:41 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2040.outbound.protection.outlook.com [104.47.66.40]) by aserp3030.oracle.com with ESMTP id 3cyju8gt9r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:41 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FcvYR+klSMB8Juyv/NgA1cv6DiYKkT4uyyotaKI18XeqnMqNPPuy30Cv096vhEzqzfJNRuW2ZXWoeF8xvKDvVZuyvMT9KaraejfODVT2Tidb4n660yT+CO+DihdX/zAZgLRgHLWY7BgSQ3Ed7QEMs26qU0VXIHLf8HS46e0AoiejZufVDBZQdC+0A1ju7X+CsojUoj5bOc9gbn7839x/eGycpgUjo0mbhwJx11rW6tab+C5Ni6u5k987GN2Mu4O7CSWIwmeWz2LR/YSh/QGTAO+dHBU+MTe0OcShoWWPliCmc9QLagXVo2AEfcW/f8kgSdz0qg3ksnmF5lhG8CU8Rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7Nm+AnbH7D0QdDWIFxF9ZgOS5s4qHtaae24twTaC60o=; b=VVcNgTYl0xcjNAoF6K12TkXKPMniElw9S7Z4n9ZRA5IxeH0Yedh9vbCroi3N6LLGo2HZKkOSbpvAziYREHlUHcKe2qbUFBxTFgRzxlB0LeN2GFZf9v0RHu88RUQ00qyuRg7eFTbUwo3LPDq7lu9KWA2mEyrpPG3JH6di3bSUUiY4zx4vv4Mv+uM1QwZ9Gqs+ita0iK5ZMPOq6sKGoBjfhDl+P26Bdsw9q1GHSaGnqiajqRD60emxBm+jNpuIJfo+C6eDSA2S/qvAyunQ4UUehhf4qAzWggYV/XA7Wb10mor2YZL0o+Qk/7D/G46kdG6rPLfjBxvimfUWFTVZOtZW9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7Nm+AnbH7D0QdDWIFxF9ZgOS5s4qHtaae24twTaC60o=; b=AFWPxLILC6ywOWx0ospZYr88inG9E2OEU0NzgcnltwVQtoKOfuvIJXys+7A94wdGwfKtu/voF+7RpEnmQEDPIEqjFYWjVEvDaLXEu+48ZEdpq8TT6JyqZPjMMRBEzFF0+0UE2juJ8WgxQLHu50PFVR/h+lTsIFISHBDPaRMYRkc= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4094.namprd10.prod.outlook.com (2603:10b6:208:11e::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.14; Wed, 15 Dec 2021 15:35:59 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:35:59 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 04/14] vfio-user: define vfio-user-server object Date: Wed, 15 Dec 2021 10:35:28 -0500 Message-Id: <22b0d82a54d53043323bda2ae0b409fb54cbf007.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b220cca9-c8d4-48ab-1440-08d9bfe097ca X-MS-TrafficTypeDiagnostic: MN2PR10MB4094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:457; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CAE8osozZP4vlzaoJNT3PwQ9kPIwFiU8Xy+RDMXpuAOOeKG7xaJ0W/oGXcZHPmfsTVMvHbCCz1EABJSSGZL6IRp39LUd2L9aGvSnb4Aty6rMMF89SwCZ76Tm6KtIuf6hD+wtkx9P32qRhcF3xM9eA1ssJmx/feSxqt2OVYybFbezj6z+mF29aK+Uzw/GTo1yZWOil0SfEs1EK4BcLs6teb3l7Q/UqzadNIwf8doPiJZCfu9i2VIP6zB3eY/Vuks1b44p5Iu3Jd+9fevUuOAzVxhthKktzvWbnqg8adhOksPwTA1ue9GCSthKxC+HcFVv0ziLXtymt4KpAcv5kvHXhQ8fR2N3bz0obHr2TtedCOTIFam/4tSJk0QzcUOGN8D2hS9DTVLhY9nZ9Mu1xvPAy/xzvOF+zthddGKYXc9mOeL9FQtTNhYMPahP/UGea64ebnl/7QuQwogznonTG6GTni+4u0sfBlQbZ5Rn1jONCyjQujuZGMdKzKPNo/839z3NJHPr2UrdYbpA4nyp2PdCaf9SsW8O9XCnWWFLuK7GD2A+ko7BuKtOvALjbuYiK1f8XtOqOBYNLk3myu7zjFBYGLjvO2klBeGCLhXYGBrop4RfJfqaZpmHDCaeZIIRnM969EQu6W5VVIVXRp/dnZjOvDB2mj1uu6SF4Tx4eRt8PMNRSTFGNf0Kp5qRxuiNqibFE247WPGg8lYNF59WP/MmWA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(66556008)(6666004)(52116002)(4326008)(2616005)(6506007)(86362001)(8936002)(6512007)(5660300002)(6486002)(26005)(2906002)(38350700002)(6916009)(38100700002)(316002)(186003)(107886003)(66946007)(83380400001)(8676002)(66476007)(508600001)(36756003)(7416002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?o61+pX4Na4VuyNPktBtiKsh1Lr/7?= =?utf-8?q?u5ssPJMFv2S4WfV6bpTfMY8EOeiHlrXk9evcF2ZWzC9d3O9QABSV/xfyHBaKu2zGM?= =?utf-8?q?Oh+Wpi0zu2xl2mHj/5fGmxPYlq+jDQ2JyFRkUXmNIhoBXmGwP0T8vynJiiqGP8D/1?= =?utf-8?q?7/ehAd+9HBjglrvkm8u2qDnno4m615Yg1fcpgOiL8dDUJvOd3q21comgKnUg8icMx?= =?utf-8?q?war5zCrlkwEdaFMwjJflSJnsc94nEZhe5r+h8ekbVddH0aLcs5kWzDZwqyIWPf3ol?= =?utf-8?q?BvsRUqgguK+VJ+JBQ3m+co+wtPFtrsfTXdDX0qHWEhuz8TnyTW0N9SyIiqVOEtCDM?= =?utf-8?q?ul6SWNTDXmHk6LJ5OGT/rclHDCOaaqVlgu6ZXhhEY6qIvyEC1sUUnGOcGsLc7wpTl?= =?utf-8?q?uemoF99ZKdmUtYe1uELkKYh1/kE7iWfSs5PevIbkReiMoM71Qx7iAV1jgHqT8cTkG?= =?utf-8?q?e9qgdZqZfQC3btGB2GhkOll3cD/rmjvrC0DliOOf8nBtVmFD+9J48IHP+qPXybHEh?= =?utf-8?q?3wNqfJvLYJCdCWFEdmQTrOm/Nz4o8SPIjOx43RADoK3JsplJHtF9Txi1l0fJq4Eq5?= =?utf-8?q?lcRFZP/S+0Fz7EoC6m1iBn3Xq4qLpuaMSc2dk5fXWYUh7QJlch7H2wyLUUX+CS7pM?= =?utf-8?q?+lV7Lxk9VrP/0+gENcYwDpej2yDhR4Tbc/KGOaCP63q+/EduyQfl7u84tzd6Qr5XO?= =?utf-8?q?kn+JDcU+3XlcdrcrZUUCv5GCY46hznehRX4rYe9H3cxxwR4JU+imQ2zIe6SDxYMNm?= =?utf-8?q?Mmwm5X1D97wT7Ug4nzd8DfMetPsNKe0U/iyE9RuL076ANO3EXA97Hxl9EcK5hDrmV?= =?utf-8?q?YH/0vfxWAdC28zZy6tz0dCAhoVHydrqo+we5h3reLO1emLKsf44C8tMD1BFiSSzY9?= =?utf-8?q?b0IAliiuz00dfhQjixtbK6JkaXA/jdu+0l3W7F6QDfBxiuf8n80HZkR43MNEwxD18?= =?utf-8?q?8ESugnfismE+rDpWp/vCMzAEt5KhopuTwXpuICa+dvKbMOnFsl/40S/AoB+cpU6CH?= =?utf-8?q?HTIgGwxD+gSFNG2NZMzY9Cx8RVlxwn0fdUSDO3XZWqKDjRYpAIQLKI3o1vg9MTnG6?= =?utf-8?q?sef+fPEmVM5mWxHE65/Y78AL27zjJd4RsssmqlgPyzXl8DUKx+PP5ybpKEaH29AUR?= =?utf-8?q?5tkUE/FjD03K+JhYuNBB2ZjIenHhDfd8eu7VLo2gCIBbSKrPrFENQnvogDUHWSrZO?= =?utf-8?q?s1dZmNGN0tuNQ5PSzfqmBgDJAHUD2QngF5iNXBY9li4vd6sQmerKA7XufcTE5oUiW?= =?utf-8?q?7RPdKarz5jZjaX/Q14ISB8sgC1SfZCA4oKbbotkEh8YYu99uKedEonou4dodazCpw?= =?utf-8?q?jAEc1YC569KL6W0ascj5Q7od1C5ZcLujQuJGw9Niy9RjrVXdO+YpU4TxZFZavB8tg?= =?utf-8?q?pz0+evqc7hyTEr/FXCQGsddkE1UcEzssGz15egF88niKO5+kT/gvFuo5e+lb+kMci?= =?utf-8?q?ea6CVguROcdJehf8pAmKgab1ug/LSO0HX0MgWJdx56WcuIKQMg1nSgX2rBDtbRy0u?= =?utf-8?q?7yPQyanWCcqM15t7paDx6og8e9cQYOorwM95N5W4/VssNU9QR6gCNDU=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b220cca9-c8d4-48ab-1440-08d9bfe097ca X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:35:59.1693 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RKvVtZh83didiSLZrGCqrMnvsEByHDSDoo1ZcJ0XZYypRiWe9EDudXgdxAiyH46Qj4GDILkVMnpkAxui3HHvVg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4094 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: qmL5ASD4Kx8DfKm9JeaXwwzJskykAoid X-Proofpoint-GUID: qmL5ASD4Kx8DfKm9JeaXwwzJskykAoid Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define vfio-user object which is remote process server for QEMU. Setup object initialization functions and properties necessary to instantiate the object Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- qapi/qom.json | 20 ++++- hw/remote/vfio-user-obj.c | 175 ++++++++++++++++++++++++++++++++++++++ MAINTAINERS | 1 + hw/remote/meson.build | 1 + hw/remote/trace-events | 3 + 5 files changed, 198 insertions(+), 2 deletions(-) create mode 100644 hw/remote/vfio-user-obj.c diff --git a/qapi/qom.json b/qapi/qom.json index ccd1167808..6001a9b8f0 100644 --- a/qapi/qom.json +++ b/qapi/qom.json @@ -703,6 +703,20 @@ { 'struct': 'RemoteObjectProperties', 'data': { 'fd': 'str', 'devid': 'str' } } +## +# @VfioUserServerProperties: +# +# Properties for x-vfio-user-server objects. +# +# @socket: socket to be used by the libvfiouser library +# +# @device: the id of the device to be emulated at the server +# +# Since: 6.2 +## +{ 'struct': 'VfioUserServerProperties', + 'data': { 'socket': 'SocketAddress', 'device': 'str' } } + ## # @RngProperties: # @@ -837,7 +851,8 @@ 'tls-creds-psk', 'tls-creds-x509', 'tls-cipher-suites', - { 'name': 'x-remote-object', 'features': [ 'unstable' ] } + { 'name': 'x-remote-object', 'features': [ 'unstable' ] }, + { 'name': 'x-vfio-user-server', 'features': [ 'unstable' ] } ] } ## @@ -900,7 +915,8 @@ 'tls-creds-psk': 'TlsCredsPskProperties', 'tls-creds-x509': 'TlsCredsX509Properties', 'tls-cipher-suites': 'TlsCredsProperties', - 'x-remote-object': 'RemoteObjectProperties' + 'x-remote-object': 'RemoteObjectProperties', + 'x-vfio-user-server': 'VfioUserServerProperties' } } ## diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c new file mode 100644 index 0000000000..10296ef33c --- /dev/null +++ b/hw/remote/vfio-user-obj.c @@ -0,0 +1,175 @@ +/** + * QEMU vfio-user-server server object + * + * Copyright © 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL-v2, version 2 or later. + * + * See the COPYING file in the top-level directory. + * + */ + +/** + * Usage: add options: + * -machine x-remote + * -device ,id= + * -object x-vfio-user-server,id=,type=unix,path=, + * device= + * + * Note that x-vfio-user-server object must be used with x-remote machine only. + * This server could only support PCI devices for now. + * + * type - SocketAddress type - presently "unix" alone is supported. Required + * option + * + * path - named unix socket, it will be created by the server. It is + * a required option + * + * device - id of a device on the server, a required option. PCI devices + * alone are supported presently. + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "qom/object.h" +#include "qom/object_interfaces.h" +#include "qemu/error-report.h" +#include "trace.h" +#include "sysemu/runstate.h" +#include "hw/boards.h" +#include "hw/remote/machine.h" +#include "qapi/error.h" +#include "qapi/qapi-visit-sockets.h" + +#define TYPE_VFU_OBJECT "x-vfio-user-server" +OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) + +struct VfuObjectClass { + ObjectClass parent_class; + + unsigned int nr_devs; + + bool daemon; +}; + +struct VfuObject { + /* private */ + Object parent; + + SocketAddress *socket; + + char *device; + + Error *err; +}; + +static void vfu_object_set_socket(Object *obj, Visitor *v, const char *name, + void *opaque, Error **errp) +{ + VfuObject *o = VFU_OBJECT(obj); + + qapi_free_SocketAddress(o->socket); + + o->socket = NULL; + + visit_type_SocketAddress(v, name, &o->socket, errp); + + if (o->socket->type != SOCKET_ADDRESS_TYPE_UNIX) { + qapi_free_SocketAddress(o->socket); + o->socket = NULL; + error_setg(errp, "vfu: Unsupported socket type - %s", + o->socket->u.q_unix.path); + return; + } + + trace_vfu_prop("socket", o->socket->u.q_unix.path); +} + +static void vfu_object_set_device(Object *obj, const char *str, Error **errp) +{ + VfuObject *o = VFU_OBJECT(obj); + + g_free(o->device); + + o->device = g_strdup(str); + + trace_vfu_prop("device", str); +} + +static void vfu_object_init(Object *obj) +{ + VfuObjectClass *k = VFU_OBJECT_GET_CLASS(obj); + VfuObject *o = VFU_OBJECT(obj); + + k->nr_devs++; + + if (!object_dynamic_cast(OBJECT(current_machine), TYPE_REMOTE_MACHINE)) { + error_setg(&o->err, "vfu: %s only compatible with %s machine", + TYPE_VFU_OBJECT, TYPE_REMOTE_MACHINE); + return; + } +} + +static void vfu_object_finalize(Object *obj) +{ + VfuObjectClass *k = VFU_OBJECT_GET_CLASS(obj); + VfuObject *o = VFU_OBJECT(obj); + + k->nr_devs--; + + qapi_free_SocketAddress(o->socket); + + o->socket = NULL; + + g_free(o->device); + + o->device = NULL; + + if (!k->nr_devs && !k->daemon) { + qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); + } +} + +static void vfu_object_class_init(ObjectClass *klass, void *data) +{ + VfuObjectClass *k = VFU_OBJECT_CLASS(klass); + + k->nr_devs = 0; + + /* Later determine how to detect a daemon */ + k->daemon = false; + + object_class_property_add(klass, "socket", "SocketAddress", NULL, + vfu_object_set_socket, NULL, NULL); + object_class_property_set_description(klass, "socket", + "SocketAddress " + "(ex: type=unix,path=/tmp/sock). " + "Only UNIX is presently supported"); + object_class_property_add_str(klass, "device", NULL, + vfu_object_set_device); + object_class_property_set_description(klass, "device", + "device ID - only PCI devices " + "are presently supported"); +} + +static const TypeInfo vfu_object_info = { + .name = TYPE_VFU_OBJECT, + .parent = TYPE_OBJECT, + .instance_size = sizeof(VfuObject), + .instance_init = vfu_object_init, + .instance_finalize = vfu_object_finalize, + .class_size = sizeof(VfuObjectClass), + .class_init = vfu_object_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_USER_CREATABLE }, + { } + } +}; + +static void vfu_register_types(void) +{ + type_register_static(&vfu_object_info); +} + +type_init(vfu_register_types); diff --git a/MAINTAINERS b/MAINTAINERS index e0daf349ae..b5eb306662 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3465,6 +3465,7 @@ F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h F: subprojects/libvfio-user +F: hw/remote/vfio-user-obj.c EBPF: M: Jason Wang diff --git a/hw/remote/meson.build b/hw/remote/meson.build index dfea6b533b..534ac5df79 100644 --- a/hw/remote/meson.build +++ b/hw/remote/meson.build @@ -6,6 +6,7 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('message.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c')) +remote_ss.add(when: 'CONFIG_VFIO_USER_SERVER', if_true: files('vfio-user-obj.c')) remote_ss.add(when: 'CONFIG_VFIO_USER_SERVER', if_true: vfiouser) diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 0b23974f90..7da12f0d96 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -2,3 +2,6 @@ mpqemu_send_io_error(int cmd, int size, int nfds) "send command %d size %d, %d file descriptors to remote process" mpqemu_recv_io_error(int cmd, int size, int nfds) "failed to receive %d size %d, %d file descriptors to remote process" + +# vfio-user-obj.c +vfu_prop(const char *prop, const char *val) "vfu: setting %s as %s" From patchwork Wed Dec 15 15:35:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B55A3C433EF for ; Wed, 15 Dec 2021 15:50:55 +0000 (UTC) Received: from localhost ([::1]:43178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWYC-0001fB-Pa for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:50:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKC-0005Nv-8Y for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:12 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:62424) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKA-0008Th-AB for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:11 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFELE5P028543; Wed, 15 Dec 2021 15:36:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=TQdgYe05Ql0eIEvKeCcD0ED+VmMuLrl2ibuR9EfwY7Y=; b=Ykcsik3RXHMjpVdWe3vgXvVmOjoIwYh2WZ2GlZpNnTYoZrRhGIp3B+TyNA32hRJvSv/L ptNjtQsSiTvmO7GunyohWLM2SYGXZTIl9bZq2NxTE40tmSZJ1NB3BbdeNJUyxqCWs0uC 2PNC7TyiRaKO9JdFy1J9fv97J961xye2i7cgi3Mi3wvorL0xuIC4eDS+MAqLnsOyQ5z1 5pkLZeJ2piX//z8EAyQAUY3+e4x1JLAFZywF2aBi3i56aAJgyO4F/ZWGNiZXOH66bTWZ XiiTUPQsMfYKDWJ7G5CWYvnDxDY/TMVII03vtBvOPZMIxevrPVzUCB+vt84/Nn4rAGaA 2w== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3ukf6ru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:07 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUWiZ094235; Wed, 15 Dec 2021 15:36:04 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2169.outbound.protection.outlook.com [104.47.55.169]) by aserp3020.oracle.com with ESMTP id 3cxmrc0tra-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h96ISAgfobIS2vVH5w8L9Iy/d3ud8VDDyIoVXGiWS2XfyV/DPOt90TMgAzNQS3klOz62f3hXbaZlE2YnOg4OxFX8Ew4rL03gixJZhH25MdO3+g2QX4c7RYTSzg7G5BPC2IlZzqyFwzDpaPch5613xSme1JhgRQ6/4ApIyKwpZSqgqxhSQMUZCfOOGE6cssJCdRYEbaxTB1TUxDb54ZIxIdlPR4Fgr8dpIKaBm+bGwcoXMn5+TM/gSNnGNLjiqM5Z2JLl3JEVSBldIhlHAjVp9yjRZyxsjdZNeUFk2uIcaGMjBV88S7+B857joCKfkLIx/NN3iREdEEj+48N8vykUJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TQdgYe05Ql0eIEvKeCcD0ED+VmMuLrl2ibuR9EfwY7Y=; b=BKrIolNHwGvVsEGEBO6UVFKByGDvD99vnYelQJA3VUdhH/sAd/S3YyIBDJ9ydxCahPgzJwksHfyqfESbz4WKBSUuc5Lye/pBAIsky50K/rytKTcZyAk1DheNYvmaCBza99Xi11RtQKUlny7jh6fBD3ytXrw6LxnGYK0SVi33awlr9tV76XRnWa8x40VhBMWATw/koB72/L5cS4xvPiFt5EnKbEj28S5qUXS2DXfjBR92dehuVyQtwKC6ffk2xGR3DRmDPfw8l3DYmedBpNzzGgZQwpxkzvLHNl4+ZhMAuAuAU6GygOx0sqA4CYOFz9lu0EMSj24h5i4/jXzNeMpinA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TQdgYe05Ql0eIEvKeCcD0ED+VmMuLrl2ibuR9EfwY7Y=; b=Hs9VCLqBIu10e8nFea81X+QaISWZJRSm+RSzk2vPAoQ1QvwKHXXWUlMf+N1Y/3AS99DTdEnoxoOrzMiaDRJAjLvLnZbCZyhiglQUk55qH3DFBc19e5QDbcF9V5Huy2ivxFqI27R/kVru0vF/gBOueYF7H5mErK1VzwSqtMLVkLU= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:02 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:02 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 05/14] vfio-user: instantiate vfio-user context Date: Wed, 15 Dec 2021 10:35:29 -0500 Message-Id: <661e4b32da4b2ae7e0027fb483d74b96faff2cb8.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c8eb8de4-8871-4edc-5633-08d9bfe099bc X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:179; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8n6itNPSBw+Is4ifRkA5xYzICm2asoF6MqaNAF+5xI4x7KZMs3fiG1IN1wZx1psZy+cTopb8afH9WN9B02Ta/0W+lGUO/G60co2ILh/0fpvxOczRJdqQxN029y9oEGPufwaaj5V+cYYN+TKWoVBhaNMgO6Si/TTRHkXNqNFfwBC/5o9gLozrToVujsZQaWsyzAblufL5V7d76taxkH1dJi2zou2Ir4cQ57KtVdIR/F59RzSJME4mTQcQulkjaDtnq1X2hecLLTtSNqfb8YbUHTPk9GRCpGS/YMVhbr5zqMwTcLltf9Q/ZaGxOzF85sNA/MIJ3MAGuLOw9I34SEZw0q+9s+i7L62u9ITt84gbQAnd9shCrdKHBkWwp2gslBczxeXvDsQkvfxpZkod8PdGQJRJN1jwa9p4pUurhLuGJsXE8w3Mt+h+MXS86b6aEGb+wL0gVUPhRwt3AXfOzSAoWx5KC3f+7f2mOao4aosW/Ygg2yL4lvl49/+NbmfqjstBPOO+bHhGt+hMGNhcdQPR7QO8Xdl105iaW24VTWqnyFdz638YOB2YkErFIGYgm/JSqNhtOmUdzw5llNu6XGyjmEMM+VoPNffPvjMFtI1KIAio6RSOtoQ2dN0vD1CahfZZPbtPoeZN7IN1tctrabmOS4RtCvid2+KQbVZnDw4AOZxnmOtbsWsIilg8vDRHPeYdow8p+CZvlv3VwMATLi4zmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hLOsCbrjo+Kik7scwKI6Iqp3aWVwGR3MKniy6MOkYkiAdOxi6kdQxjQ+8zYOH4LnzYwWlJvt4bDI/Yt3z0EQBs0vkBaMUt7wfazl0TxN9C2rlN5FwRteZrvEjOnArlFIl9yaCzv08z2RUTFV4yK22WEkRZSBxuc6CRlOd/rRBQpFBCYUowiCrgg9HeWpG6Df5Bf40Sh9tk7dd4bFJzNr9k1sGnG9EH/yx2pb3CoPZ5iDUc9qcWKcQOlt8y3YHXUO6nrBY+X/gFmpvuK9j9wST/+HOsFdF6ClJc3+CfTtFr58FAk/F/r6ASAjoIla2A7gEBsFu2yVRzCtqAiv/gxlc6AXceiuH/w7D6QjxatRy8Xp6guB3ee7Nru9z7tHweqUTmWJxJMwzknzpumWdzJ5f7gqfa+mmpVq1pxGrn3nY2D2V2v/5zj44FX4k/EIpURyv95AgKBQQi666WFqJYUW0LADbIIg14AV7elqdgQ1b6CLLQbXTzVICsaoJYxuwRVcAPZ3Sz/2/uGhf414WTFHXkjJbtgecfYW/qrjaYTXI+z/abGVbsDsVBhhGLjQY/TS16B9Ykh1u9oqzdvJd6nEL2vAAyeED9TxHmSLnaGnTpbew5SqHgEhWFLowvd0U5Xct3fUeQH9KEB0jc8nFY0rwuNOp1QzSjZSdDxruTblGooEpImiBG+NrbZjMr5yZ0AUiMTscm4/2ETsTuqBXWZRFWJTeALOAoA/5UBn4UAWGUdiQY2zTF88FUmfV9oaKdzUnqayDNREEcjl/bpVyoTe+JmrWDvEl1OBmXYjJNqlwAKGJHX2yjy7rcCLJVsKj82ZcOFYcmYf4dPHF+AgrtT2qM1pvStJCvArocQS9PhP7CR1A1rFTH2g9LcggsHL/44UyMTNWPKqxfvwtnbkUm3Yz0jhtUyCz7aKGYPAt+EJMYTkEhy8/6Hj3V0QEuRpPEXY5Mb6GqJnARwNlpX+8jSn1JNvyv5ylmElOGV2U48JvQSSma48kZ6cwohzI9Wh3Cw2W6mvS5OTBwW5qEhGiO3b1GzSYR9iD4QKGb73UzC6oUOCQ5gi3b0kESbMFsLsdf7IDwdNzvVh1Q62VNS5Bj6nW1CD7R9mYS1tVwjDnDq2E0maNcw8q3DwRy9llWkwRjYt0y91KgXzefCrPBwOYaj4H4D6E76oM0ZDxZR5j3h5CEKythGoMBK7c7siGpspiFkh0tNgyAq22jQ/i491QuxdhZO3NzTOQiFSQm7U1JxxSOxYy+l3Km9lOw2VobktfIP0gOM4ZPY3vySYt+5y62NVpEDxHcMb/aW49hB7iq6rH27Ly/Q5zbD96dRKM8Frd40saSZAkXm/z95UPH62GuDB7ThvmGdGokmf3eeXoZGf3LCgf+yoyIj8azMmSi42WShANs0+/eA8nhV7tUqDhnTwv6bpyvLU2wlnNv8fcXn9SiczRkcShiP2Z6G5wwmfrusEFL8W/uU/Fpf1H80qVbaBJQ2Mkag7+YMk/kNclj5+/WcRq/g0gBzCC0pYrs6IJFHIWcpj7Y/wjnh8/blhM1IWtkbsRTji15t5UTPTLw4YwmhGDP3Qi1ES8PYwPO/LaaPyW6IwgEUKviKlmeYSTzQ7Aw9xMsI+tJIbbCJokKiOHho= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c8eb8de4-8871-4edc-5633-08d9bfe099bc X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:02.4190 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ngNSfazZqOdKqsCzLzjStgrUiibln+wxKj9EEYwjwFdwRL56bcbT1mZp+9nXJXvgjvqPHbqfjyJm1DO2Fu4YnQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-GUID: ozLXfZTkurh7Y-znEwn69BJJ0AKaRu5K X-Proofpoint-ORIG-GUID: ozLXfZTkurh7Y-znEwn69BJJ0AKaRu5K Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" create a context with the vfio-user library to run a PCI device Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 69 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 10296ef33c..f439b81787 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -41,6 +41,9 @@ #include "hw/remote/machine.h" #include "qapi/error.h" #include "qapi/qapi-visit-sockets.h" +#include "qemu/notify.h" +#include "sysemu/sysemu.h" +#include "libvfio-user.h" #define TYPE_VFU_OBJECT "x-vfio-user-server" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -62,8 +65,14 @@ struct VfuObject { char *device; Error *err; + + Notifier machine_done; + + vfu_ctx_t *vfu_ctx; }; +static void vfu_object_init_ctx(VfuObject *o, Error **errp); + static void vfu_object_set_socket(Object *obj, Visitor *v, const char *name, void *opaque, Error **errp) { @@ -84,6 +93,8 @@ static void vfu_object_set_socket(Object *obj, Visitor *v, const char *name, } trace_vfu_prop("socket", o->socket->u.q_unix.path); + + vfu_object_init_ctx(o, errp); } static void vfu_object_set_device(Object *obj, const char *str, Error **errp) @@ -95,6 +106,50 @@ static void vfu_object_set_device(Object *obj, const char *str, Error **errp) o->device = g_strdup(str); trace_vfu_prop("device", str); + + vfu_object_init_ctx(o, errp); +} + +/* + * TYPE_VFU_OBJECT depends on the availability of the 'socket' and 'device' + * properties. It also depends on devices instantiated in QEMU. These + * dependencies are not available during the instance_init phase of this + * object's life-cycle. As such, the server is initialized after the + * machine is setup. machine_init_done_notifier notifies TYPE_VFU_OBJECT + * when the machine is setup, and the dependencies are available. + */ +static void vfu_object_machine_done(Notifier *notifier, void *data) +{ + VfuObject *o = container_of(notifier, VfuObject, machine_done); + Error *err = NULL; + + vfu_object_init_ctx(o, &err); + + if (err) { + error_propagate(&error_abort, err); + } +} + +static void vfu_object_init_ctx(VfuObject *o, Error **errp) +{ + ERRP_GUARD(); + + if (o->vfu_ctx || !o->socket || !o->device || + !phase_check(PHASE_MACHINE_READY)) { + return; + } + + if (o->err) { + error_propagate(errp, o->err); + return; + } + + o->vfu_ctx = vfu_create_ctx(VFU_TRANS_SOCK, o->socket->u.q_unix.path, 0, + o, VFU_DEV_TYPE_PCI); + if (o->vfu_ctx == NULL) { + error_setg(errp, "vfu: Failed to create context - %s", strerror(errno)); + return; + } } static void vfu_object_init(Object *obj) @@ -104,6 +159,11 @@ static void vfu_object_init(Object *obj) k->nr_devs++; + if (!phase_check(PHASE_MACHINE_READY)) { + o->machine_done.notify = vfu_object_machine_done; + qemu_add_machine_init_done_notifier(&o->machine_done); + } + if (!object_dynamic_cast(OBJECT(current_machine), TYPE_REMOTE_MACHINE)) { error_setg(&o->err, "vfu: %s only compatible with %s machine", TYPE_VFU_OBJECT, TYPE_REMOTE_MACHINE); @@ -122,6 +182,10 @@ static void vfu_object_finalize(Object *obj) o->socket = NULL; + if (o->vfu_ctx) { + vfu_destroy_ctx(o->vfu_ctx); + } + g_free(o->device); o->device = NULL; @@ -129,6 +193,11 @@ static void vfu_object_finalize(Object *obj) if (!k->nr_devs && !k->daemon) { qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); } + + if (o->machine_done.notify) { + qemu_remove_machine_init_done_notifier(&o->machine_done); + o->machine_done.notify = NULL; + } } static void vfu_object_class_init(ObjectClass *klass, void *data) From patchwork Wed Dec 15 15:35:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74879C433EF for ; Wed, 15 Dec 2021 16:18:33 +0000 (UTC) Received: from localhost ([::1]:52008 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWbK-0007k0-Td for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:53:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42956) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKF-0005WJ-Lo for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:15 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:15518) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKD-0008U2-DI for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:15 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFERbsh005943; Wed, 15 Dec 2021 15:36:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=ojOIe1ACRQP2iGqwMnpl0nt+/HigPHu+fBoaIJ9Ag94=; b=fUrweJOf8K2E+fgKZIKjJXW7b4vRV2B0kWhmTfgfiTZ3K41I9cHu/3x2sA2K7K7iX1y0 Lv4O51ri8Grs1jGetlmg81o+vn/YBPL/RZbdmMnqEUbUl+yEmVy1W8oUYJWtXhx12EGR tyl4r4Xu2J7HH6uWaHm4kotyKmilqOb05pPn2YyhOY125ryLyV37dGL69J1f1aC2FCeP zsYEFjWBSiZ/zujtYGC8eq9cCiYcP30DDR7aUu2df5NutfQgozYQcjnB1zfWXGX6JTI4 ZvcVOXWhQ+/QGZvVTUVw4GwUGnRNJFYKNPbaOBVuLMbiZ2JxniMtbFgS7YkSEInpKUeq wQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx56u6ufn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:09 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFU8bw165468; Wed, 15 Dec 2021 15:36:08 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2168.outbound.protection.outlook.com [104.47.55.168]) by aserp3030.oracle.com with ESMTP id 3cyju8gsfw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RRKGhvMz69ofw4FfZnBbBDO8MTg9EAAa2T12byPjY24Rlkz0xE8W4rjSnggdzw1/b7fetCVqqkWfJL6Ou/bB+h7oLEBOrtKOtxaHceTS621xsdA7g/Kg3YXnCQWuNIVYApHM2BEWg10+BbprfrnY1FnkS7ki9zxHW3EjJHrXTLzNP7P15qr8kiv289kUMMqSWxXlmACJSdqIYWnlgAZGt+gBhu1BO4Yz96EXqwZYe9zrBGW8gQox9ognTRfxpVJfFWZ343/J9rQnu6WFg2fbnU+YB4v7W86NeWEDc4xDv3kocIjSDMTGDMAqt5qYRCeA54keaK174itDYVEVlIyJEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ojOIe1ACRQP2iGqwMnpl0nt+/HigPHu+fBoaIJ9Ag94=; b=DSxNCx2IvWRsIzFtSdWQhSmmUUSCEgGrRJdE4jhqdz53NHtiN26b6AxUy3ML20gLYO2coSO+xv7+WBA8s2sk6YRaq/OV3jSgzHalTv781GUsuDpvgzbr8hSFpJw6aoqZrZG6DDnG6/1T4BuuGflafuMGWhNkSjfhJja9GMUPakdLfgr+0izwz3QHmR9qzNPo1DWGCmwuacNzd6H2tuSHLEbrFw5aIm6fIcihLM49v6Ahm1hREUP0F3+o/IqYtfQ3XoTh/4WneQdB60iMvNi3eWsTMZtx4pyh4RsifBaHc0VWE0hTaj/udXUzqCmWPwyxkB26f4voHHmztxB0B2XCsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ojOIe1ACRQP2iGqwMnpl0nt+/HigPHu+fBoaIJ9Ag94=; b=zUmR3OqB8Vv2l2ORtLY6D2sUEr5faOXzqczyGI9UiRjW5v8C5jOhMWPxBkP8ATDWeA18k2F6gfekXXPvdov0cVtjvNq5T10+cf6ykBLMbqgaE4tTq6nHdkZ1IwfPnOcnECSrcCL70enCp6ULbtRpiS2rG8CyHumS+Y4iJXAOHEI= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:06 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:06 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 06/14] vfio-user: find and init PCI device Date: Wed, 15 Dec 2021 10:35:30 -0500 Message-Id: <06d8cd0be786fb6786d42c9251b37094bff813a0.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f9e651fe-2cea-48e2-0af9-08d9bfe09be3 X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:119; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 84iDTfi1l5i8+XZ9xlhMbozs8KF5Op6MI8Pj0UQP5Pku8QPN2D1YqBKmwf1OJTSb9GF0nDGarvDqusFRIPbc8YelmF/XUl9n9xCU22Aow4xKb/Z4IWUxOsOHOY3diMhQ3rKwDA973JNV+1xrfNBdlx8jKdiUfz7yYBronULm3NUCkZC8m1gruGm5njae1iKWFbgQnC1mc+qmOfQw7Rd8yr4HhBAQWbjboCThVCVXsxnFXgl0gUCH7VGMM/j0bkhbmKLohE7guRirM2W1DUtWwaAITLbypBZYlggk5+0EomnlF/6A39vUkVog9HtLpZIcKcBurtGUJshg9BpwssYrFdZDWcC15k5G6Ik1f9Csfyu9axXy6sspRa0AJnfj84GUQr5BtEksXnAVliZVknLyLhSz1ia4/Q2LNMwU+9zeZWCfBpkHwKLtE4cpP7SyGb3niEqUUSMPulqi7fMqkpWh91Q23KaF1HPZMYDyv56pWViAj9fYPfwuwMBfqfutwDJwKlUyksUOVvaMjEcOH5b1xJz9qRrsUDbMIBsyfcwKcghtLfEbI2/7fyNwMp9EJjinNMiYbBZ5MCEIm97FhJt6Ns33r6pOBwnVClOGukpdB38+AmvvAJBpBLNHNF9uqP/TPDo/qDuQ2tzYle56v2y1L98a1TyuDUZ4w2F7C6n2SUuIcG17jBiie8MoLXkJXXd+6VZE4J9uWgeJ5qPRVsKH6A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /ndbK3BlbG0EtWgTTTQbzkaUcrPSJgbUoYjVVTCM6PTM9QimYpqIVUq9/VN9F7h5Mh34IEnoCQ9oQ8e+rBfoX8JKndvMm0czbIHL/eoJk6epkZnQHrQa3F7H5H8NtbTkA/Ksqs5lgYnoZU6jIilJSueK4rKuCv9sm010qmtt40LSxlTyQlHRVBtVwHKSXn0gfpEuh+u6tp52SzdeQ1AEY0kGclc1GcK+R5nuC7v6vJkX5V8yqaCBMYrWPeqPuYBSbwQ+fNfHJIym22ehGZsoCedbrFQdnyC6EF7awgA+qLv4DHkDBWe7aFgZ2WaqfepAJ9vNZGqarSkBIBlpK83thZlMAXPSUXyNJrv9zEwSOZBsRt423x7dgPjhGVvc7PZq1iEoXTgIfBHhlr6IE5XR66xH/OsTnM15x8m4EKQfRevxMHEvjzf86GzH4Md/hqMMmaabem0XDtvUmTsDc3ai1fdXBAUma4TLnoNWj1aL1xmyiWc53hBcUi/jtzWpxbbieqVORzAFn0jmghSfNBkjJEo0Yu294mVt65H/AnDN3I17NMO2fP1NNkKwyosmNParCFUxkW9vn6h7NUvK3ez8X2gvbbQPtgXGzm7Mt8yAXnP9+wVsDuwvvfK1FGEkfebF4j6GmhCjNEql8aOOMVAGmuGeUgfVqrWNoZaDYs3UojVsZ65y4G8Mc4l1+6WGS34FnbPKtPZbblcJ8tg0CeUXlzDNww+9AftWnaI2Th9efid75jvSYiKXrFPkQ+Mr4GEVucP5Y1M6zEmgdZtTYu/sbZxPg3Rz1/CZArRifVbsS8nWVH+d+3mY9n2vX9gnqAL9YPWXEVdKNEW68IfsT9De842Jx+fVke3dZvb/mPhlAFdfZCJ/GW1tg45joSicRT7cSJkGV8r++8c4JfznVmGRw7uU/SEX75/l5zO3scSmx+y+ZRVlZJbSWFFf0RaJpLt/mmoUnk8SJ/pFjO1zK+83rSR+srwTcBk7b5bATVh5grn0DP8K9BF+kZUUOZINHvxb0SwjDHnvExmkpQ3HMTlQgXYBcfl4cwylabrQJAYYL4RQY3OgpGObjvccUeEMUTI/Q2/wPglQTguUq3HKr2tRaoPgcbUsdpdzP0awgyVXltg23TNRnQQyXfV2NyhaPw221P7rXSPifvvsKBoPlL9GVNdBoA4zXiPNkqJCAAaV/jdckMMT63P7ojXES/AnV5r5QPbMekeHOkz0NRbGPsLTqq853giGMAWhlcCnjFcX667T+rFXwlWLk5R8gSyH73An9ak5V2ijppTy5kXP1wI7eQ7I4eJkTVlAYcqkr3HeSsIYPeaFYiwzA1cBpZlPIA8bGZZkOY+euGkBc3Q/mKL8lTGVZwAhWsJarMNIWUCKxTM7ixrqyutRcI0Ez0C8RuI7TtViJKRKe75q5IiX9RurRhhLPJCmInbK4ym/DXgnJwlQI9HQsDxTVlk/xXuz7hlqsMLPdnRnFT/xqA2r7Xy8YJkqAv8h40WMALrRPdj/iR7HhUNcKIMSsjys1SCH511sDV4cp//fI6p44F3wr0/MgPVqDBw8tBaWtQ8ox0vfiUV6rbLkqppyqiIHW+Z2EGuMWAiDxBIEIY7eu/nkJVDZxX2+rBnKEOXi6F1be0iWFyE= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f9e651fe-2cea-48e2-0af9-08d9bfe09be3 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:06.0125 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8Ab3IWb1iZeGviz282XG4Bl4lJmAWVU2fGYUps2hyejwC4SRQQKVdewcTGQ14fwMxlmh+6nmlAm/Hd0ljGwgFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: 7b_Cc1-4Gwle-140qYnu213__DXteqc9 X-Proofpoint-GUID: 7b_Cc1-4Gwle-140qYnu213__DXteqc9 Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Find the PCI device with specified id. Initialize the device context with the QEMU PCI device Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index f439b81787..bcbea59bf1 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -44,6 +44,8 @@ #include "qemu/notify.h" #include "sysemu/sysemu.h" #include "libvfio-user.h" +#include "hw/qdev-core.h" +#include "hw/pci/pci.h" #define TYPE_VFU_OBJECT "x-vfio-user-server" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -69,6 +71,8 @@ struct VfuObject { Notifier machine_done; vfu_ctx_t *vfu_ctx; + + PCIDevice *pci_dev; }; static void vfu_object_init_ctx(VfuObject *o, Error **errp); @@ -133,6 +137,9 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) static void vfu_object_init_ctx(VfuObject *o, Error **errp) { ERRP_GUARD(); + DeviceState *dev = NULL; + vfu_pci_type_t pci_type = VFU_PCI_TYPE_CONVENTIONAL; + int ret; if (o->vfu_ctx || !o->socket || !o->device || !phase_check(PHASE_MACHINE_READY)) { @@ -150,6 +157,38 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) error_setg(errp, "vfu: Failed to create context - %s", strerror(errno)); return; } + + dev = qdev_find_recursive(sysbus_get_default(), o->device); + if (dev == NULL) { + error_setg(errp, "vfu: Device %s not found", o->device); + goto fail; + } + + if (!object_dynamic_cast(OBJECT(dev), TYPE_PCI_DEVICE)) { + error_setg(errp, "vfu: %s not a PCI device", o->device); + goto fail; + } + + o->pci_dev = PCI_DEVICE(dev); + + if (pci_is_express(o->pci_dev)) { + pci_type = VFU_PCI_TYPE_EXPRESS; + } + + ret = vfu_pci_init(o->vfu_ctx, pci_type, PCI_HEADER_TYPE_NORMAL, 0); + if (ret < 0) { + error_setg(errp, + "vfu: Failed to attach PCI device %s to context - %s", + o->device, strerror(errno)); + goto fail; + } + + return; + +fail: + vfu_destroy_ctx(o->vfu_ctx); + o->vfu_ctx = NULL; + o->pci_dev = NULL; } static void vfu_object_init(Object *obj) @@ -190,6 +229,8 @@ static void vfu_object_finalize(Object *obj) o->device = NULL; + o->pci_dev = NULL; + if (!k->nr_devs && !k->daemon) { qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); } From patchwork Wed Dec 15 15:35:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C71DC433F5 for ; Wed, 15 Dec 2021 16:02:37 +0000 (UTC) Received: from localhost ([::1]:53180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWQi-00064W-Er for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:42:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43008) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKI-0005d4-Rt for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:18 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:23970) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKF-0008UK-PI for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:17 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEWeMk023139; Wed, 15 Dec 2021 15:36:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=YGx4Go3VGehYmW1KalcuxWiAmbNP9Ygla7Mfg/QIVxc=; b=1N//bwECAjLJ5UCGeGf7Ack0SW6fhQDSgy1hUVjvYWCiP4ORw+ytz0eKlpf+g8BWsFqL a9fS07PAl2jJAUvQ8lwASg1ckkAbBGw99bdTVKTqIJ7J73BE/P9eUCF3FBA9zYozgxit 3VH95yrIRyR5wsUl+i8q1AQxBW2XPK1u3uTVo7rDiutgeJRbuvtffSVvqyEzlb8LbPyO NPnBVaOZA0szst7hbOSdIQX1MDuKQ2VbFZcg0591FnycI9Rh0H6R4HfVYfuo7IeE05Dg zFz3rT3jNHfML8ce5gP75mKU+FYgIAH17NaICKsm0YZ5cPJqXKmgWKl5epwsO/unjueQ 3g== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3py7782-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:12 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUXsp094439; Wed, 15 Dec 2021 15:36:11 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2169.outbound.protection.outlook.com [104.47.55.169]) by aserp3020.oracle.com with ESMTP id 3cxmrc0tyb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:11 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h2/yvLnVoyFn44YQio8DTVRgpaoQZIP2aY9G/8uawWb+I+kw8CrGsJjVFpQQ/L5m0YIfdwDyajg3KGeiaKJjvm+0ZNb5T3QJGJB3qhiKTrqhZs40Fg8KAcPrMrSnDYTPUWmVzes6dT9d7BUGxRKxNy8MyHGuOtMOUz8rXngwXsQD+qFr5zIAYpypX/lM2VyKa/2e9VL1x9HF1k6KyIlszFjzB9tISkBvDtmRkqSYmWgMcZtOxphfqBxoXW+rJzdYQ0cNN7f7zQrhDgtunBouF+MqY2jMcUzvA80iO6TZQGi3AlE1QF4gxcBEgCyMIhea4n3Yj1+RVI9MRemoxNxsqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YGx4Go3VGehYmW1KalcuxWiAmbNP9Ygla7Mfg/QIVxc=; b=DZ5YHld6iiR7KEdAaoeg12WnVLSCOQE+ODsws6hvqoUoMknHYlQ9N+iNsiGp0dvm4Dpw0rBweK+X2cELYXBzMaHQ15x0fPjRbJxfycQdoupHAgLc6msg+pO69RC3H1DwvG8/BDgTV4eb7TRTDHh3ywqYceXi4H6qk/eURrlz9a0WLt68cuj8UPU8xGebZWAcGbrPVLwX3UUst/Sxqxvww0kf9M9XorcCr9wQTGPWXYkqGtvf5hxtyz2GLR18+el6pd431xcSBOOwQMLubEqUVvYYYxSjw2JJ/wBGtatrL7bSy1cK8rv4hhtDhYwYnRrEqehUV5SbiHVqfMQMG9vmCg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YGx4Go3VGehYmW1KalcuxWiAmbNP9Ygla7Mfg/QIVxc=; b=kcdWvWiHIXzZqrzq8XM/LH+kEQmRVYIoMr32xzhscLFdPItPRCpVCCSbOO1T4GRdz3+GPb0Y5xAbkdVJH0a71oW2y6719DBmbvQmBzEvKgc3NQlcwzobIrDsE7g7C02lvLTYtjPop34rqyLm+CWny6h070NO03A6r0W4H0tayu8= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:09 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:09 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 07/14] vfio-user: run vfio-user context Date: Wed, 15 Dec 2021 10:35:31 -0500 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4b58e9bb-9c40-4b7e-d62e-08d9bfe09dde X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:428; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MKuB80HYrfSXFtguxJZMNTnoj4CocnPyKrLJR2Kohq/3cr7lomWkhPiw9xQev97xySh6MjBThXwy4VNvxxSKL67LAd9gHjufQtXXqS6kgHKyTO/tJlNZQBrhpaI3H7HQoSD47ycyaYqXN4fY+dMcoWbmwXqIFbsyJ0AILckFLLVcLdnRsGoWzfIPtEygo5ZJyxqlzAcdXMm7rYU9FycE+xTyI0Wnq3shh6x2FFiRbP9PQ/xLg4prNTXeAZ1sE4wCJcrnSquBYs1iU2CbySNrHX2W9SwdYJnEu/sL7+SYge9C09ZC5WrCNyTPQO6FOwgkNFd3JCSQAhXUxMPh4OsQlj+TIG3CTXM+C6Aw16g35cGFw+OCyDEugjYo1TBsJx6e7S/WOF/7puLPZogPjezmnltrRzZronuaDf2qTQu+KTn27V+e6vdcK31dXsZuNSaf6d/ej5EWHGQ61lgb1aIEwfobPKX0dU86svkG9VItOMZbs/dp+myIcvrfLDpLqeZUD1fxLW7vr6fbspnqw+NLphSUq53U8ElVR0zsJ88OCrf5PJqb7VABy8qSg4WAALkXgN1JFcXnY8J8UA/RcdVzJ7hxdtIUSVTUgTOJ5gUXtYHaHZMcwNG7EQcKUM95wmrmCgGF2qyX6k5x+Jbm1FgPEppdppC1QXEYrl9H/vsWetPrnc/VGDr2552snQHLQ9wXOBwl6P981ll5NPhkDGR4pg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: vsGQOgaKt2SEvNkDmCMUTeOpDjw4cBJn9n0YNF9APJaVkjrRFKVjfwAKatncB1Hcf67jW6SM8jlszwwN/LXs+iqpqYUymiV4E7Ag8TIvSYrX8RkJOYOH1RBAHa2IH/2TDygKQbmqUc6zRSmw+UNwzNGvSsc49pQ6inLGXVFsVvijfpqbxWmWPomqLuVt4l0ZJZ8wK0ep34tQsBk0DmuPQxUEwXlmLvGWSCU3X/7C+vqRSbyauzsF1kXGA6iwpp+8nWCNrEb8nubkp2b6HmKQ6lpeGaAjMpGxR4Usf1hwTXv6mOOCuit2bnVjNmQGOVf99h/YH4ALwYm5AtzTn6dIJGpmaqiw7X9MUGpAwFaYdilXpOJwJn5vGRtQbxuKpv0lkPp8cDLGBaa14U8+PvTPsF5pSW3LgLJGTwNpCgW5BvtCe15WDXU0Oh50iWplSefF8w0sJE14tWn+6VO/YUK3nTGhfdTaou7skRmRPL9o0vaXAYWUcLr0PH9gdLLJa1me1j09pnWmtqBo4zm6PKHoJvcvS8BkoEKaAh4of73GdsSm+V03EyycbtLes/sg+E7CqTXGkiXA3DOZJgzOPa46wnuMm1oXKKwsVH2+llPBY2omf4CwlgkirXzj7cfc+ASOCh/1ao9D7AUzp2yIZuGP1EVs+aukjQdEqPwEgBxZbKUmDNKtdPQ6ecrRCElFhlOdcpXkA6vFqZZl+O+NPZLa2HMDMwbSMIrbmZiMFOEvZoY7Gcw7D1MMtoMnGeSqNL6nwkulsK/0v+AX1TPNASWmkYaPv7fQgAT88PeaqPLMEYfTh4rDs3fnfbk8hSHYHcv1mDnO/h3aouHuOWv742NVuyhFFTsD7Qo6upMy3PJbzoM02zVTmfAUKvSzkoYbb96zE/Z8R20cRdfkagk1Wz/3YrihyKlrtZNkVqjUoOin/nqaS3O1iLDCV4Qfbi20VlJ6r2EIF8aVxkO6zbNUTHMaefSyvooQ8lmXWqDIhNrn0rf/qOlsOu/tLFJmEu8K2pN78JLlGIfjvPQSi7wWdzCuUhNwDxDsyT7NjG9gybKuWaFxdE8kwa3n/56+FVuHW8rRTyfxBAHnGycyfIj3xUuFqeKhVqN+QR3PRwDwO2KHFLp6Kud1qCkdK+FzlZfhEjdAKUAGwF44m6Zj/n57FXVUtWch92N0lSKGRlDEarmwv/ZamLXcwjp4C46Aba/ViRHK+MpB0mU4/Idf8OmBvi5ymNK3ylG0phTOpxEXCuaxZkRO6jESCJiBK+96JQYWHRf2WXaZraGVg9Et4cRoPhrRJbwUwOYL6x7+PL4BFg0Dyao5NcwV3jv9xtWlcfBpIXRhylPnUvj1EfloGn06WlspaxXDy0ZW2dV6rLIaIY3HFX7gqPiDk+j2GwwRzB/+qf5aSyYOkicLQICD1m2dZYJTGJGsWBq4njsZQ3GaIH3Zwtz7W7sS7gchimRp3QiiX0T5XWtXSiRN27PYsA/C0fOgTqMwQ9UcLBqHmMiE6+T7RCoUbOdpJtXikik31tbFOoUv7FFCplXDmQUYj8eI6yXUom8h4kXiPXbkyzuDVY21ZaBTTJfgHclVec0nK29sb0VTseqF2r7NKAkpvmUzLjihHIgjhsWdZ8D7CQv3J5ks8Zk= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b58e9bb-9c40-4b7e-d62e-08d9bfe09dde X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:09.3716 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fzLcLeP3TpidvzCUyWgc9Oct+EHsBrJr94SnpSKC8nLTtI6IplyVQwTMG/xcYC37lhmQSnHbLeonji+8hR7+Ug== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: sGKuAL_-AK43UTZLYkTN839lOJNiBTwk X-Proofpoint-GUID: sGKuAL_-AK43UTZLYkTN839lOJNiBTwk Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Setup a handler to run vfio-user context. The context is driven by messages to the file descriptor associated with it - get the fd for the context and hook up the handler with it Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 80 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 1 deletion(-) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index bcbea59bf1..a01a0ad185 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -42,10 +42,12 @@ #include "qapi/error.h" #include "qapi/qapi-visit-sockets.h" #include "qemu/notify.h" +#include "qemu/thread.h" #include "sysemu/sysemu.h" #include "libvfio-user.h" #include "hw/qdev-core.h" #include "hw/pci/pci.h" +#include "qemu/timer.h" #define TYPE_VFU_OBJECT "x-vfio-user-server" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -73,6 +75,8 @@ struct VfuObject { vfu_ctx_t *vfu_ctx; PCIDevice *pci_dev; + + int vfu_poll_fd; }; static void vfu_object_init_ctx(VfuObject *o, Error **errp); @@ -114,6 +118,62 @@ static void vfu_object_set_device(Object *obj, const char *str, Error **errp) vfu_object_init_ctx(o, errp); } +static void vfu_object_ctx_run(void *opaque) +{ + VfuObject *o = opaque; + int ret = -1; + + while (ret != 0) { + ret = vfu_run_ctx(o->vfu_ctx); + if (ret < 0) { + if (errno == EINTR) { + continue; + } else if (errno == ENOTCONN) { + qemu_set_fd_handler(o->vfu_poll_fd, NULL, NULL, NULL); + o->vfu_poll_fd = -1; + object_unparent(OBJECT(o)); + break; + } else { + error_setg(&error_abort, "vfu: Failed to run device %s - %s", + o->device, strerror(errno)); + break; + } + } + } +} + +static void vfu_object_attach_ctx(void *opaque) +{ + VfuObject *o = opaque; + GPollFD pfds[1]; + int ret; + + qemu_set_fd_handler(o->vfu_poll_fd, NULL, NULL, NULL); + + pfds[0].fd = o->vfu_poll_fd; + pfds[0].events = G_IO_IN | G_IO_HUP | G_IO_ERR; + +retry_attach: + ret = vfu_attach_ctx(o->vfu_ctx); + if (ret < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) { + qemu_poll_ns(pfds, 1, 500 * (int64_t)SCALE_MS); + goto retry_attach; + } else if (ret < 0) { + error_setg(&error_abort, + "vfu: Failed to attach device %s to context - %s", + o->device, strerror(errno)); + return; + } + + o->vfu_poll_fd = vfu_get_poll_fd(o->vfu_ctx); + if (o->vfu_poll_fd < 0) { + error_setg(&error_abort, "vfu: Failed to get poll fd %s", o->device); + return; + } + + qemu_set_fd_handler(o->vfu_poll_fd, vfu_object_ctx_run, NULL, o); +} + /* * TYPE_VFU_OBJECT depends on the availability of the 'socket' and 'device' * properties. It also depends on devices instantiated in QEMU. These @@ -151,7 +211,8 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) return; } - o->vfu_ctx = vfu_create_ctx(VFU_TRANS_SOCK, o->socket->u.q_unix.path, 0, + o->vfu_ctx = vfu_create_ctx(VFU_TRANS_SOCK, o->socket->u.q_unix.path, + LIBVFIO_USER_FLAG_ATTACH_NB, o, VFU_DEV_TYPE_PCI); if (o->vfu_ctx == NULL) { error_setg(errp, "vfu: Failed to create context - %s", strerror(errno)); @@ -183,6 +244,21 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) goto fail; } + ret = vfu_realize_ctx(o->vfu_ctx); + if (ret < 0) { + error_setg(errp, "vfu: Failed to realize device %s- %s", + o->device, strerror(errno)); + goto fail; + } + + o->vfu_poll_fd = vfu_get_poll_fd(o->vfu_ctx); + if (o->vfu_poll_fd < 0) { + error_setg(errp, "vfu: Failed to get poll fd %s", o->device); + goto fail; + } + + qemu_set_fd_handler(o->vfu_poll_fd, vfu_object_attach_ctx, NULL, o); + return; fail: @@ -208,6 +284,8 @@ static void vfu_object_init(Object *obj) TYPE_VFU_OBJECT, TYPE_REMOTE_MACHINE); return; } + + o->vfu_poll_fd = -1; } static void vfu_object_finalize(Object *obj) From patchwork Wed Dec 15 15:35:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 76706C4332F for ; Wed, 15 Dec 2021 15:57:53 +0000 (UTC) Received: from localhost ([::1]:35270 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWel-00074m-3P for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:57:27 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKP-0005uL-C1 for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:25 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:33144) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKJ-0008V7-8F for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:25 -0500 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFE7ijs009518; Wed, 15 Dec 2021 15:36:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=YkEOGLe3egtN92CF62I+zOI5JsngT2NI8O4xQCSgSvI=; b=f8sqBCBXQp7fzlHFHUYq5/CqBNcOU2c9p8TXx4/D8EPp1qnyM/Gd2Lm8VO4uBmTyi+Mp /HtsvDXo2ffqoG0ThKUHS/uwjPu3S4VrJXJtHkpQjAYee6Gyq3JJErAzJ5P5D3nBalpd +e257YfoPokwjW0BQhGU1MMOumAzAlGZNmzbV+riqtBHMERskdLmHUvd65NOK4EsTj/Z bwj/Jc/ehqy1gblMoQY1kId7pmjvRrR4DWtvLhVDZESk+05yUHrYYNiokcodOaebDGj2 jKFwRsVSi1ENTWYclsGzXrtFc8uRHx0MoTYfkZ4CPdb1SCqCxe/316v4H61vcBpoLXAH Vg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3mryb52-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:16 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFU7Gq165372; Wed, 15 Dec 2021 15:36:14 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2169.outbound.protection.outlook.com [104.47.55.169]) by aserp3030.oracle.com with ESMTP id 3cyju8gsnk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aBAFd1aqlSnM8wmIGvSXVlIvCRsrssBFDuuftrFQffvSrj9m7gtqjh5Dky7rtLIOgU1w/R6ONnVfO5GgQEP117OgrC2WUTvS962C61TjHWpxwCC9c+XatF0CV4AkTN0HQdHXP4WK6BuWidsEOmAx7GTXE83JJOd0vbBYzs9Gp8hAky59EoD+ZQvu4Tn760dePLz6nnF6ovytv1vionocp6MSpojKGqaEs9OOcliSv/k8thcEPGAyFhC7qZ6eoa2u8hAizYNmV/Cu5DP9G5iPwT7uMahYoUX9ghQRCtXqCveu02diJhWodh2uPiaivfF2mHqwu1RTzlewtQ58mFRKGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YkEOGLe3egtN92CF62I+zOI5JsngT2NI8O4xQCSgSvI=; b=Q1wtX75Q7R5D6vpz7gMJ807KPI7H4LbUW7BidYFNsG0Bht2FNzxbHI0y6dc1m+T6V5vbjCJ8Eu1KZ+iymwBZdjGH9FyjNHm9BojkLKT8266ShiPIl978fWhFRZvrVSP/g8h9uLzf0V//VIDqqGS5x5AedKLRjhbX4ro6nHu/JfrXamvPIjtwGX57n1C1X3VQNOpVZW33BKeJJPJFFVxxNiGIIAPM/ji1SrE/XnwQ8IElkQyXzB5XpPArv5zh2WAx4ei7PGs5uLepDMmR0emIz4Y2x2Kxg9hdB/1KfIZGWi2ezk4dnyipsWBWpOlVlgI2e19TyY43ycqTTNrxky0hJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YkEOGLe3egtN92CF62I+zOI5JsngT2NI8O4xQCSgSvI=; b=Z0ho3X0VvgEoSUdL1T2YIjFSs5SgywlGnprgWRivLbqn56lycACUelWwFOB4rHzQGOogxPnZsF28IC9NkCUbEVcRoTSQAtgmeTlcV9aBLNn7er46+51hORndHkKO/nn/zbQuK6W4Nz8h3C1o/kvcxeGZwbfNe4KvING88mD7PdI= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:12 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:12 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 08/14] vfio-user: handle PCI config space accesses Date: Wed, 15 Dec 2021 10:35:32 -0500 Message-Id: <34e287d0a8d585f104bdd06681b32fc93e8746c7.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a5511c36-4138-4d6c-dcd6-08d9bfe09fd5 X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:409; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bxGc9LQEwGVZUwOvrsWF5JBwJBvtoGDifbGbHRu5hJUHbIYzje2RLNm28KZ5eSoK2rOtXbG1HrDpfcG7CstDz+ftKk6Z3lvj3RNPqdHNyWiFGmv+O9OutSq07D8N9qxBNQLJMW+wDFjrRojfZqMWXd6/phe7GtEKQ2psxNSWZ01nEjBpOWhntTvn7hf2ahlDtgEXhBZKZo/kBN6e/QIQTqOM+35d9KUnT5cKZM4ZCIkw4vstCYEhxfjBFzBhc+uQAuaGw4jGsXD1XmEplGIzJJ6M/JVNtC5pQFFrbVWsBUpLoPnRpsGy8412dBtd3BO9mOPU9w3iggXq5jx28bwgjJ5urS0mYEeqo9ezRlLfUKm7AWIFVRLgB7gIm42i6CBAiLR7g/9tylOq2kFAeKQqpaGAvXNxPUYa26NvA8yBOQtbWMj0cXnWuR9+gSHFjUYbS5dUgMIcXwe9yC+2O10HcGor8s+aWybvcQwBueQ1BUKW9n1wDGZg0s3eG+lfLqxoNvCbupsVSgJy6bAib0FmuwSv4n8cUSD1g7fOOjmMqd60snTkpRkBsVFADgmxnGrSK/dBOQsNHz55Ps6J5trmJej0fWPtJsuB95UjrhV8CS4/2OGjXcwZT7M9VdX2Q6ExhBWEQIJ63ihC8q47yp1bohZptVXaiHY8rOpSqelD4q7H4Xz/2ZP90O+cxImy7gPPAx4gMcdk7MFiGSlRWz6Sqw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: OqIvieuSG7D9Zq1VyNxtcIQvs7kpIPX6HX0XXSm31TT+2icphvbKuK/EyGRvwWCz7pEFpnSB7UOCzf0pCP93kixd8Z45AYPuoOS+e7wWrgh8yyPBr1ndE6EpfhZdbZYBmZ3esRfoIXYmYXkkupXh3dSxtXhsZi9gBXt1JxOHfSgS6lRazjuGzKJ+L8OrJMAWmfJPrQhDF3+yWtMr3OcGxg9zq3L/WIGhR/UuX2vFT0aQBNpTuX72lzzQGkkFAsCVH4j8TMyy3ftUmJxrL6PdBBCC+iBwTPVI0hMDrlYTDVlI4Toh1Rb5/ZcHeQx6Mg3F+yGp63AdgoeEpRNSdDdXSYhOrHi1Q93plDFdu7BzbehSy7eEAzYN1LwKLlDQgHDM4sX/NY2sJdlxhfTX++kaLFmvlP51jkQLgo+mIn5m/msAMdMz+1iSE5dEKmd1VDmSHyOzChec4/mKz9JgR+s48HqyV6Hl7QVe08GpzkDoTATiYFdnUWuZHEjnzBin2/Zq71+bA0SmVO7XRqwTIoW4UaUGK/yOs4cYz6mt1PCC5r1I7V9g4nvT5kNcsQaddCxI+M8ku7pp30dKYYi7e5PTP4CeIDbPL7eL6UMFSM7bV2vvAdLPdUxoCgiXg7SZjY6MM4bm7nIXzAwRVxxwXC4uU+tvxILViR6d0hbclbZvQF8tppwx2Tv3LT+pF3k75GJu18YlC5oqwiR+ItA+pLhZ0SlKhUwXI5A5+vP58TTe/G3jDg6mhtax1Zv8GFVX9Yn+Y5Y28DGOiBNyhvExKbWyX+QXq/VEjCP6C4P4nwoY/4OSu2iheBsi2C6JyXWVZUuzd8kHdzap1ScCOVgoDCSo0oPs370QZkoq+wei3eAcLcJWwdLmh78UEevwTddKbAU1YRqYtlll81+foG4S2Ok0X3XWQRKZzmn1scqb5WmjzJBCNPLgqtMifEC90XrGEJ5bHlxbLPHtRZU6PYi5k8f6n0anilMTMIMwnBtc1aJMZFT3RsUXXV0EbbU+j+IVwqJZBW9niVe15nAIsTHkvMy25ixEmy49aAhnumPn+ruwvthpR4ClY7hlY/bokF6CR8452qYno0MyDw7VhNLVEMsO3NXJAX/KdlLMLuMENa0QYKEslgRz4JvS9Py9CkTWJIsfm5tbPJt1jGxgqENtB01/ntbajD6hD/S8tDtCCGvzAaQHZsRjHD5FUwi5/PD5hdIoRt5Ln8po2nuRHsq3BMqrlSVEw63uTcFT64R6HBctWXNjNNQ1hHbp6UZNUK6upjDcd7QzVeRx6ZxPlw7MYoPHRF7OcD7U5oN1WpkbWGiO03jj2esM85/YHo0NSmHXhQmTZeUJlCwHifWZEmdoZq6oMuM8xlYUwC9baZOjoih4nXqH/tUT2ouDGKqKRRaVjG0VgOsaZz6vnq8h37sY4ZvPLx3MQTiDnJeqrWwP7am1jtUqJE6bzIdzYIs0qp0WrOD85Fa+YvQVhb4E2Dn921Tpr4XkzEIOViexiQ7zmwcP3ky0rIYJJz+igwM3ytUf2AgkLhm/nrjYggdE5BrurBlf+RVAv8T6VRbu68jSDgpSgBPNT81AjLGiD0feanUbeTfDqs+WAAl83u4+5F1XxB3RPdfCF8KCf5pM8aPWqKcsl2I= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a5511c36-4138-4d6c-dcd6-08d9bfe09fd5 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:12.7464 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: j0dKQqLIgOpfIFPH0tYpYjKInNiwl1oYVFdEv8IXRH8DR42K1KjvHFL4NTn8v/Rw1+/ek9yjunxSyQkHZUSzyQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: nDDznOFjaMA-t07IsESSfL4Dp_wjE7wQ X-Proofpoint-GUID: nDDznOFjaMA-t07IsESSfL4Dp_wjE7wQ Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define and register handlers for PCI config space accesses Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 45 +++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 2 ++ 2 files changed, 47 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index a01a0ad185..c6d0c675b7 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -43,6 +43,7 @@ #include "qapi/qapi-visit-sockets.h" #include "qemu/notify.h" #include "qemu/thread.h" +#include "qemu/main-loop.h" #include "sysemu/sysemu.h" #include "libvfio-user.h" #include "hw/qdev-core.h" @@ -174,6 +175,39 @@ retry_attach: qemu_set_fd_handler(o->vfu_poll_fd, vfu_object_ctx_run, NULL, o); } +static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf, + size_t count, loff_t offset, + const bool is_write) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + uint32_t pci_access_width = sizeof(uint32_t); + size_t bytes = count; + uint32_t val = 0; + char *ptr = buf; + int len; + + while (bytes > 0) { + len = (bytes > pci_access_width) ? pci_access_width : bytes; + if (is_write) { + memcpy(&val, ptr, len); + pci_host_config_write_common(o->pci_dev, offset, + pci_config_size(o->pci_dev), + val, len); + trace_vfu_cfg_write(offset, val); + } else { + val = pci_host_config_read_common(o->pci_dev, offset, + pci_config_size(o->pci_dev), len); + memcpy(ptr, &val, len); + trace_vfu_cfg_read(offset, val); + } + offset += len; + ptr += len; + bytes -= len; + } + + return count; +} + /* * TYPE_VFU_OBJECT depends on the availability of the 'socket' and 'device' * properties. It also depends on devices instantiated in QEMU. These @@ -244,6 +278,17 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) goto fail; } + ret = vfu_setup_region(o->vfu_ctx, VFU_PCI_DEV_CFG_REGION_IDX, + pci_config_size(o->pci_dev), &vfu_object_cfg_access, + VFU_REGION_FLAG_RW | VFU_REGION_FLAG_ALWAYS_CB, + NULL, 0, -1, 0); + if (ret < 0) { + error_setg(errp, + "vfu: Failed to setup config space handlers for %s- %s", + o->device, strerror(errno)); + goto fail; + } + ret = vfu_realize_ctx(o->vfu_ctx); if (ret < 0) { error_setg(errp, "vfu: Failed to realize device %s- %s", diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 7da12f0d96..2ef7884346 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -5,3 +5,5 @@ mpqemu_recv_io_error(int cmd, int size, int nfds) "failed to receive %d size %d, # vfio-user-obj.c vfu_prop(const char *prop, const char *val) "vfu: setting %s as %s" +vfu_cfg_read(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u -> 0x%x" +vfu_cfg_write(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u <- 0x%x" From patchwork Wed Dec 15 15:35:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C9191C433EF for ; Wed, 15 Dec 2021 16:26:00 +0000 (UTC) Received: from localhost ([::1]:45090 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWN9-0000YA-MW for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:39:15 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKP-0005tK-R8 for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:25 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:25714) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKK-0008VQ-Vx for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:24 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEKA5H010344; Wed, 15 Dec 2021 15:36:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=Nsg55cWonSTJ2WAnYrWfyzuLAoWnpcBTa9+mC+yqQio=; b=k3wmgMlaDtl4nnNQR3h3YfSOcxsn2uZhkuVklffG9ZlRcuU4IOP7+EgaVRh16Md8fizV JPJUlcnBDLLSzWLuYL5VrlLDNo2H6fglLHFQCBmNFeb7gjrgH/v5tVhxPNeL23jcfiIu Y4laUdStn3lJlDhZAfKtFV8f2EiKFPygYsyl48R2JSDqcewbp5xa0Izh2DYSVrwtAB0H l2994LjD6BweX4AzCIq1JzNHVnUd+l6StbD9Pm6EqRIjRgOW9S4lA+3m5/S+53zS8qn+ oi3NSlD2rlLDZXzwT79s1tkmrQLfYnKiTNJQVZ24lAlEWh0Y6/ElLkcpL0q2rMpVwj0Y /g== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx5akey0m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:19 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFViMA127851; Wed, 15 Dec 2021 15:36:18 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2176.outbound.protection.outlook.com [104.47.55.176]) by userp3020.oracle.com with ESMTP id 3cvnes2d2m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RkVey0v2k6/wEFPiw27eFXwrVQfI260DgnBWwQZSjJtY5i9qRLelJ2evsJvcJMG2FoUy0LzEXyMjMNFtIxhluZlVXeFIaJrd/UM0AGkeDCBXDxLubxYXAwnLzlOugEJ7kuVE5EJFRruiVP0FG8vq15V5ohosXFH6KoC7ZivYhtd3See8AX79OUz/FDx6GnD74E+lIrfTr5+2dGzPQocw63edlzJhCbEjbltg+cocTOFsqDvtVNNuMOEOjLmRwTW4U/1P/bFfosQSKcJ4lXz8ZkqA29Zo1toTIsPbLUwfGm9FGC2zxNe40DAoIpUTH0fV0qsXAMdpvLerBFDh61lKCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Nsg55cWonSTJ2WAnYrWfyzuLAoWnpcBTa9+mC+yqQio=; b=B3zbf/ccnPrukFyVg4usNtR0b00k4fb3obc2dmU0kavfoanN5gDasP2dykwuLJyQjRY4PztF7/vylBgy6ZI9CtJtNXIVJN01PqhGkaGNyTp5HUXK6UzDo3nZn4POfziDpZfaUzBp07oiwuFqu+wmy2DbDis8FH2mUf8crl4L0Ckp05sHtGFZv+20eTTDwvDoAAHwHmMTZb/VVvSAKWV2wiRJp7NKSX4F8xcmXvzQMN7T+C4u1M34VmJ5bAbHxRZu9w+/q6dTO02LKcbPImd1m/Cl2KbvbWgPqG5gc3yMRXktQLvGEwoTsmlLEnFNb4T6McxD20w8qrQMCXpmSvt14w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Nsg55cWonSTJ2WAnYrWfyzuLAoWnpcBTa9+mC+yqQio=; b=DMsGro52BlCP7eLJfnH9EBrX9DIAZQ0fGMr1PycJ3WxqbLy6qPj+nLMfII483U3OVqUG8N4zc+bpCjBa2e1EVvPbxURZwAgeIFNn/pmx2+PckaqMqt2IVq50KTBu9W7EGOuMrL2rJFLFRM6AGJsKIo9+ubpsKyIe5Qe1KLbr0uU= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:16 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:16 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 09/14] vfio-user: handle DMA mappings Date: Wed, 15 Dec 2021 10:35:33 -0500 Message-Id: <2c9baf82a342cfc4ff3d35e017908b9050faf409.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 390eefac-8010-4142-1ce7-08d9bfe0a1e7 X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:483; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: axm5KJfkjwv296AtEd4T0+mAyK6IoXAx270WXAj00len5fn4iG/Wcm0IRrND2fJC6my6PKYfMit4AmFdhyioeuCH5j7pBr0HXvxxs5+VT07FX/lModi/PNlrB3+meKkQ8xU1o1MR2z9EGbTGL53KsIutsWkNnCo6AopsQqB84z4WkWZvyNIUrMyjZoKzR1X4Z20povxU8+wScF6Vfssc4j+tpPIokXNYVubsr8YH2/U7C66hx/qjKN8apSGs6k15YHuN6vHcs2bcIkIci5oOOapwpI7EdanwXXwru+5lRuI2p5hfKitodBoLjYz0AnthGgujb4/skvagR7kNvdlI8K3u4WsqdlIaGXFLfYDvihdNNoRqqNMprC90GnInok10OC2EVmztTob1C/cILkasydD8NmdRZCXGsYsM8vG3og6hgqUCAYRk3ZdJM7A2TlhQmvZelZBXKQR/reSdTk5ARjv8SgJuXPap2MBE3c44iBgwDWsTksz/zO8rfhXBGIe+DuvX2NRzmsudRQiZfS7Ozu/nQA27cgF5X4Fj0kKTVwqWs0YDPmm+SP0pVEH4LtVQTT/fNAXswglw6MUtn0tfzVxiOooPX890TVnQKLcCtmQ7A581ZXW4qpQ8OCRbqgio+EAicfgzTTuJTijU0LFx01KuQrAM+TKeduiimodympS8aVwU/kmMfHxPXSCru/7WQjyd+1TpJJsC5YWc7MpO1g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IV+9Y3OOexuaE0330T8vjyAeRiiDqBTO+SLWk+e9wTahfDB4RNwkxcF1rLTgdBHc7DYuXxSxkcrSSi5MAJCKtmSRe6VyWicQ86tMweIdfJkuW1LcFsTPlgToFCDHrHL7hXzyXQDQA4Mu0XD8IvFPPa5/ZcUQr/HYn8u0stFIFoxpjXRGFPJcqzT47hT4BEBho7sj1IViVdvC/NmxdrZgZvAmXmaidPdmGtCZVp2EXjbCXdFEbOmy9pJO89S7VuUfaQm97qvEN1CfVZnNUG8vpYIuSdoxKwfMpjPe/WD22vidL4vnsnFyYLxr+SdSIppH4oQ2tBfKv4OqbkdpUaKJF5I/5fwsaHBgB4r473ZXYBhjs8RRdz9Sn0kgkoganw9VfJ7+MGlLmiDLCwZ89BOf5sa0oZ0941aECHZDWx2GoEIs9NAwdqHPnHjP5R5wIVZivtg1TKz1DrYgzrxxOrKJ9HN4s9lNlsb2mG6Ah4vkvMjHvXK3bGbzfh6rpXGK4FEG8R9s2ITjQ4ORaOQ3PxfKmMx8kDcr2mV1Y6etbLKUQIEMI32mZBnkisGX18S64f5NE4c382Y//t2mn/gsVOv6+dwxt5ut2TloNhU7arQELG3J9T+KdQw9ODmjMd7yd5/VYyxeW5tf53cp5CMSBjv7uZi70gWVIzkWwUPIQanDQbaFRB3iGtz7xPz60SQ7Jxv42vkpbhBNNGtW/MKp7wU0rxPqXeyZbeXQrcFX1ID667hi21Y+2IrZ7efi5nsz7nMZoUycR0tuTXsy9vyxpAUZO/nxCzxEoIvRs1miq+GG/WPbZIORJdXvES+r+tISFEuQQuhQiCrVIaTkvilzHwQ6ZfceSbXesPZXjK/G4wWD6gQo49ucTF+7PREYaPacMNHYTChCjfLMXvdtAyOKoY4iELCDEydCRDzO+EAKKCfkUgpzJprUoCwBcn0M8hhBkoUhxAw+y9C+E4D4CIprPwUx88fG1IylQ9knhjNKiWzBEirqg56XcRjztXtHbbzXURpukwy9bm8hjmf4yGmB2POAB5ffTzIqciDz7h/GEBwCKHPOE55f+WwSZ+QHPNFCuQ5knODHsEyjO4SdC1ORsbD1t9QxtHp5F30YegHQkjBZ9Ob4OT3Lfw57MZluR0EfMbRvfDKKwNeHKdVE9TqXtGlLCgNu1bRqf602q+JIjvUScSPBQsEaqn+TVEtxNsC3IXjL9wGxdVBWuoJY4S0O4EmvcpRx2GrtbGTq7Oetmgl+PS+UljUpbDqQabe8iKmn7cGcEMXYVB+Ow89bBJxZmo3ars/umjU9C4ioOkBxZfpG4ZkobDqUfZ9LdQvZKi1cSnGHXOOTV5B3o9pR2oyoLV6O5hGTIEb1uw4bE3DybEj9/Us+UQlUqA88cAwXStWAU3NiydBHvYsOYqSxwWy3X86xslpIGruGe0Fjde4Rnzwv/kh8SP2AXRwKxD3lt2OsdZmMOAqXJjWXbh7D5lDsUEvmKcUfXSb3tucCPcfiS4HP17OJH8n+8UIvu74xY8KXSy3PJiq5/Y6coI/hdx/B7WK2IG48OrROUhkDlCfFdkDiXos0u/wJzB4BxQEvvgt8Zu3w5VVJUljCnVMyN3rXCvf5p5TFhuGkkmr4secRIZ5xFCM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 390eefac-8010-4142-1ce7-08d9bfe0a1e7 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:16.1211 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: AXVYhoz6KrU3U/M+RAhkoGjUCyCebELBdMwN7TeOCa1HhA7wDJwntXHNXBFkZ80yHL9boiHmGIaNLC8XRaltHA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-GUID: 9Tl74gekw-G6D1a5ulmne0pttdRlDSUe X-Proofpoint-ORIG-GUID: 9Tl74gekw-G6D1a5ulmne0pttdRlDSUe Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define and register callbacks to manage the RAM regions used for device DMA Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Stefan Hajnoczi --- hw/remote/vfio-user-obj.c | 48 +++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 2 ++ 2 files changed, 50 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index c6d0c675b7..46f2251a68 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -208,6 +208,47 @@ static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf, return count; } +static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) +{ + MemoryRegion *subregion = NULL; + g_autofree char *name = NULL; + static unsigned int suffix; + struct iovec *iov = &info->iova; + + if (!info->vaddr) { + return; + } + + name = g_strdup_printf("remote-mem-%u", suffix++); + + subregion = g_new0(MemoryRegion, 1); + + memory_region_init_ram_ptr(subregion, NULL, name, + iov->iov_len, info->vaddr); + + memory_region_add_subregion(get_system_memory(), (hwaddr)iov->iov_base, + subregion); + + trace_vfu_dma_register((uint64_t)iov->iov_base, iov->iov_len); +} + +static void dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) +{ + MemoryRegion *mr = NULL; + ram_addr_t offset; + + mr = memory_region_from_host(info->vaddr, &offset); + if (!mr) { + return; + } + + memory_region_del_subregion(get_system_memory(), mr); + + object_unparent((OBJECT(mr))); + + trace_vfu_dma_unregister((uint64_t)info->iova.iov_base); +} + /* * TYPE_VFU_OBJECT depends on the availability of the 'socket' and 'device' * properties. It also depends on devices instantiated in QEMU. These @@ -289,6 +330,13 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) goto fail; } + ret = vfu_setup_device_dma(o->vfu_ctx, &dma_register, &dma_unregister); + if (ret < 0) { + error_setg(errp, "vfu: Failed to setup DMA handlers for %s", + o->device); + goto fail; + } + ret = vfu_realize_ctx(o->vfu_ctx); if (ret < 0) { error_setg(errp, "vfu: Failed to realize device %s- %s", diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 2ef7884346..f945c7e33b 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -7,3 +7,5 @@ mpqemu_recv_io_error(int cmd, int size, int nfds) "failed to receive %d size %d, vfu_prop(const char *prop, const char *val) "vfu: setting %s as %s" vfu_cfg_read(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u -> 0x%x" vfu_cfg_write(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u <- 0x%x" +vfu_dma_register(uint64_t gpa, size_t len) "vfu: registering GPA 0x%"PRIx64", %zu bytes" +vfu_dma_unregister(uint64_t gpa) "vfu: unregistering GPA 0x%"PRIx64"" From patchwork Wed Dec 15 15:35:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7535FC433F5 for ; Wed, 15 Dec 2021 16:14:35 +0000 (UTC) Received: from localhost ([::1]:33790 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWUA-0003jX-PI for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:46:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43098) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKR-0005y9-LJ for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:27 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:50514) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKP-0008W3-NW for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:27 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEKYEx005935; Wed, 15 Dec 2021 15:36:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=pGe56WaMi8qjh8aTIOmj0bSj7cn6tx35Eirkv2aBxQA=; b=nSVLSrqZQWM8OmFqd30MYh3AxJ3E4OPkOTbh4h+dZ20+wN0HZyAipTzA8BTympx0Av8I xsQ+k8y59mnj5ajm8gSbDnj2J15TvprYBBBP2Bgentina3KgVwtLUgTjynVcnXOzFa8/ hsEuldWdhpyt02TN/33THRXDtcvmcaYAEnQPxrIbBSA7qIvUJPMJ7HX3Nen3rYZs1sHx LPx8Ym/FPpM3ijXPT05v01KEqF+8MaTqDdTYvE8qfHA0LIKRjGrmxoGMmvHetDwK/C/O RYRy2i9tiNFxbtNSA1P/pjqD4D5XRtDigKOoNRfOCYVfM6GFIqJlpOPN8hNNJzvw612m FA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx56u6ug5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:22 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUWTY094284; Wed, 15 Dec 2021 15:36:21 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2169.outbound.protection.outlook.com [104.47.55.169]) by aserp3020.oracle.com with ESMTP id 3cxmrc0u69-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kCrzTwlRf9PVVYhNpyBgptmQFc55D/16FxaLf5HM0LgQyDgYfqIjd7Ygt45WrXB1olDqLKXBnyQSbGe6t08mg+py+UwddMh1F7DSLpS8XA/gqwlFtb6NuNPkbPzTj8+69DGHDaTG+LSoQiXF63kgzgLeKOrx0mO+TAQ+oRH43/joftjf948I1jzMjjjYl7ny0eoAOE28Xs8SMXGkMiIY09sd3j45iUk6QUS2ct6HTH5V+nXMjhzv+Tlu9epM8mJKbV48ZBF/W2J+UA8P/SfX2yffQK9By3lW/TJD/IpQXBKT0xtU1GSGOqhcp8hN2Q8AH2JssgnhmEVK5xip/f5Yhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pGe56WaMi8qjh8aTIOmj0bSj7cn6tx35Eirkv2aBxQA=; b=nDMxptsm9I5npOLnNrScIs0Yei59SLmop3d7C2Huw7ZczUuIUG/mTMiZOmYRuV0IF4RbOR9BKXwurTysJk/nFwfNRuYK8TbUWZn/c2CACwsMIY5yJyEwBuGvjcrasWkRCZElp3bcxSWjgbKVk+MxcURofrCcbb8UJrFzGuJIWFMOjRxPkhF/GwyPLsk6O6U+0aGkEIcEr5pVqjI8cNeJ/ktFjMCTWmBYs/LOHE+JIF+Lr3OLkH6syhWbw5TrwZnF/hkTVdUZMc5kluUymy9I5BPzVrMI6J9ikqRSj+WGdTp1V/V7oPTImAXsocgHJN3ycySYgn78ySVBgGO0yzt4oQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pGe56WaMi8qjh8aTIOmj0bSj7cn6tx35Eirkv2aBxQA=; b=UPErEZCwRjEfDdTYu03jyCNXYv00t25h9Y4MEzi1Fr5NlTKlJfVJR+eONFJgv1i59SJkUYsuxZJwoY0vWGQm1sCVflxR/SAEgQTEwGnzBsO8q4X7ehzkoTOPaGE4rjWfyxPnWfae4slRX8KWGFaaw/S8u2JJff5G0g/BRg8DGw8= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:19 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:19 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 10/14] vfio-user: handle PCI BAR accesses Date: Wed, 15 Dec 2021 10:35:34 -0500 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 67270b04-e8a5-4b7b-fc8f-08d9bfe0a3d6 X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:390; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 54UdTnH1kfGwWuRwAyj1Gi31DTygviq0EJrFqckxhuGONyrsb+PPKGRvcdQdee6L6BYikAuIwK2pHeFFvag0w+Aa3GwF+s+Q/kdlzfZe1G1H7TAWHe6SUQatqyFvBzPE2s1EK5bpF9/9zWXpCMiu7oTl4pfh+YX6VSLE4724kKVmJoqMj/BdHXy64gk/JJAf1CIXkQI1uX44jwdYTqJYvS10a10YZuqqcRnXFVSpdiW6TucdV5ccze4R1+fmPzwf2Q5SDOm6Uy9D8270quE+VFuRhNsDlLNeUkZQ+pz1li2nl11o4oDu6WeD6YtjTfPnjOA4l521gX71PZosW8MZkqQfags3EmwlBBNdk2E1Y8ahqeozLfrTJ/GbfKMpaCxbKw3y/7KNX3n4+2okGkLYkILJStSkFdsGRn6RphiMAewr1MLvZJ9t2G7owL7pqxiXGgacuQxXKj8T7pKgtkfe1jO9k32ddsznZd3LBtVzw47nkc/5hTjlp3GuE/3pUCUEJJ2oWzRF+MFvwuyG4oiFQOVNqBXlAgTaubKdyyUxBJzpU1tLmLRtHQeMh6a2iH9bm+Zjd342AKYmvCrGC8Pw5M8YVEmzHsPrl4qmLPcVCTtHpJ6/UEAd8hXUseIkVag+wKuG97VF/Ar2UJTk2rYLBDCVgrKLU5YHPz6inhjgi/TSCla+YQtkQVumOdfpcuN3HDH6eJqLlacVub3FhQsf+Q== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NVFU3HDv9Bq62mJUOE5+aAsMKpdgzJarsyiYQ/7hAzOBT0pB5KAQvz3PxEZqgvP25/pax2dEG2FCIGrteAy1GzKfXS2K+yhx0SNzuzCLBMjt7wxcuK6bqw+NBiJNWOo1ErmgGeqObNCVtvX6D1wrGGaXmjATAAAFnJbWWayeoQ0UoRBon7R3jO2fjXdutC8phALdLCP0wzMfPYS52u9gZKtUbaRrakoUpcceEV3eyaPJ4qdoDI9Jf7nauquBMcZ5k/nMfyM9ne1POMdNnnHyYDCovf09aFRiN6MzdbEcMai4Ms1CShRmE6OWBK6sqDcfdy/RfvLVNRJ2m6H8dSZBU3t+ZtsVb15MbpVM+h9cEd/qEiQwRErvM+klrbfjXMQ3EVz5vsxvfG+jkMXnNzBY63MDk3k28aUJ+pA6o4tA00Z/cnBwzmdmDH4r/kEClpF/vmYwF2iaRY9MAJsJuuouwHsCdaGJ+ARqi0bc1JkyBOCpQQzCDZIL9Lm6oZCT9vNxpfOaLuOgOayuEwwDedx/H5LH1SwX0+L90ZYPS6+PTrWpJvoRME70WW5uNHo0RyIHYNU5sgwAy57nUZj3AGFMCBU0PljuQ+4stWLe8Z/hARmVcneDrGuEjOMZYo0amR4gB9+d5jHjGX2jZPNBNfgQVzb/KFdQ6bPzlFPatyTzPvl0h9FikoMTROT/KfCn1telnpfJURLomHeP7ciQUOhAP9TIVVTCMXmn9fDQ4IaZefHQ5BmYgUdDDaO1I2490ALXyiEqGY4fpRfRI9ACl10KOm7DAOp7oLaWjW7fBiA9GbBOGuc60BRc4V3BojBVYN9UtfcDfTUjkHEX4F806vAumG8tZmCIzm0We3opPiCbJIRosXzuWGrsn9rtQN41hpTfYSverJHp2GYxHZq3nzuIjpqpoP7EFgSzezZSi9cPitujFto+EEqKAAZEpHpYZQP+4PYzIDGipVQ0qPGAHzxhpwZtU6Q3PlSopi3wP6lHAYH2uMmZER+e8ZCubeCFAbjFm1SjXT+XfgVs30QzLt/qbaUCRpce9VhpGVa8BewF7Pt0+Ot86PTAkwxwSVaJX7u+rMe7z+K+mhrNv3FJ0Oo1N9kkJbNZDRUbkhUuK4yZ4OHMtTm2lCuoSh5qsPVuk72uTAExbgLcArUpOfUpn6ROjWNoBZZn48M8ICe24H1Ro2ckfRkgLgvBtxKmiEboANChAf3giAlZC9nroZ/Z3UijLoIKxuu92mD/SqgcF/fMbnUvJZEbRu7S9KkKe8JctT5BikFzYBQKAOp+b+LTL1kgxyRc1Z9SolyDAHsUOKownmhZvgKiUtlXrodr2IQPOTYFXnGPj5Ttyi+q/586yz5qwxOb5r+y/uudy4ydZ7RXQzjvYyL3mzABmlN4hajU3nl0/2as35eebkMaUQuyGrcCzLJYVpoXf4H4dbwtVz37RUI8VX/jYU/Es29dgyRhmg7a/9yccikzfyW9qFiruZ6JIy6x7B7HO1MjIMLu6m+yDk8vXUiQ4As8AT2WEZNTQCzsUPk2uIuEi2k0cmaIWBLCd/rLqVlsQavT+CMF6g5fhoQ/JrGmelA3ULqKgbRomsj48AWq5zuhFw0I0nPftiMO4w16cPhFqjNHH75089Z4a5w= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 67270b04-e8a5-4b7b-fc8f-08d9bfe0a3d6 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:19.4178 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: iRiqrz8oxmI5ogpaQr8SoB1SykKX/kxmPh5OBC0SeXtejDt8nWEzqALnzih8DE2dUwc4o30MdsRd/k7DGwk3oQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: ExPJOvHzy4y4Lm82b6I5r752fBto4Ccq X-Proofpoint-GUID: ExPJOvHzy4y4Lm82b6I5r752fBto4Ccq Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Determine the BARs used by the PCI device and register handlers to manage the access to the same. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Stefan Hajnoczi --- hw/remote/vfio-user-obj.c | 90 +++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 3 ++ 2 files changed, 93 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 46f2251a68..9399e87cbe 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -249,6 +249,94 @@ static void dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) trace_vfu_dma_unregister((uint64_t)info->iova.iov_base); } +static ssize_t vfu_object_bar_rw(PCIDevice *pci_dev, hwaddr addr, size_t count, + char * const buf, const bool is_write, + bool is_io) +{ + AddressSpace *as = NULL; + MemTxResult res; + + if (is_io) { + as = &address_space_io; + } else { + as = pci_device_iommu_address_space(pci_dev); + } + + trace_vfu_bar_rw_enter(is_write ? "Write" : "Read", (uint64_t)addr); + + res = address_space_rw(as, addr, MEMTXATTRS_UNSPECIFIED, (void *)buf, + (hwaddr)count, is_write); + if (res != MEMTX_OK) { + warn_report("vfu: failed to %s 0x%"PRIx64"", + is_write ? "write to" : "read from", + addr); + return -1; + } + + trace_vfu_bar_rw_exit(is_write ? "Write" : "Read", (uint64_t)addr); + + return count; +} + +/** + * VFU_OBJECT_BAR_HANDLER - macro for defining handlers for PCI BARs. + * + * To create handler for BAR number 2, VFU_OBJECT_BAR_HANDLER(2) would + * define vfu_object_bar2_handler + */ +#define VFU_OBJECT_BAR_HANDLER(BAR_NO) \ + static ssize_t vfu_object_bar##BAR_NO##_handler(vfu_ctx_t *vfu_ctx, \ + char * const buf, size_t count, \ + loff_t offset, const bool is_write) \ + { \ + VfuObject *o = vfu_get_private(vfu_ctx); \ + PCIDevice *pci_dev = o->pci_dev; \ + hwaddr addr = (hwaddr)(pci_get_bar_addr(pci_dev, BAR_NO) + offset); \ + bool is_io = !!(pci_dev->io_regions[BAR_NO].type & \ + PCI_BASE_ADDRESS_SPACE); \ + \ + return vfu_object_bar_rw(pci_dev, addr, count, buf, is_write, is_io); \ + } \ + +VFU_OBJECT_BAR_HANDLER(0) +VFU_OBJECT_BAR_HANDLER(1) +VFU_OBJECT_BAR_HANDLER(2) +VFU_OBJECT_BAR_HANDLER(3) +VFU_OBJECT_BAR_HANDLER(4) +VFU_OBJECT_BAR_HANDLER(5) + +static vfu_region_access_cb_t *vfu_object_bar_handlers[PCI_NUM_REGIONS] = { + &vfu_object_bar0_handler, + &vfu_object_bar1_handler, + &vfu_object_bar2_handler, + &vfu_object_bar3_handler, + &vfu_object_bar4_handler, + &vfu_object_bar5_handler, +}; + +/** + * vfu_object_register_bars - Identify active BAR regions of pdev and setup + * callbacks to handle read/write accesses + */ +static void vfu_object_register_bars(vfu_ctx_t *vfu_ctx, PCIDevice *pdev) +{ + int i; + + for (i = 0; i < PCI_NUM_REGIONS; i++) { + if (!pdev->io_regions[i].size) { + continue; + } + + vfu_setup_region(vfu_ctx, VFU_PCI_DEV_BAR0_REGION_IDX + i, + (size_t)pdev->io_regions[i].size, + vfu_object_bar_handlers[i], + VFU_REGION_FLAG_RW, NULL, 0, -1, 0); + + trace_vfu_bar_register(i, pdev->io_regions[i].addr, + pdev->io_regions[i].size); + } +} + /* * TYPE_VFU_OBJECT depends on the availability of the 'socket' and 'device' * properties. It also depends on devices instantiated in QEMU. These @@ -337,6 +425,8 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) goto fail; } + vfu_object_register_bars(o->vfu_ctx, o->pci_dev); + ret = vfu_realize_ctx(o->vfu_ctx); if (ret < 0) { error_setg(errp, "vfu: Failed to realize device %s- %s", diff --git a/hw/remote/trace-events b/hw/remote/trace-events index f945c7e33b..847d50d88f 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -9,3 +9,6 @@ vfu_cfg_read(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u -> 0x%x" vfu_cfg_write(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u <- 0x%x" vfu_dma_register(uint64_t gpa, size_t len) "vfu: registering GPA 0x%"PRIx64", %zu bytes" vfu_dma_unregister(uint64_t gpa) "vfu: unregistering GPA 0x%"PRIx64"" +vfu_bar_register(int i, uint64_t addr, uint64_t size) "vfu: BAR %d: addr 0x%"PRIx64" size 0x%"PRIx64"" +vfu_bar_rw_enter(const char *op, uint64_t addr) "vfu: %s request for BAR address 0x%"PRIx64"" +vfu_bar_rw_exit(const char *op, uint64_t addr) "vfu: Finished %s of BAR address 0x%"PRIx64"" From patchwork Wed Dec 15 15:35:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AA17C433FE for ; Wed, 15 Dec 2021 16:01:33 +0000 (UTC) Received: from localhost ([::1]:44286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWih-0004mC-Pb for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 11:01:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43146) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKW-00067D-BU for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:32 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:61428) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKU-0008WR-6v for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:32 -0500 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEa9rN005911; Wed, 15 Dec 2021 15:36:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=hOSBlTBFB+s6ea/5oe5bG28IG9JqPOZLvRE8szeF1lM=; b=tyrjb7j4VX2HZkpW01NlNgO01HFWV2MU2WflC+HrWXFNFF+aUFpY9b4FvRKeiFiuFkKK Souc31Efqy4I5REUdxI36LiHgLpFhCBk2KI10B4pSewps6aUdEAn6ltiASLLT0a9fuqT vMDjYrQJnQGJRIdp9ZxpMJuirt+IIi+zJWLGqPgJWjYsItK4fyofW4GBqi3mCuF59pGD m5TAC/tITXhXTGGgWgbUmrqvY+gvfXFL3KbFfSchhhdLZlNhim4vO4jG8BP4R4HDUVSs MLMksXbx1hBXZLiWGmY5khcE8TNjI5ATB58mMJziRlzkJa1Gbp7o1i7E720NLQIsCWZb jA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx56u6ug9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:26 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUXS6094355; Wed, 15 Dec 2021 15:36:25 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2175.outbound.protection.outlook.com [104.47.55.175]) by aserp3020.oracle.com with ESMTP id 3cxmrc0u86-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Zgb7Vh6z8+C6lNfaqvjgnNwiaJLuWwWeI2aoK7MlANoMse8alr3ipMgrl0AvqniFXDmT4JBHeJJaKNsEHqaCfGCjyIohCpljdhC+EQ60UXgk/tDtA/E/tmN+lGQxQvi9FSb7I+dL/3csZQORhB6+m1PMLKRuofmoy0f3D0mYdpvJLiSdDDbpw3x1X422IyfKibRpt5siwa0/UP6/+F6W0H/eR+0/TXqKoxcH7DVhsrkzCz/mg/jZ6n37XenNgblDiGcYjgvh0NCvxTW1jMzrAIvh7VqKjabCUUGIiJyXvBAmWR3TGk5rhVVA+cQmM3aGGkOnpxDickkC+K4Xznj2mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hOSBlTBFB+s6ea/5oe5bG28IG9JqPOZLvRE8szeF1lM=; b=A41mI0B4PyjRfJwEpFHI3gkWlu4KlfWACCaC26nNlJpZ5yp39MHHGNat07HnbtszC3jMZ76+vQSdEOOuazO9DVc5z60I7MMJGLJ8BVGFEo6cpQYI0uTLPyetWt5O45d2O9v2gsL1W3HPR5Ni2pxM6BsYTvFc1mNy2B5Rq3lmyz7cphHMc3BDHa33K6y8RIJQitF25U6yPZ8dHliFHabt56pFFbQ9g8YW/FRZYCEwqo9gt7Rm2wbbbhmt+lvMygX5DKetDr6iJx0wnIZsTA6FIflKzf5qgXqnp6Ry2+0AGJAGYFk9ZfgXwBfJOntGgZsB4qyaQpGwb9+c78z30xqyzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hOSBlTBFB+s6ea/5oe5bG28IG9JqPOZLvRE8szeF1lM=; b=Wb6xp9YDmf8g+AAuSMM9sdOmxS+0kjKTR9NxDCsRuT/alDpThvPect/xJPqKlQvV7XLeb8zn2un168+ygO3cJijzVl65zmV4nWW9X+iErXQckTbn/zkTJZnPIUvvT26QO8o+4O/0mW1pXv0bVZ+sEFL6URE4u3zMZdJK9HJ+Vko= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:22 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:22 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 11/14] vfio-user: IOMMU support for remote device Date: Wed, 15 Dec 2021 10:35:35 -0500 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e57d8d15-4eb8-4282-032e-08d9bfe0a5ce X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:38; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: f8DkJEPyRndYDK0oMkm5mKjOBFKA9U+4VwYVnf5rrLCZCCoRt7G9dqxtuJv7p5ZNgol2Ql2X72FbTvUZOkOI4f7g65IMPQOTCtIvD96QQySKEQHe9b/CfqgP25u/KwYeiH/TgrI9xQCPNwNepgoG//7v4tTXH4Nx2Swv7zpy+OPUWbottUkf4erneCMAi659Crr3NfxBdEsDOnad8aECvpJV4WBHkRA+J7aV/GPo15up3cA84BLL2OV/5DhkRknf/jOJBWiKDrJQNK5Lo18SMklUen5yPDOj49yZECWOt6JxSXGDSheGto2l77j0ZHQnQqfMYK/H3WCahQbZl920zaVXKgiU36ohM9ckxYuLPS6+fsxG4SYwNG5okvQ8U8BjSqrIzu0SBmxm88T3cuzIPeBNMdrIfZjRUSyzlEZJKmwHH21OVrbDvFDwaDXJgpsT5tH6LB6b5HXfrBRJL1PfpJErblKzd2+XRRvjeLXT1UC6S5TNIjnrygpyuJUcNPUmT+A3i536u2a0KkUWtwaEziVe67Q+0OPBRgZj+doe0j/npa0B49IKDkRWTa+43OclZyiyxLEkBTM1FQCvWwUcbsKWaJDgjMrOGtbVIwQ6KRDZT2NfsQYsJ3E7BJ1IODhlhnek1+ExVb8+BXKkrBjzhIXE6uDihCbQriIW8WDBD+XBC19nNC1eWfJtyBNRMypJtb2USD0gIvAxr1HA9iqxAA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?JEQC0D1mIqo4XPcPCEI6k6mpj1Bc?= =?utf-8?q?eH9Gvy38ZFWaPr1UrOSWKoPjmbJTuv0UDqLVl9um9ucpbnuMLx7qgjnkqsi9WmACx?= =?utf-8?q?p8he1/VwJNljuS60vO73aV8c1NUDDWonhJXsqDA+tSDOX3LXAM7ueQkTDFSeMxCxG?= =?utf-8?q?A9zsZs6xo+wYb2KgfQamawPE0+ipdfmPUwWHadAzQ3Bb2erEIfzyHl7KWqgLtbrHb?= =?utf-8?q?8+IUp9tCyfy5+1urZiWndmnfxuAL97bK9Qf8U5pNNk2B3AtsqQ0rWTvwHQswSxlOR?= =?utf-8?q?mCguTHZrsmmDpfdWnFbfIv2OLMNrakZSQK99uFWmvm97swyS0aEy4ersPOCVQF9Ye?= =?utf-8?q?oxB/tHCjuVaaAEI+m8BnvOeJ4Fzmn9UwKiAskdeJATiuIIOlSnD6fummesn7wtlKs?= =?utf-8?q?y5yhxejQX1eJs9MdlxEggBLmSxQ0Cc6FRzSzh/3iZz4fdh3EVIpCm5Z/SywrkplmA?= =?utf-8?q?CC0X4RUYSoLyUFKu83fdL29kJQd9aU14a9RlMi8iB0F3W3N6zff1RNDe6zE/+oicG?= =?utf-8?q?MVS98ofVxEMJx3DZwRGobhOrWrcjuSVElgpczynGDnWDECLCT9+H9ooI/z3JY+wEj?= =?utf-8?q?X90X1WJGam9fEFnmS8Y/Jx4AqjEox1Wp6Uby1A9NEz0qhzXuuQ5pqeG9m6zJ4w2ji?= =?utf-8?q?T8I9gs6IR56QclCRPDmZ+sH0lsjfpXzaj7t85hMJZ6rkDLq9qzJ1maOSx0sopqBPv?= =?utf-8?q?8+uIZvBJCKeo/ofbqIBdJj9FQzRhhoarijNKeM96JaGzbZ4ljnDwhaVlJmC50Lidn?= =?utf-8?q?2LYIvJAGpWbH4RC4Gmamnz6hWGvo3MVJ/lvmHRTbBS0S0x2sCTYfINwF+y4WeTHXs?= =?utf-8?q?2NU6ZJGyy9boQyH/xg3O7a2YD3MMj+uDyY3JdRGA10TllA3LTr7KilPpA9L9xG/RG?= =?utf-8?q?/VviB8xanUgpOfM8hRysy9cUlP2fV75s+DbFjRmJkWLpoBXX5N7bmcuNBOu0PbHSr?= =?utf-8?q?RxDRWdLje6q/4KLBr9O2y6bQz4MJ71H9rnhkhoIJYgn4dSrosDz5q+1cR33oByWgX?= =?utf-8?q?Sb82ieJWPokUGwnAfLHfPyfckYdRtWa3xWRwMv/1vimlzMcuN5Ys9s4O7kPYIR3+Q?= =?utf-8?q?pEZq355NWMDrmIstUt7ZiuBlQNMpTvpRFy5pnHkUbIuVSEveW3p0Jdt5ZuJkmLdWb?= =?utf-8?q?RCvE+tLYld0uz3UWMOmjWQiN2JHmZyqX4YH7W3M2vzDbiS1fmPTe5S8TuOUQ2AICQ?= =?utf-8?q?ThN3WaY7cB3WyeLT8ak6QjLofWp5CmA9JRPaw6x0v8IxomiLflDqpMDOKyV67wSnI?= =?utf-8?q?0kqnqT81amu0Jq9A/p5vdgAV92Z7snji2UIeARQTmmfYvvz3jzi03nSg4XEfhdPTD?= =?utf-8?q?VZHUtraBclRR5lXI8cAI2iWm8aOxJTg7LUf+HXn58j6XJehWAK5bs86wbzSHhVRH4?= =?utf-8?q?erDSgxVbrfXYlObUgJTgtslaPtiXmzVunLyK5oDDQU0VHukT3Fz35uGzTp1iU39M0?= =?utf-8?q?KHUGchESwJ9dCaCQjWVsJnljtaQ3Y1rNSEAiDZBnYqAzVHZRCf2HUBvsysGKbhpG3?= =?utf-8?q?+xUP9xUAnwS1Vw2m1OsueJOjAzv1FrqdX6m+UdbvVtoeSw8zXG+DHAo=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e57d8d15-4eb8-4282-032e-08d9bfe0a5ce X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:22.6832 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hgjd4BBE0DILIYYCI3NXAC/KrOp+rKq7NmHNUwvLkqnmB4kYZQVEy+pZ6YYq4l5PGEcEkyWT5kkDLlZn/+HgRQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: 1cw_7gz2Ag6toHKgpF9HONuMfeSnaYtS X-Proofpoint-GUID: 1cw_7gz2Ag6toHKgpF9HONuMfeSnaYtS Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Assign separate address space for each device in the remote processes. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/hw/pci/pci.h | 2 + include/hw/remote/iommu.h | 24 ++++++++ hw/pci/pci.c | 2 +- hw/remote/iommu.c | 117 ++++++++++++++++++++++++++++++++++++++ hw/remote/machine.c | 5 ++ hw/remote/vfio-user-obj.c | 20 ++++++- MAINTAINERS | 2 + hw/remote/meson.build | 1 + 8 files changed, 169 insertions(+), 4 deletions(-) create mode 100644 include/hw/remote/iommu.h create mode 100644 hw/remote/iommu.c diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index 5c4016b995..f2fc2d5375 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -734,6 +734,8 @@ void lsi53c8xx_handle_legacy_cmdline(DeviceState *lsi_dev); qemu_irq pci_allocate_irq(PCIDevice *pci_dev); void pci_set_irq(PCIDevice *pci_dev, int level); +void pci_init_bus_master(PCIDevice *pci_dev); + static inline void pci_irq_assert(PCIDevice *pci_dev) { pci_set_irq(pci_dev, 1); diff --git a/include/hw/remote/iommu.h b/include/hw/remote/iommu.h new file mode 100644 index 0000000000..42ce0ca383 --- /dev/null +++ b/include/hw/remote/iommu.h @@ -0,0 +1,24 @@ +/* + * IOMMU for remote device + * + * Copyright © 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef REMOTE_IOMMU_H +#define REMOTE_IOMMU_H + +#include "hw/pci/pci_bus.h" + +void remote_iommu_free(PCIDevice *pci_dev); + +void remote_iommu_init(void); + +void remote_iommu_set(PCIBus *bus); + +MemoryRegion *remote_iommu_get_ram(PCIDevice *pci_dev); + +#endif diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 4a84e478ce..57d561cc03 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -95,7 +95,7 @@ static const VMStateDescription vmstate_pcibus = { } }; -static void pci_init_bus_master(PCIDevice *pci_dev) +void pci_init_bus_master(PCIDevice *pci_dev) { AddressSpace *dma_as = pci_device_iommu_address_space(pci_dev); diff --git a/hw/remote/iommu.c b/hw/remote/iommu.c new file mode 100644 index 0000000000..30c866badb --- /dev/null +++ b/hw/remote/iommu.c @@ -0,0 +1,117 @@ +/* + * Remote IOMMU + * + * Copyright © 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "hw/remote/iommu.h" +#include "hw/pci/pci_bus.h" +#include "exec/memory.h" +#include "exec/address-spaces.h" +#include "trace.h" + +struct VFUIOMMU { + AddressSpace as; + MemoryRegion mr; +}; + +typedef struct VFUPciBus { + PCIBus *bus; + struct VFUIOMMU *iommu[]; +} VFUPciBus; + +GHashTable *remote_as_table; + +static AddressSpace *remote_iommu_get_as(PCIBus *bus, void *opaque, int devfn) +{ + VFUPciBus *vfu_pci_bus = NULL; + struct VFUIOMMU *iommu = NULL; + + if (!remote_as_table) { + return &address_space_memory; + } + + vfu_pci_bus = g_hash_table_lookup(remote_as_table, bus); + + if (!vfu_pci_bus) { + vfu_pci_bus = g_malloc0(sizeof(VFUPciBus)); + vfu_pci_bus->bus = bus; + g_hash_table_insert(remote_as_table, bus, vfu_pci_bus); + } + + iommu = vfu_pci_bus->iommu[devfn]; + + if (!iommu) { + g_autofree char *mr_name = g_strdup_printf("vfu-ram-%d", devfn); + g_autofree char *as_name = g_strdup_printf("vfu-as-%d", devfn); + + iommu = g_malloc0(sizeof(struct VFUIOMMU)); + + memory_region_init(&iommu->mr, NULL, mr_name, UINT64_MAX); + address_space_init(&iommu->as, &iommu->mr, as_name); + + vfu_pci_bus->iommu[devfn] = iommu; + } + + return &iommu->as; +} + +void remote_iommu_free(PCIDevice *pci_dev) +{ + VFUPciBus *vfu_pci_bus = NULL; + struct VFUIOMMU *iommu = NULL; + + if (!remote_as_table) { + return; + } + + vfu_pci_bus = g_hash_table_lookup(remote_as_table, pci_get_bus(pci_dev)); + + if (!vfu_pci_bus) { + return; + } + + iommu = vfu_pci_bus->iommu[pci_dev->devfn]; + + vfu_pci_bus->iommu[pci_dev->devfn] = NULL; + + if (iommu) { + memory_region_unref(&iommu->mr); + address_space_destroy(&iommu->as); + g_free(iommu); + } +} + +void remote_iommu_init(void) +{ + remote_as_table = g_hash_table_new_full(NULL, NULL, NULL, NULL); +} + +void remote_iommu_set(PCIBus *bus) +{ + pci_setup_iommu(bus, remote_iommu_get_as, NULL); +} + +MemoryRegion *remote_iommu_get_ram(PCIDevice *pci_dev) +{ + PCIBus *bus = pci_get_bus(pci_dev); + VFUPciBus *vfu_pci_bus; + + if (!remote_as_table) { + return get_system_memory(); + } + + vfu_pci_bus = g_hash_table_lookup(remote_as_table, bus); + if (!vfu_pci_bus) { + return get_system_memory(); + } + + return &vfu_pci_bus->iommu[pci_dev->devfn]->mr; +} diff --git a/hw/remote/machine.c b/hw/remote/machine.c index 952105eab5..023be0491e 100644 --- a/hw/remote/machine.c +++ b/hw/remote/machine.c @@ -21,6 +21,7 @@ #include "qapi/error.h" #include "hw/pci/pci_host.h" #include "hw/remote/iohub.h" +#include "hw/remote/iommu.h" static void remote_machine_init(MachineState *machine) { @@ -52,6 +53,10 @@ static void remote_machine_init(MachineState *machine) remote_iohub_init(&s->iohub); + remote_iommu_init(); + + remote_iommu_set(pci_host->bus); + pci_bus_irqs(pci_host->bus, remote_iohub_set_irq, remote_iohub_map_irq, &s->iohub, REMOTE_IOHUB_NB_PIRQS); } diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 9399e87cbe..ae375e69b9 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -49,6 +49,7 @@ #include "hw/qdev-core.h" #include "hw/pci/pci.h" #include "qemu/timer.h" +#include "hw/remote/iommu.h" #define TYPE_VFU_OBJECT "x-vfio-user-server" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -210,6 +211,7 @@ static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf, static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) { + VfuObject *o = vfu_get_private(vfu_ctx); MemoryRegion *subregion = NULL; g_autofree char *name = NULL; static unsigned int suffix; @@ -226,14 +228,15 @@ static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) memory_region_init_ram_ptr(subregion, NULL, name, iov->iov_len, info->vaddr); - memory_region_add_subregion(get_system_memory(), (hwaddr)iov->iov_base, - subregion); + memory_region_add_subregion(remote_iommu_get_ram(o->pci_dev), + (hwaddr)iov->iov_base, subregion); trace_vfu_dma_register((uint64_t)iov->iov_base, iov->iov_len); } static void dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) { + VfuObject *o = vfu_get_private(vfu_ctx); MemoryRegion *mr = NULL; ram_addr_t offset; @@ -242,7 +245,7 @@ static void dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) return; } - memory_region_del_subregion(get_system_memory(), mr); + memory_region_del_subregion(remote_iommu_get_ram(o->pci_dev), mr); object_unparent((OBJECT(mr))); @@ -320,6 +323,7 @@ static vfu_region_access_cb_t *vfu_object_bar_handlers[PCI_NUM_REGIONS] = { */ static void vfu_object_register_bars(vfu_ctx_t *vfu_ctx, PCIDevice *pdev) { + VfuObject *o = vfu_get_private(vfu_ctx); int i; for (i = 0; i < PCI_NUM_REGIONS; i++) { @@ -332,6 +336,12 @@ static void vfu_object_register_bars(vfu_ctx_t *vfu_ctx, PCIDevice *pdev) vfu_object_bar_handlers[i], VFU_REGION_FLAG_RW, NULL, 0, -1, 0); + if ((o->pci_dev->io_regions[i].type & PCI_BASE_ADDRESS_SPACE) == 0) { + memory_region_unref(o->pci_dev->io_regions[i].address_space); + o->pci_dev->io_regions[i].address_space = + remote_iommu_get_ram(o->pci_dev); + } + trace_vfu_bar_register(i, pdev->io_regions[i].addr, pdev->io_regions[i].size); } @@ -490,6 +500,10 @@ static void vfu_object_finalize(Object *obj) o->device = NULL; + if (o->pci_dev) { + remote_iommu_free(o->pci_dev); + } + o->pci_dev = NULL; if (!k->nr_devs && !k->daemon) { diff --git a/MAINTAINERS b/MAINTAINERS index b5eb306662..5dc67d79a1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3466,6 +3466,8 @@ F: hw/remote/iohub.c F: include/hw/remote/iohub.h F: subprojects/libvfio-user F: hw/remote/vfio-user-obj.c +F: include/hw/remote/iommu.h +F: hw/remote/iommu.c EBPF: M: Jason Wang diff --git a/hw/remote/meson.build b/hw/remote/meson.build index 534ac5df79..bcef83c8cc 100644 --- a/hw/remote/meson.build +++ b/hw/remote/meson.build @@ -6,6 +6,7 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('message.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c')) +remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iommu.c')) remote_ss.add(when: 'CONFIG_VFIO_USER_SERVER', if_true: files('vfio-user-obj.c')) remote_ss.add(when: 'CONFIG_VFIO_USER_SERVER', if_true: vfiouser) From patchwork Wed Dec 15 15:35:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F20AC433EF for ; Wed, 15 Dec 2021 16:03:09 +0000 (UTC) Received: from localhost ([::1]:47750 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWkG-0007G1-93 for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 11:03:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43182) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKY-0006EC-6l for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:34 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:52416) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKV-000051-Tj for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:33 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEULbA028510; Wed, 15 Dec 2021 15:36:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=U0PNMwpRERF29Fs9jnyBzcAsE4L/ciUMCOnwdrDp25s=; b=rwkDFLBACtkRv5IYNYp6jDLcWpHfYf+sfCdw9xrcMCfWIpHcQw5MddhfiH0KF+BuaIQ9 uUgof2qkaEUzI3aNPPiX+RqKsPyyMxDj+6dTU75FzXNRs4HGo2eY6x7NHaxj/iOecq23 Iga0NdCiSzgyhrzaOBYQtyb14soRAX/LKl6432Q6ft1hIPzy9uskfxNGAWIGEFgXLhKP onGvurHuafxp2x/QRqziE3CCUD08iNEK371JyanV1i1NmH8ger6Bpxyi9Flnlwlwg42U n2rr+LvsVDc4TG2ICQA0ywrj3WnzjBjTr3A/jq/bQJp1FbHkZKzx76jYwF3Dnmz/kGPi xw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3ukf6t5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:30 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFViKh127804; Wed, 15 Dec 2021 15:36:29 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2175.outbound.protection.outlook.com [104.47.55.175]) by userp3020.oracle.com with ESMTP id 3cvnes2d9k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:28 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MISUFb9rBTvSdZznrg0g9i0eZeTs9vruKa3IwwGK5gGGcHO2GQznBG9flG8Ly3U+096Sgcy2UTAmcPlxHW1OmtiXzWGQRoE9oMog7OWVBb21WkndUdFFiTgQkl69cFbJaFGer8rT2lmBnB10hW9jOm74c3xUG8RHw6jvVhIX7naLJwXl1y9wkdfyLkBNkw+eHK69a01l+/lOQdKKMYtUQDChcYgpRda1ClLxiuMDaIy9yoz1nSDbOn8jkDb/1wc6/6iaJILgl2pXQiZzEPB1w+lW2Fn124vO2ll+2uvMtO/xr0p4Q9riIBLLjNwmlQwaxe3ZkoKZU5WThPeFzzf4qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U0PNMwpRERF29Fs9jnyBzcAsE4L/ciUMCOnwdrDp25s=; b=QTtsESY+uxZRiIogoWWE5PRMfYuvG06oshbrHp7M9DnmhdCVmvOUQacXy3NKmbFEW+UKJsRRw4uWj/1DIccdLSKr83FKqAR7kSGtHLeVU3nxOlFuZX0CpNmylyxPztP60qdyZ3NtbCtCSMfmJTmJ8KEOXvfNA2F724gKwwX3y5E1ooQya7E/pqPq8QRCYsh2y9Voyd9wLJtixq2OY9b6UyqP+jN6X/UKA9hwZpeNE4KTd1M2PTftsnloW8VHS5J6ZcrTH4nFRbmtWCp1Zzt2KMaaeS5/CXpL/cJLomsGTyTdEN1TvfQS0CA2ZXdPaZGvnheMRroDKY7nD6LbRRAwEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U0PNMwpRERF29Fs9jnyBzcAsE4L/ciUMCOnwdrDp25s=; b=IZtYgSCIJ++rRTzuRvGlXq4Oq8hZbAyg53zzNTkU6YX9Zdw8SRuuVq75Uysjhrkf6KLt9toJ9ot7vzLxelJaDkOsLZS5p4A8ocpA7XDXBr1VmZlMA7rXj0ex6qE4pECXr24FowcodhwJUaaEaZmoOddJ6/1NSzLFIatluJZc2+w= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:26 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:26 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 12/14] vfio-user: handle device interrupts Date: Wed, 15 Dec 2021 10:35:36 -0500 Message-Id: <219410a610c55ac7f8db806aa8a4bc2e124e9d5c.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 246c52bb-ae24-4882-e0e6-08d9bfe0a7dc X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:279; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cWGfsTgaZGv7B4HZz7hJRDt6YLD4XY/mPSeqzZLsFLJqOUcC+7lbeXaKg7xjOR3JwJxEdQykDpuWpyAcXH19CMqaWEIK81700BiBEOf24l60BWCBhu9L5qLUSMCRaeL+dutHk6zbtqGDFSqXP6tqyLJ7u/DURFKnbHXVjuNX03icVwMdn+LE1zeESc/8Ji41nPJqcdqipAOXu6hsg4S8GpwtupVEYXAEDcy+MyzWZ6byPieAiB9whb5zrse7C1j2kXMA+WwomS4Z9Cl3V5QSYTMa+8MDw7ocgenvjZWwYhLN1amjqrCrKnMv7bUBaGAW8X75x4k5HCHGgzkslPirddksivD4VvM8DpUGTVAXHYvZFZpqq3sGrBVswowjzeLkL7KBg/3NNvClk7FcpZKv1UC2l5srUCyQQLiwZw1dLnmNLQcRSwPhaCVjcdnKpy06zwTaJ1C/dL/rYVhy2MC9DJbchYJ4zeaHdaoux09O66eWmDOjwERvZ+yI9wSFwSdlQj1PrkrX/SUKFtRwqs8IgYobJrQUUIq49Y5PJzgsjGZ5OLYQb53yFtk7iCtr0yy1GRkbCuBtPv9LJC4t1V9X+HnO09UtpW5Hlm0Ko/ugBsXPS7Bdf0ioswK5fHyNTQN46qilRm64ptcTvRbxSHrVOeaGwVLkQUgiB0AwfkMlqG1FtGYZM/vsNaeurmxy32vC5m8cV0nKWQ7f6XU/9iDvJQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JzVkpahkhFdj1hncoGjXvUz/k3jqH0Xy3New2v/4I5JJ/sHuMNqkIPCrs5eFfxS3u3xWPSNgDz59gmdFPY8JmlqQ3rSeYpXEOUu9EPoZWBj85zRUVwVuIosS6VDvybwKmhpxNe1zHQBEDBe6fWImE4orRUddVh0WEJ2u1ZUYO6p0FjZTzwPUJAYFYaBQp30hD1mT6OeWIW6qD/hF85IXctUaiQ9jyWwNnNFXndgf0dWaUvbpbEdqS3hWYfKFCD/rfdw8liu9yl/mCZBQzEwf1Rm0nZ6qcwSIQek1kXDpRN2hfZ29p2RS/QjKxqXQZR0ExuvYXotQLDNVh4oHKd3Qw6IDIuW1uUBf4e8Dg9EkrIVyQeTMbeJ+Qv8c94d/S9B8zawWaK8j6BYBnPE73MfDoXa8pByBnPtScGahvKaen9KhSkvRPXIryMwexbNFUwFACmu6kz6UeLEZzXq8z5KjDDR/VtcWRr3q584Ur6LEnIxSy/hw0nCQR4XvQ3ZUxq0BsvGBjOuVU0IyvnLviN8m4r+GrHdA3h2mw2plGo1RfaKvFmbRBmLy/aNEAz0TcKr9ZzdV9aTIy3+M8SBjwxDr2oQFUpecRrAQNtwawrbyiw5yFL5uTUmkfi0+1UVYwmznleI3eOqDb/XDyqzEdUgkgOcSUh3rxpjUezsBVIdkza2gHjdtwwXEVHUsOH0oGSdM54MCgcCzx6/liBA7+VnyNJr8iyylT/fVfc1QPZARWZCOx2aNoFRy+icFDq1OJbSFu4LpKM1AortPbt4XElwGFB1yH7tZx7evquw4w5WWNOQkR2my7lmzsW9cNqU45yxCXTe/RnVEBAWac0dxr1UliQuQcRV9hbplZbVt2aUVAlBdOZv5pnmn9zVgKYqs+XabCxOoaDaXsiVRDFCRM2EGZVlFXpXYaEIXKQgdBW9Wr5b5R0eBNAhxCwg4OHO5Y/VzQaBcSpKP2JV0HDWpt3gps955OOiwMItIs32VBa6atA3R7ulqloG7KArHL2J7iIIffvXZEMFOT9zWEPFgra6wf10xlpn3hRmqktCizZGFNGLa3TmTaAPV2P62gBG3EHH6VG3kmDRAnD2rJb3rV+g/XHHynVIMGZWnkUjGJx+NhNE4r56VGmo3MkGqlRIhy+fn2D1aHEsiTYDLdWTATFzTRMROPRDt4SgmoF+3bLUq4N0O8Cm233R1MVgw1ZQD83aDDkku7MKLWYSZElS6sVcslKHC8bO6PYK7wVTt3ziXA7E9D+RUwxWgEnuE7gs164At8atz/K1GgQM6u8oamLixCU6x0bfSvDTQeiKKaNY6UWXc4pJir6PqXaQZCXr/IFy/Z3ECjC2w0bkRVRQDgjmlEgrW0BUj1UZ2hTyIDnyv8l5UwzJH4tPhXq5ueJB+bxMd9AX8CjRw4WaRLZNnu0FDbrJDqyyAcPNSiufBTx4mqpEJ187mM2Tsl1xX+asCXU+w7hOPTr4/NhfW3T4cfnzWIE911EGFL2JbW6OF/RHk1Zz48G0OMPlx8MDvoJDszWBdVhsd7uFDZGJRt0Dc/rQGnlneWnhknVcT5zIU4f9/wKUhRDuUKxMHD4WsgvEv8bvpKaYamJssJePIkAzt9Hl999nM+jbDFnTcBsZgkGBTIR8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 246c52bb-ae24-4882-e0e6-08d9bfe0a7dc X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:26.1829 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aF+hRPiSjgKYoGksbr3IQcUQSoKejbV7UfJdypu9+ricLA+raFuOT9sbwWI9MpRzSlAKH1Sg5q7yeBDUntXe8w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-GUID: _Hd1nFp_OIaSYQeYRHGGOf5PMWEVJOIq X-Proofpoint-ORIG-GUID: _Hd1nFp_OIaSYQeYRHGGOf5PMWEVJOIq Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Forward remote device's interrupts to the guest Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/hw/pci/pci.h | 6 ++++ include/hw/remote/iohub.h | 1 + hw/pci/msi.c | 13 ++++++- hw/pci/msix.c | 12 ++++++- hw/remote/iohub.c | 7 ++++ hw/remote/vfio-user-obj.c | 74 +++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 1 + 7 files changed, 112 insertions(+), 2 deletions(-) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index f2fc2d5375..ffc030d9ca 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -128,6 +128,8 @@ typedef uint32_t PCIConfigReadFunc(PCIDevice *pci_dev, typedef void PCIMapIORegionFunc(PCIDevice *pci_dev, int region_num, pcibus_t addr, pcibus_t size, int type); typedef void PCIUnregisterFunc(PCIDevice *pci_dev); +typedef void PCIMSINotify(PCIDevice *pci_dev, unsigned vector); +typedef void PCIMSIxNotify(PCIDevice *pci_dev, unsigned vector); typedef struct PCIIORegion { pcibus_t addr; /* current PCI mapping address. -1 means not mapped */ @@ -321,6 +323,10 @@ struct PCIDevice { /* Space to store MSIX table & pending bit array */ uint8_t *msix_table; uint8_t *msix_pba; + + PCIMSINotify *msi_notify; + PCIMSIxNotify *msix_notify; + /* MemoryRegion container for msix exclusive BAR setup */ MemoryRegion msix_exclusive_bar; /* Memory Regions for MSIX table and pending bit entries. */ diff --git a/include/hw/remote/iohub.h b/include/hw/remote/iohub.h index 0bf98e0d78..70d98b38d0 100644 --- a/include/hw/remote/iohub.h +++ b/include/hw/remote/iohub.h @@ -30,6 +30,7 @@ typedef struct RemoteIOHubState { unsigned int irq_level[REMOTE_IOHUB_NB_PIRQS]; ResampleToken token[REMOTE_IOHUB_NB_PIRQS]; QemuMutex irq_level_lock[REMOTE_IOHUB_NB_PIRQS]; + void (*intx_notify)(int pirq, unsigned vector); } RemoteIOHubState; int remote_iohub_map_irq(PCIDevice *pci_dev, int intx); diff --git a/hw/pci/msi.c b/hw/pci/msi.c index 47d2b0f33c..93f5e400cc 100644 --- a/hw/pci/msi.c +++ b/hw/pci/msi.c @@ -51,6 +51,8 @@ */ bool msi_nonbroken; +static void pci_msi_notify(PCIDevice *dev, unsigned int vector); + /* If we get rid of cap allocator, we won't need this. */ static inline uint8_t msi_cap_sizeof(uint16_t flags) { @@ -225,6 +227,8 @@ int msi_init(struct PCIDevice *dev, uint8_t offset, dev->msi_cap = config_offset; dev->cap_present |= QEMU_PCI_CAP_MSI; + dev->msi_notify = pci_msi_notify; + pci_set_word(dev->config + msi_flags_off(dev), flags); pci_set_word(dev->wmask + msi_flags_off(dev), PCI_MSI_FLAGS_QSIZE | PCI_MSI_FLAGS_ENABLE); @@ -307,7 +311,7 @@ bool msi_is_masked(const PCIDevice *dev, unsigned int vector) return mask & (1U << vector); } -void msi_notify(PCIDevice *dev, unsigned int vector) +static void pci_msi_notify(PCIDevice *dev, unsigned int vector) { uint16_t flags = pci_get_word(dev->config + msi_flags_off(dev)); bool msi64bit = flags & PCI_MSI_FLAGS_64BIT; @@ -332,6 +336,13 @@ void msi_notify(PCIDevice *dev, unsigned int vector) msi_send_message(dev, msg); } +void msi_notify(PCIDevice *dev, unsigned int vector) +{ + if (dev->msi_notify) { + dev->msi_notify(dev, vector); + } +} + void msi_send_message(PCIDevice *dev, MSIMessage msg) { MemTxAttrs attrs = {}; diff --git a/hw/pci/msix.c b/hw/pci/msix.c index ae9331cd0b..1c71e67f53 100644 --- a/hw/pci/msix.c +++ b/hw/pci/msix.c @@ -31,6 +31,8 @@ #define MSIX_ENABLE_MASK (PCI_MSIX_FLAGS_ENABLE >> 8) #define MSIX_MASKALL_MASK (PCI_MSIX_FLAGS_MASKALL >> 8) +static void pci_msix_notify(PCIDevice *dev, unsigned vector); + MSIMessage msix_get_message(PCIDevice *dev, unsigned vector) { uint8_t *table_entry = dev->msix_table + vector * PCI_MSIX_ENTRY_SIZE; @@ -334,6 +336,7 @@ int msix_init(struct PCIDevice *dev, unsigned short nentries, dev->msix_table = g_malloc0(table_size); dev->msix_pba = g_malloc0(pba_size); dev->msix_entry_used = g_malloc0(nentries * sizeof *dev->msix_entry_used); + dev->msix_notify = pci_msix_notify; msix_mask_all(dev, nentries); @@ -485,7 +488,7 @@ int msix_enabled(PCIDevice *dev) } /* Send an MSI-X message */ -void msix_notify(PCIDevice *dev, unsigned vector) +static void pci_msix_notify(PCIDevice *dev, unsigned vector) { MSIMessage msg; @@ -503,6 +506,13 @@ void msix_notify(PCIDevice *dev, unsigned vector) msi_send_message(dev, msg); } +void msix_notify(PCIDevice *dev, unsigned vector) +{ + if (dev->msix_notify) { + dev->msix_notify(dev, vector); + } +} + void msix_reset(PCIDevice *dev) { if (!msix_present(dev)) { diff --git a/hw/remote/iohub.c b/hw/remote/iohub.c index 547d597f0f..d28d9f3ce2 100644 --- a/hw/remote/iohub.c +++ b/hw/remote/iohub.c @@ -17,7 +17,9 @@ #include "qemu/thread.h" #include "hw/remote/machine.h" #include "hw/remote/iohub.h" +#include "hw/pci/msi.h" #include "qemu/main-loop.h" +#include "trace.h" void remote_iohub_init(RemoteIOHubState *iohub) { @@ -32,6 +34,8 @@ void remote_iohub_init(RemoteIOHubState *iohub) event_notifier_init_fd(&iohub->irqfds[pirq], -1); event_notifier_init_fd(&iohub->resamplefds[pirq], -1); } + + msi_nonbroken = true; } void remote_iohub_finalize(RemoteIOHubState *iohub) @@ -62,6 +66,9 @@ void remote_iohub_set_irq(void *opaque, int pirq, int level) QEMU_LOCK_GUARD(&iohub->irq_level_lock[pirq]); if (level) { + if (iohub->intx_notify) { + iohub->intx_notify(pirq, 0); + } if (++iohub->irq_level[pirq] == 1) { event_notifier_set(&iohub->irqfds[pirq]); } diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index ae375e69b9..2b28d465d5 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -50,6 +50,9 @@ #include "hw/pci/pci.h" #include "qemu/timer.h" #include "hw/remote/iommu.h" +#include "hw/pci/msi.h" +#include "hw/pci/msix.h" +#include "hw/remote/iohub.h" #define TYPE_VFU_OBJECT "x-vfio-user-server" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -81,6 +84,8 @@ struct VfuObject { int vfu_poll_fd; }; +static GHashTable *vfu_object_dev_table; + static void vfu_object_init_ctx(VfuObject *o, Error **errp); static void vfu_object_set_socket(Object *obj, Visitor *v, const char *name, @@ -347,6 +352,54 @@ static void vfu_object_register_bars(vfu_ctx_t *vfu_ctx, PCIDevice *pdev) } } +static void vfu_object_intx_notify(int pci_devfn, unsigned vector) +{ + vfu_ctx_t *vfu_ctx = g_hash_table_lookup(vfu_object_dev_table, + (void *)(uint64_t)pci_devfn); + + if (vfu_ctx) { + vfu_irq_trigger(vfu_ctx, vector); + } +} + +static void vfu_object_msi_notify(PCIDevice *pci_dev, unsigned vector) +{ + vfu_object_intx_notify(pci_dev->devfn, vector); +} + +static int vfu_object_setup_irqs(vfu_ctx_t *vfu_ctx, PCIDevice *pci_dev) +{ + RemoteMachineState *machine = REMOTE_MACHINE(current_machine); + RemoteIOHubState *iohub = &machine->iohub; + int ret; + + ret = vfu_setup_device_nr_irqs(vfu_ctx, VFU_DEV_INTX_IRQ, 1); + if (ret < 0) { + return ret; + } + + iohub->intx_notify = vfu_object_intx_notify; + + ret = 0; + if (msix_nr_vectors_allocated(pci_dev)) { + ret = vfu_setup_device_nr_irqs(vfu_ctx, VFU_DEV_MSIX_IRQ, + msix_nr_vectors_allocated(pci_dev)); + + pci_dev->msix_notify = vfu_object_msi_notify; + } else if (msi_nr_vectors_allocated(pci_dev)) { + ret = vfu_setup_device_nr_irqs(vfu_ctx, VFU_DEV_MSI_IRQ, + msi_nr_vectors_allocated(pci_dev)); + + pci_dev->msi_notify = vfu_object_msi_notify; + } + + if (ret < 0) { + return ret; + } + + return 0; +} + /* * TYPE_VFU_OBJECT depends on the availability of the 'socket' and 'device' * properties. It also depends on devices instantiated in QEMU. These @@ -437,6 +490,13 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) vfu_object_register_bars(o->vfu_ctx, o->pci_dev); + ret = vfu_object_setup_irqs(o->vfu_ctx, o->pci_dev); + if (ret < 0) { + error_setg(errp, "vfu: Failed to setup interrupts for %s", + o->device); + goto fail; + } + ret = vfu_realize_ctx(o->vfu_ctx); if (ret < 0) { error_setg(errp, "vfu: Failed to realize device %s- %s", @@ -450,6 +510,9 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) goto fail; } + g_hash_table_insert(vfu_object_dev_table, + (void *)(uint64_t)o->pci_dev->devfn, o->vfu_ctx); + qemu_set_fd_handler(o->vfu_poll_fd, vfu_object_attach_ctx, NULL, o); return; @@ -504,9 +567,18 @@ static void vfu_object_finalize(Object *obj) remote_iommu_free(o->pci_dev); } + if (o->pci_dev && + g_hash_table_lookup(vfu_object_dev_table, + (void *)(uint64_t)o->pci_dev->devfn)) { + g_hash_table_remove(vfu_object_dev_table, + (void *)(uint64_t)o->pci_dev->devfn); + } + o->pci_dev = NULL; if (!k->nr_devs && !k->daemon) { + g_hash_table_destroy(vfu_object_dev_table); + vfu_object_dev_table = NULL; qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); } @@ -525,6 +597,8 @@ static void vfu_object_class_init(ObjectClass *klass, void *data) /* Later determine how to detect a daemon */ k->daemon = false; + vfu_object_dev_table = g_hash_table_new_full(NULL, NULL, NULL, NULL); + object_class_property_add(klass, "socket", "SocketAddress", NULL, vfu_object_set_socket, NULL, NULL); object_class_property_set_description(klass, "socket", diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 847d50d88f..c167b3c7a5 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -12,3 +12,4 @@ vfu_dma_unregister(uint64_t gpa) "vfu: unregistering GPA 0x%"PRIx64"" vfu_bar_register(int i, uint64_t addr, uint64_t size) "vfu: BAR %d: addr 0x%"PRIx64" size 0x%"PRIx64"" vfu_bar_rw_enter(const char *op, uint64_t addr) "vfu: %s request for BAR address 0x%"PRIx64"" vfu_bar_rw_exit(const char *op, uint64_t addr) "vfu: Finished %s of BAR address 0x%"PRIx64"" +vfu_interrupt(int pirq) "vfu: sending interrupt to device - PIRQ %d" From patchwork Wed Dec 15 15:35:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 023DCC433FE for ; Wed, 15 Dec 2021 15:51:24 +0000 (UTC) Received: from localhost ([::1]:43060 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWYB-0001am-Tt for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 10:50:39 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43278) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKe-0006bB-No for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:40 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:15566) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKb-00006j-8t for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:40 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEWeMs023139; Wed, 15 Dec 2021 15:36:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=a3a4s641EqWZclBacc/JHp8P7eQoqD6hY1zM7rlLmTQ=; b=iYbyKLp6HvBFxs4Wu9pJm3yzHCAYeIGyk8B88IvOLRD9lYIXZxj98Pg5Vi7nzIZsr1cI sbKt+ZyugKUL53J57YpZ2Hmw8Vpg2j5roagj1r4JXvbn99jy4SBEFh73ZqVaGEVjITKf 8qUYuWl76ejsGezGrZ2J8p+0OqjS8ymD6M4LV1bLOcBPM8vwLcfBSUaU7EUEj6ClVsea veYFIMn0m39HtFtqf/ljypUT9+x8VkorTFEg4/xSDb5VJQ/E0rehYGXKa91IKF0QK7D5 WEuuOHnNO4FB/UkQuIc3sGVh4nNBWATqZ2y2rrEgzUtTk9dG72AM2Mcs9fbewpo4pFQM Tg== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3py778y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:33 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUW6d094227; Wed, 15 Dec 2021 15:36:32 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2177.outbound.protection.outlook.com [104.47.55.177]) by aserp3020.oracle.com with ESMTP id 3cxmrc0uc4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EnItfu41dh/hwSsP59rnlz0nvuax+8XICXjXyxHTLrdDQ4Qob5SVe8g10LvJ8N/kxyUDJsQ14deBcfgZN8GzQLQEe2CwE4LGkGoE/IGQwrBpwhCw64coquPCQW0kTGjBFtNkxNpph5NuHnFW2Kn5SrsW6rmj9xBol4gcKC0pUTCNzPbflJfkHOqKsSW73nZCdqLTyErWDzpgxZexv5sNiHlBdpTJHaYS+HV5fA1P2VVDkFs2emQptqyOjydYU6Hpet4f3/5/li7kfcSbxMK7r9rWAtlF20SWUs7EMjgPhVD2tfTW0+nN4eLfcJC3YzP1U4OUzwmWRx59z0sHcJ7kBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a3a4s641EqWZclBacc/JHp8P7eQoqD6hY1zM7rlLmTQ=; b=aNvsEPixa1ggincDww0uHbDPzbr+8k5F7oEDMQW/Xrk5G0qxcxj9R7/cZgMeftXFS8/Uu4XRIHF+tS2qguaGuGcqwvNuEykU2jN4Sglp+r/2TAVjLwDL+6hoEOr9E+f2uDcB8MD++6OKlrY22Kwyaf8Hq1cIop5btshg7YhcuXWCAA5GOqF5B5ck9K9bvYkTECuNbwgilibPzJYc3gCMbgFoT50hWsuCfKklD8bCO9mSQfZ7p6lQRbZ9TfCv0MIBWz5RAFlytNcG1VWOAYYLqWT1EmIPZlZ/uYr3xh2HWAawtdLNdRFIzsANhWVdNX17a5Xa6Ot4iqpXNz++ZVcCYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a3a4s641EqWZclBacc/JHp8P7eQoqD6hY1zM7rlLmTQ=; b=ADvjqWdX15kkkQlwExqXC+GygWaPUgPo+apkh+ruDm9ZbVfY3BpAD2XqR667+yCX1HtLQM2FAPK9e8zzbLIRcgJOLsKnHtw4g3xV4J1SiBKnr27GD26NzRobjODu5i1dNdbHRSFRipAS4vOV0wb89POPXceVnFecDgKWmREv4/c= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by BLAPR10MB5282.namprd10.prod.outlook.com (2603:10b6:208:30e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4778.17; Wed, 15 Dec 2021 15:36:29 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:29 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 13/14] vfio-user: register handlers to facilitate migration Date: Wed, 15 Dec 2021 10:35:37 -0500 Message-Id: <5f3cfae32d974c9a7ef3a1a9a2897557b9106de6.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e562d64e-c6b5-4cb1-0e66-08d9bfe0a9db X-MS-TrafficTypeDiagnostic: BLAPR10MB5282:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1247; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wd8GMTtN2ZQXXe0S4G7scX9l8bhNrEaeENDvrG9UZXQTHgmcnly5ANiM+364Kwg07cvFVHYUzVdjuAiQkPvlstXVWiEX/++puApMwdy4qEp3O8uqOAbQZjNBQtKVT91lOvCsXEaTIuRTkk6E4NlDmDGwL0N+3LdTzjmmNfZQ58sbE9fwse9zpjso+bIGHppWEP5io0drWOWBzManzIv5F/Sdu9A5LKsEZmRV0rh/VnuNMSwdoH/mytwAHXIHF2dxcGw11eJihswuKTfygB1mEODSPE15nECAQhZes+4Iw+suYncWYgnHb1xKAGx0duqTcZCvJ5dOEmlRqUjVXZlLk/XBxe0zUJ9q+iq05ZzFCHBR5cCcsuaqxYf16C/sq+kWzRY5dgra3Dii3jPTKiIpVxt0tWCXLcyAFZhDuq0W3TVHlOctuta5NH9IJsk7Je8wNyZj1NUCJm9MCP8bSfY20GNgJtIN1oj69DahAqoqzghDCnhX15exdO+BdgxdFNcmwa7hsFteVumgft///PlGMVHH7TS8xClXU4ijzMUkkAwPHggNc2hzR0QF0r69NgDiYloOKBYGcK/4lzxdRPBf6bs5SzsQu6vz/pf5y7aqskoLfsm0n6dEKvIfIssPTHmjE5AbN7DpFURaAb4R4Zf7rSMW80uF99UokU4oE0J/eQL23JtsBTouvu2FJeQdrPJqeNyquO/TSCy6ElLlzbDz5g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(2906002)(86362001)(36756003)(316002)(52116002)(6512007)(508600001)(2616005)(6506007)(4326008)(6666004)(6916009)(6486002)(7416002)(107886003)(83380400001)(8676002)(26005)(8936002)(38100700002)(38350700002)(30864003)(66946007)(5660300002)(66556008)(66476007)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KOs2JtGYssjbW6q+YdpttBelSJeRG8odg3TosNwTxMpMXLogVGbLvGAViy3fCZLfzhGCi1HKLWhzLR4xsK1CipZr1d+iKD7ihvrOoFWKRLKpkFL5UVU4PxyR2j5y2od861QsrPrEphkGW8F9h7cuff3EJdCH4LhslWm8RQH5XEviHGnb8Ps2FS8Oo+1qoM/nU/8ZgMAEBge2ETA2tH/wBINqI4oGh6nOgcJsi4tiquTTWFzcC7zugqH30QFdY9rlOBQyFdNO7UMZyW/x+zN/8pkwUD+/dh+Z27PrJpq0xMaTrga/4ePVQn0t5i9Icdx0wECjpcXZrum4xmMvQxYNtZ/FkmuyOtXyLV5UxdwtPIjdc/9Ctheo80aBBp9OeTK4IPCSp7EHIoFQ7z5HxzpC/qFiUbSdDV8WqENjEMRxK8sskoKqHV9DIq2ZADJMlmh//eLP4+HXu/FpDxZBTA9YB7Samf5eRulzQpxKOX2viBcaCB9fnGOV0tn6eP5ri05n1BPUZu+lT83/pR/7k7oB2s/YqbUCHR2HHu43tDrfzkPT10BxQD+xn6B7NK4PQB2lGxrAki+U2f2w42b52qLhKiFTsYhgGGSmV1wjfgnuGGnMwTk1hsqLYyQRwV3+eVzuyP89qJoyvB0FysM54cO/KlfR5Hk+lveVwWumNQwbQz2sksEU38O2DU44nKQuZ4tSWH08OvPWFBew/x3K0ErK5SnFo9ctYawUxWkTnkgvfsurOtJ6TzV4zhmtJ0f5uvJ/D+QljlQPPGoUBKkLDuYhVC1gYXz9zZSgd6+BF9XNTv7Cv16pEpCFNMvm/IFR9uOdAjoJf7qCnE/fuwBTfJoDp6uoeE1g1IkzyJ4a3vXH/HUHLhtIy60MlnnCGt4RNc2vU//MNNcecmwH28rJiN4Kcit9duRogfdnJOlUMgUQ9AkgKfClNL7/oIdcdBoHRjqXNfUVqMJ+IRQv/et47p8DhYgcnezFS2J3HWBBaiI+3LTOxYnjj+NxYBukk+hTT6C1ZJxJ02XO4Vr07FCImAknk5fQukYkJUdbbHPWK1vpzT1dzNcIqAKKaZ45Nu15bF5y+kZ+tGSs6WEug1XzfEA/vWNm7dg8LjS1ldJqRiQh65bG9thGRHobgCIgPWM+8dQQcUlBZEdCwOMT0KDhdMfu5MhA5LY04SmrYx8V1lZhTM33ajeoBq11bbcEFCZcRAD4vHNGEi8dWQyUS7DxpXNqRyXg8qgf0O2bq6Ah9NVKewCRL2v7LkN899BV1kVfceo6n8r5eYkktTGRbsda7RB0hxxN7MYwdwgRkwu8jzKczDYJN5Wj05ksTCd/LypXPQe6ZBEDopRzze1HxOYmRYVcQQEyJ1bjy7FrDu8J09dO1OU46lPph2nCTT9C5H6xSWpd8H5NJI2fiOoSFiZJzUNpKn65t5JI9mZX2JttAA9J6aC4TRRmn0htBNmylPCNRduZtIUyGAygK6pWmovFLvT6/X0xorRqp+Dy53AxfEZOn8ufo6psnBFvTDqsmLoWKqY9eA7G9BQqYMMbDrYcVzUajRIAG7MtarC0C8AZfMObyPYZoUdrMvxXQKbuvUu2doJpdqxqGbNgZabxHeUb4W9ODSoV3lglCoVuUm7OH2kg1GQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e562d64e-c6b5-4cb1-0e66-08d9bfe0a9db X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:29.4952 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: b/x8eQ4/QmRBP60hbxTsNmo1/D/4LqC7jSuUzqI9Heon2QCJX2UztChuZt0LAjMi8YM9UhKvDvAa8JKo2rmJqA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5282 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: X8RSICUC5YjjF2wJtCASiTsx3vsocnWn X-Proofpoint-GUID: X8RSICUC5YjjF2wJtCASiTsx3vsocnWn Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Store and load the device's state during migration. use libvfio-user's handlers for this purpose Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/migration/vmstate.h | 2 + migration/savevm.h | 2 + hw/remote/vfio-user-obj.c | 323 ++++++++++++++++++++++++++++++++++++ migration/savevm.c | 73 ++++++++ migration/vmstate.c | 19 +++ 5 files changed, 419 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 017c03675c..68bea576ea 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -1165,6 +1165,8 @@ extern const VMStateInfo vmstate_info_qlist; #define VMSTATE_END_OF_LIST() \ {} +uint64_t vmstate_vmsd_size(PCIDevice *pci_dev); + int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, void *opaque, int version_id); int vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd, diff --git a/migration/savevm.h b/migration/savevm.h index 6461342cb4..8007064ff2 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -67,5 +67,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis); int qemu_load_device_state(QEMUFile *f); int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, bool in_postcopy, bool inactivate_disks); +int qemu_remote_savevm(QEMUFile *f, DeviceState *dev); +int qemu_remote_loadvm(QEMUFile *f); #endif diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 2b28d465d5..cc0b82445b 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -53,6 +53,11 @@ #include "hw/pci/msi.h" #include "hw/pci/msix.h" #include "hw/remote/iohub.h" +#include "migration/qemu-file.h" +#include "migration/savevm.h" +#include "migration/vmstate.h" +#include "migration/global_state.h" +#include "block/block.h" #define TYPE_VFU_OBJECT "x-vfio-user-server" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -82,6 +87,35 @@ struct VfuObject { PCIDevice *pci_dev; int vfu_poll_fd; + + /* + * vfu_mig_buf holds the migration data. In the remote server, this + * buffer replaces the role of an IO channel which links the source + * and the destination. + * + * Whenever the client QEMU process initiates migration, the remote + * server gets notified via libvfio-user callbacks. The remote server + * sets up a QEMUFile object using this buffer as backend. The remote + * server passes this object to its migration subsystem, which slurps + * the VMSD of the device ('devid' above) referenced by this object + * and stores the VMSD in this buffer. + * + * The client subsequetly asks the remote server for any data that + * needs to be moved over to the destination via libvfio-user + * library's vfu_migration_callbacks_t callbacks. The remote hands + * over this buffer as data at this time. + * + * A reverse of this process happens at the destination. + */ + uint8_t *vfu_mig_buf; + + uint64_t vfu_mig_buf_size; + + uint64_t vfu_mig_buf_pending; + + QEMUFile *vfu_mig_file; + + vfu_migr_state_t vfu_state; }; static GHashTable *vfu_object_dev_table; @@ -125,6 +159,272 @@ static void vfu_object_set_device(Object *obj, const char *str, Error **errp) vfu_object_init_ctx(o, errp); } +/** + * Migration helper functions + * + * vfu_mig_buf_read & vfu_mig_buf_write are used by QEMU's migration + * subsystem - qemu_remote_loadvm & qemu_remote_savevm. loadvm/savevm + * call these functions via QEMUFileOps to load/save the VMSD of a + * device into vfu_mig_buf + * + */ +static ssize_t vfu_mig_buf_read(void *opaque, uint8_t *buf, int64_t pos, + size_t size, Error **errp) +{ + VfuObject *o = opaque; + + if (pos > o->vfu_mig_buf_size) { + size = 0; + } else if ((pos + size) > o->vfu_mig_buf_size) { + size = o->vfu_mig_buf_size - pos; + } + + memcpy(buf, (o->vfu_mig_buf + pos), size); + + return size; +} + +static ssize_t vfu_mig_buf_write(void *opaque, struct iovec *iov, int iovcnt, + int64_t pos, Error **errp) +{ + VfuObject *o = opaque; + uint64_t end = pos + iov_size(iov, iovcnt); + int i; + + if (end > o->vfu_mig_buf_size) { + o->vfu_mig_buf = g_realloc(o->vfu_mig_buf, end); + } + + for (i = 0; i < iovcnt; i++) { + memcpy((o->vfu_mig_buf + o->vfu_mig_buf_size), iov[i].iov_base, + iov[i].iov_len); + o->vfu_mig_buf_size += iov[i].iov_len; + o->vfu_mig_buf_pending += iov[i].iov_len; + } + + return iov_size(iov, iovcnt); +} + +static int vfu_mig_buf_shutdown(void *opaque, bool rd, bool wr, Error **errp) +{ + VfuObject *o = opaque; + + o->vfu_mig_buf_size = 0; + + g_free(o->vfu_mig_buf); + + o->vfu_mig_buf = NULL; + + o->vfu_mig_buf_pending = 0; + + return 0; +} + +static const QEMUFileOps vfu_mig_fops_save = { + .writev_buffer = vfu_mig_buf_write, + .shut_down = vfu_mig_buf_shutdown, +}; + +static const QEMUFileOps vfu_mig_fops_load = { + .get_buffer = vfu_mig_buf_read, + .shut_down = vfu_mig_buf_shutdown, +}; + +/** + * handlers for vfu_migration_callbacks_t + * + * The libvfio-user library accesses these handlers to drive the migration + * at the remote end, and also to transport the data stored in vfu_mig_buf + * + */ +static void vfu_mig_state_stop_and_copy(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + int ret; + + if (!o->vfu_mig_file) { + o->vfu_mig_file = qemu_fopen_ops(o, &vfu_mig_fops_save, false); + } + + ret = qemu_remote_savevm(o->vfu_mig_file, DEVICE(o->pci_dev)); + if (ret) { + qemu_file_shutdown(o->vfu_mig_file); + o->vfu_mig_file = NULL; + return; + } + + qemu_fflush(o->vfu_mig_file); +} + +static void vfu_mig_state_running(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + VfuObjectClass *k = VFU_OBJECT_GET_CLASS(OBJECT(o)); + static int migrated_devs; + Error *local_err = NULL; + int ret; + + /** + * TODO: move to VFU_MIGR_STATE_RESUME handler. Presently, the + * VMSD data from source is not available at RESUME state. + * Working on a fix for this. + */ + if (!o->vfu_mig_file) { + o->vfu_mig_file = qemu_fopen_ops(o, &vfu_mig_fops_load, false); + } + + ret = qemu_remote_loadvm(o->vfu_mig_file); + if (ret) { + error_setg(&error_abort, "vfu: failed to restore device state"); + return; + } + + qemu_file_shutdown(o->vfu_mig_file); + o->vfu_mig_file = NULL; + + /* VFU_MIGR_STATE_RUNNING begins here */ + if (++migrated_devs == k->nr_devs) { + bdrv_invalidate_cache_all(&local_err); + if (local_err) { + error_report_err(local_err); + return; + } + + vm_start(); + } +} + +static void vfu_mig_state_stop(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + VfuObjectClass *k = VFU_OBJECT_GET_CLASS(OBJECT(o)); + static int migrated_devs; + + /** + * note: calling bdrv_inactivate_all() is not the best approach. + * + * Ideally, we would identify the block devices (if any) indirectly + * linked (such as via a scsi-hd device) to each of the migrated devices, + * and inactivate them individually. This is essential while operating + * the server in a storage daemon mode, with devices from different VMs. + * + * However, we currently don't have this capability. As such, we need to + * inactivate all devices at the same time when migration is completed. + */ + if (++migrated_devs == k->nr_devs) { + vm_stop(RUN_STATE_PAUSED); + bdrv_inactivate_all(); + } +} + +static int vfu_mig_transition(vfu_ctx_t *vfu_ctx, vfu_migr_state_t state) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + if (o->vfu_state == state) { + return 0; + } + + switch (state) { + case VFU_MIGR_STATE_RESUME: + break; + case VFU_MIGR_STATE_STOP_AND_COPY: + vfu_mig_state_stop_and_copy(vfu_ctx); + break; + case VFU_MIGR_STATE_STOP: + vfu_mig_state_stop(vfu_ctx); + break; + case VFU_MIGR_STATE_PRE_COPY: + break; + case VFU_MIGR_STATE_RUNNING: + if (!runstate_is_running()) { + vfu_mig_state_running(vfu_ctx); + } + break; + default: + warn_report("vfu: Unknown migration state %d", state); + } + + o->vfu_state = state; + + return 0; +} + +static uint64_t vfu_mig_get_pending_bytes(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + return o->vfu_mig_buf_pending; +} + +static int vfu_mig_prepare_data(vfu_ctx_t *vfu_ctx, uint64_t *offset, + uint64_t *size) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + if (offset) { + *offset = 0; + } + + if (size) { + *size = o->vfu_mig_buf_size; + } + + return 0; +} + +static ssize_t vfu_mig_read_data(vfu_ctx_t *vfu_ctx, void *buf, + uint64_t size, uint64_t offset) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + if (offset > o->vfu_mig_buf_size) { + return -1; + } + + if ((offset + size) > o->vfu_mig_buf_size) { + warn_report("vfu: buffer overflow - check pending_bytes"); + size = o->vfu_mig_buf_size - offset; + } + + memcpy(buf, (o->vfu_mig_buf + offset), size); + + o->vfu_mig_buf_pending -= size; + + return size; +} + +static ssize_t vfu_mig_write_data(vfu_ctx_t *vfu_ctx, void *data, + uint64_t size, uint64_t offset) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + uint64_t end = offset + size; + + if (end > o->vfu_mig_buf_size) { + o->vfu_mig_buf = g_realloc(o->vfu_mig_buf, end); + o->vfu_mig_buf_size = end; + } + + memcpy((o->vfu_mig_buf + offset), data, size); + + return size; +} + +static int vfu_mig_data_written(vfu_ctx_t *vfu_ctx, uint64_t count) +{ + return 0; +} + +static const vfu_migration_callbacks_t vfu_mig_cbs = { + .version = VFU_MIGR_CALLBACKS_VERS, + .transition = &vfu_mig_transition, + .get_pending_bytes = &vfu_mig_get_pending_bytes, + .prepare_data = &vfu_mig_prepare_data, + .read_data = &vfu_mig_read_data, + .data_written = &vfu_mig_data_written, + .write_data = &vfu_mig_write_data, +}; + static void vfu_object_ctx_run(void *opaque) { VfuObject *o = opaque; @@ -425,6 +725,7 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) ERRP_GUARD(); DeviceState *dev = NULL; vfu_pci_type_t pci_type = VFU_PCI_TYPE_CONVENTIONAL; + uint64_t migr_regs_size, migr_size; int ret; if (o->vfu_ctx || !o->socket || !o->device || @@ -497,6 +798,26 @@ static void vfu_object_init_ctx(VfuObject *o, Error **errp) goto fail; } + migr_regs_size = vfu_get_migr_register_area_size(); + migr_size = migr_regs_size + vmstate_vmsd_size(o->pci_dev); + + ret = vfu_setup_region(o->vfu_ctx, VFU_PCI_DEV_MIGR_REGION_IDX, + migr_size, NULL, + VFU_REGION_FLAG_RW, NULL, 0, -1, 0); + if (ret < 0) { + error_setg(errp, "vfu: Failed to register migration BAR %s- %s", + o->device, strerror(errno)); + goto fail; + } + + ret = vfu_setup_device_migration_callbacks(o->vfu_ctx, &vfu_mig_cbs, + migr_regs_size); + if (ret < 0) { + error_setg(errp, "vfu: Failed to setup migration %s- %s", + o->device, strerror(errno)); + goto fail; + } + ret = vfu_realize_ctx(o->vfu_ctx); if (ret < 0) { error_setg(errp, "vfu: Failed to realize device %s- %s", @@ -542,6 +863,8 @@ static void vfu_object_init(Object *obj) } o->vfu_poll_fd = -1; + + o->vfu_state = VFU_MIGR_STATE_STOP; } static void vfu_object_finalize(Object *obj) diff --git a/migration/savevm.c b/migration/savevm.c index d59e976d50..69b7ea8b09 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1605,6 +1605,49 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) return ret; } +static SaveStateEntry *find_se_from_dev(DeviceState *dev) +{ + SaveStateEntry *se; + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (se->opaque == dev) { + return se; + } + } + + return NULL; +} + +int qemu_remote_savevm(QEMUFile *f, DeviceState *dev) +{ + SaveStateEntry *se; + int ret = 0; + + se = find_se_from_dev(dev); + if (!se) { + return -ENODEV; + } + + if (!se->vmsd || !vmstate_save_needed(se->vmsd, se->opaque)) { + return ret; + } + + save_section_header(f, se, QEMU_VM_SECTION_FULL); + + ret = vmstate_save(f, se, NULL); + if (ret) { + qemu_file_set_error(f, ret); + return ret; + } + + save_section_footer(f, se); + + qemu_put_byte(f, QEMU_VM_EOF); + qemu_fflush(f); + + return 0; +} + void qemu_savevm_live_state(QEMUFile *f) { /* save QEMU_VM_SECTION_END section */ @@ -2445,6 +2488,36 @@ qemu_loadvm_section_start_full(QEMUFile *f, MigrationIncomingState *mis) return 0; } +int qemu_remote_loadvm(QEMUFile *f) +{ + uint8_t section_type; + int ret = 0; + + while (true) { + section_type = qemu_get_byte(f); + + ret = qemu_file_get_error(f); + if (ret) { + break; + } + + switch (section_type) { + case QEMU_VM_SECTION_FULL: + ret = qemu_loadvm_section_start_full(f, NULL); + if (ret < 0) { + break; + } + break; + case QEMU_VM_EOF: + return ret; + default: + return -EINVAL; + } + } + + return ret; +} + static int qemu_loadvm_section_part_end(QEMUFile *f, MigrationIncomingState *mis) { diff --git a/migration/vmstate.c b/migration/vmstate.c index 05f87cdddc..83f8562792 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -63,6 +63,25 @@ static int vmstate_size(void *opaque, const VMStateField *field) return size; } +uint64_t vmstate_vmsd_size(PCIDevice *pci_dev) +{ + DeviceClass *dc = DEVICE_GET_CLASS(DEVICE(pci_dev)); + const VMStateField *field = NULL; + uint64_t size = 0; + + if (!dc->vmsd) { + return 0; + } + + field = dc->vmsd->fields; + while (field && field->name) { + size += vmstate_size(pci_dev, field); + field++; + } + + return size; +} + static void vmstate_handle_alloc(void *ptr, const VMStateField *field, void *opaque) { From patchwork Wed Dec 15 15:35:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12678637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 01E35C433F5 for ; Wed, 15 Dec 2021 16:08:57 +0000 (UTC) Received: from localhost ([::1]:58926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxWps-0006hh-RQ for qemu-devel@archiver.kernel.org; Wed, 15 Dec 2021 11:08:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43342) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKj-0006mJ-41 for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:45 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:28434) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxWKg-000081-AB for qemu-devel@nongnu.org; Wed, 15 Dec 2021 10:36:44 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1BFEMpeo023149; Wed, 15 Dec 2021 15:36:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=/tT6da/bYxsNG8JAdvbdg5Qc+57FS8nthI5B8zzLovw=; b=vUXw9H+ZhLbQa9tKu400K4RWC9wSGGzXV6eUOZfSMvOFaOjf4SxvvRFy6Q5vAA0ICmtI Iy61YEZXOAJKmNHpnz2StOGhNOG6g9uEEiCDt6ppIYzRq36gg3cggnLbLo9aKPvXi/Hm Pe6wun38VPfw8lYtJYd1yhz4I2Wg+5QESXFoTSfGk2RF66jCMsYCaRmClk4VfZRzcmUL 0detaH64HUIosUELykEtJqq409bwZqGPfZD4S34IPxsVPdOGaSu5wKY6ysxoCxigpZta 91C0w9lf2XHlrO1SAdro9X3k4f0Is1hLLVt+N044Oar7DvMKxkmpUW6Oq8IdbAu+wCQX GQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3cx3py779c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:38 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1BFFUSaX083829; Wed, 15 Dec 2021 15:36:37 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2171.outbound.protection.outlook.com [104.47.56.171]) by userp3030.oracle.com with ESMTP id 3cvh40b3du-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Dec 2021 15:36:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xyip5rC0roq0onT1kWad1zSPSCi7RQiErWh3ra9I+q/1/0EfvFIV11Y0y2g+JBB7PtbhWCL8cu55dQ8CJG0AuiDrfwB8XvhniymAXiOeIqoqxUaLOQsAFJu7L8HGgMZiCM/FTsWfFqcR+TomiyHYr7a1UduGLpaq4H5DPzpClr7CfMwDnKZTQTvvLF3mqZVVpTUUpMJflnRhPWycBK0EH2txEGZssH93pNX0L7J5etQ1jM1hc1ZxX2qXzsqird1/paxGLBamdbHBbzadlQfbpmz5HkLjiKIIbmjNkC30cSmy9KLOJh/KRre7WQ6G+TQfaHeUPVM7zQwZAfeYhwNogg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/tT6da/bYxsNG8JAdvbdg5Qc+57FS8nthI5B8zzLovw=; b=AQ35fV4f2NsTHshOXWU7Ug0j0AxRIHCazpmZCAjJDQ99xLuCcgQ9WhZEuC/0zzZ90krphwHoxNMsQ4RN4wUuOq98HgJLQ5zC/uPg0zVkgE/fPpHkl+blzqR+CajABAsBSwX2EMAQQwCHlouwKvLRoJmBb+OMIDl2tfJBKY+g6KE7qEMPAOVXzb6Sk5IARSd1UkV/4W/jxdZgDOiMak1q8w823Dg6Po4ktvwkD6aJbus+n/X8elaT2p5PWFXT65kHiHjnX42MMhYsDCHFz06YX10h95V4FeLp4N2ew7VSC5YXxFB/5NeKaaVvHzCN25dwv8QPkxjxpvITQh5oBMy+3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/tT6da/bYxsNG8JAdvbdg5Qc+57FS8nthI5B8zzLovw=; b=uKd9jGApyTR7Md29jcZKk0ZjqZafVr82r4u+nVKQJ7hZYX3augpv6QPQprUJHI6uJnDbg/TfxxrUzMEyJ047eyUm3q8uYmyArxRFm8n/hem2rW0DCx/42BDAyxTpCm3YDxbkiSa/HwZt9fQDntLGgWu5yF8+oS1DKJi+WA+OmO8= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4048.namprd10.prod.outlook.com (2603:10b6:208:181::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4801.14; Wed, 15 Dec 2021 15:36:32 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::b911:b919:6383:970d%5]) with mapi id 15.20.4778.018; Wed, 15 Dec 2021 15:36:32 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v4 14/14] vfio-user: avocado tests for vfio-user Date: Wed, 15 Dec 2021 10:35:38 -0500 Message-Id: <0ca430dd43d35ff2ae4667247f6d669a84e5ce99.1639549843.git.jag.raman@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: X-ClientProxiedBy: BYAPR11CA0105.namprd11.prod.outlook.com (2603:10b6:a03:f4::46) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 600b8c18-1c7f-4dad-5075-08d9bfe0abcd X-MS-TrafficTypeDiagnostic: MN2PR10MB4048:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8FYOGJSg0WfqO/FapQMWremauBvpuKKXq+J4yrHf3JGSBPh3zB/oVYMbFfzfk1rre4s6Yiz8onApHf1n/6cb5khR/+moB/ShziY5HW3csOXr4V3Zkn/ohWiUPACJ0GPdhuAV94ZJR3p6PmD92naAoXUuIm1CWiKgkk2JDhCBWurshMSpSW0b/AU9pDmLbtzLyKE9lEVb0vgWmZgI8Vfm5jCyfhXIcWnIPbeqaFAMXZzZhGp+3owxt3P/N1SQg/bsOnABi2lZbUoPu7/GiMNobVBo6U1RoRmK1b6TNZyRyzbM5xb4J97QMfY27IUWduFSgDNCoKMPCiuJU/Wj/fUHclQzGtf4KzWkFjXWUVMhIa2+dk824Mfaq1mjymuSkrY2d0QELCZ3g01gpvZpVehipUk5lqjL7kfp4GYwp9SnP9u1913aW+YCOR2rshAnGR4dlg+CwJRcrjrHHrolqVT15atjqHeOBFtKQbSd4cTTqSC6RViMQdQk35dkryeLg9/ic0hMqVb6ULdaRB6HtSKD66cCrn9a/sQ/WtF2d0CQ3BXuS2AjquFQsqgNojiCQFXBKWrXw8zp5rJAfGlpUL3GhyncohKZDwyK2cgx/BUJ49SycOuOdkWs2D5pPAwKkM+v+EKGTgc0NzjVe5FpfybtFrZAQIGJ6TGLATrDDtkiwKL3hjR/HKPZ6q/D9IFXU4gO68BVGsJC3huMEhi87Ogboe5FK9CbhKa+RolJiPyP9iWSY5bHVic1xSRwXWbna0lSIg2sX2GBqsMQwT9lwwnudbKgPAQAfbDg040R5tdiPb0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(508600001)(8936002)(5660300002)(7416002)(6506007)(6486002)(26005)(66476007)(4326008)(2906002)(66946007)(66556008)(107886003)(36756003)(186003)(2616005)(6916009)(38350700002)(6512007)(38100700002)(86362001)(316002)(8676002)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: v9VmGW4rGRnOUFyCHNZIwIpNQ5rRVk6UkT4YsV6bd2reHTUp8y4vn+atwBRzzwHPnIuiKbK4ko4SXevAUZzi8g7J32jszLYDwVdNZOay4ZhOv1Gt+cfVnLpP9kONAdKyGGqkP1+4wb2+7aAbiqoLb9W2lofzTckiYLE2cvO7q8oo7jkxqIkCcZKw+9TTw4bVfz4HA0DGtFS64VAekGxFMIvIxmqV/dpEb/cM0M5vXdybtMl/a0skYxJwN5pVmgQAvLK45elME2BA4bz7sdvi6VgWihNu+Ov1t672L4xKEJITMmzoJh8hDpeZGQJ6Wn5GidZ30O1WHyku/9V5pAWpUENxAq3mc1zrhec+22WVIIg7H9wCCkDrQQr3hcbsTTYLm2Rm/4aCb+7cve09GAO9jqlSefW/RtVGZYErbPCbQ6eKZTwVf9Ka4M4MKRH/TvUpt1FQWSyuvWPoNZaQwctWJxcocHUo9ldgPU9qhb7C3z/A+9kegukXSJ72cGvm5HU9V0ycFUDh1xkJ1Xcm3zMmvdzpfdxyQ+61JJN3pLrdT/c+c3Dqv8ajB3e2YCQhnS2E6aVw/KVIsg9ChWQaUnGuJvMZXsdfv00dG3Hh5rxdWXVa5l1Vm2A0rudhxvjb2RhSpexpQ0kugq7eKYk9qasRUuK2PzMwjJFufAb4HXropHMBTL8yry5amqikKa+YAv3RJxGMWEJCrgJ8YQkASaQtxCoar9LLCV8Pyjt20aMVyWIw1tje/rmh5tBXtjpWeUm+N3sLm6C2Om2hpm+cgY2oSpsPQWLo2T1/iT95agqc/yJnwk5FQIIqGW0u5FvTEIQxldwxMM/Loj9jgf5qIVHAkc9vl7YKgPLzF0wTTwCzYw7o39lV6b6a7ak7VnoVMUVdc2BZJ3zmyIAnqXVAJp20XBsBHIUoCfbS9xDsKRqh1kQj9Yp8VuBz/20CiYdiwckq/RNt1IADUQBTbyuD2oTWc1nSnY1jmISH5E5lQbdzN0+mIzw6MRj35VVAnjndPBMtYshlVvkfOFXFNqZK59N+fZa2N+rsmcqwtn2Y+MbHQtP6jna030N7SyNfAhjW3xRI+yKLHqckNLIRj4//xn87cCKmuUrNodfduPdX4CRFUAR9tGuCK3o0TjGETUVBV5zQudJY69Szb1A6uXhOqKB3nYK8tvS1+7OFiz2pFZGwvxBQP2hyzlsj+rpYe2rZGSRp4byTjFrAKjswWPKsJWvsDsiWw4frVx2GurqbmeiV1JVSLbES6W5mkGSMxLPLzuCw9XVJPX5qlpdUsduIdFUSaLtZSn3R7xC2bQTDAOUctHPaVRFqBg5MQ18/NDKG1ZxqYOBUvaDOBSRK/e4RZOGrInW7fcMZGRCCW1Uhkdpwii2Ta3yG70kanYHlpShW0/jJ6iNkF9oIdQlrciQyXHXj12wXLJNvQ6cM5+5d+aXm8uQKnfjmMpT/rcRb/yJ7WgFxlGgzF14EkJJsLX576qMLFZCFpAa88d0zj4S7jMY1BH9oSVZuzCZn+THC/GXmGLZUtQFXtEOuxcjeppKLZvmrfVMQZu2L6CGj70wF07+svwmorVKm5RuuV3Ff60zx3azePX+2y4xK+sopsjGCcw6/G7YxVGXGQsopxGnlBPv39WQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 600b8c18-1c7f-4dad-5075-08d9bfe0abcd X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Dec 2021 15:36:32.8387 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: N28bQGNSBDYulS6RPdTWFxrUHkteOicKfujHLilK6rLwObFEY1nr9DH+6bjtF5h+8D8yHxkOMlQlT4dRQMZ7RA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4048 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10198 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112150088 X-Proofpoint-ORIG-GUID: 1ZNUKBAGaDIlmn6Q7LssKIM3sLat6vC5 X-Proofpoint-GUID: 1ZNUKBAGaDIlmn6Q7LssKIM3sLat6vC5 Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, thuth@redhat.com, jag.raman@oracle.com, bleal@redhat.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, philmd@redhat.com, wainersm@redhat.com, alex.williamson@redhat.com, thanos.makatos@nutanix.com, marcandre.lureau@gmail.com, stefanha@redhat.com, crosa@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Avocado tests for libvfio-user in QEMU - tests startup, hotplug and migration of the server object Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- MAINTAINERS | 1 + tests/avocado/vfio-user.py | 225 +++++++++++++++++++++++++++++++++++++ 2 files changed, 226 insertions(+) create mode 100644 tests/avocado/vfio-user.py diff --git a/MAINTAINERS b/MAINTAINERS index 5dc67d79a1..385a4d8869 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3468,6 +3468,7 @@ F: subprojects/libvfio-user F: hw/remote/vfio-user-obj.c F: include/hw/remote/iommu.h F: hw/remote/iommu.c +F: tests/avocado/vfio-user.py EBPF: M: Jason Wang diff --git a/tests/avocado/vfio-user.py b/tests/avocado/vfio-user.py new file mode 100644 index 0000000000..376c02c41f --- /dev/null +++ b/tests/avocado/vfio-user.py @@ -0,0 +1,225 @@ +# vfio-user protocol sanity test +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + + +import os +import socket +import uuid + +from avocado_qemu import QemuSystemTest +from avocado_qemu import wait_for_console_pattern +from avocado_qemu import exec_command +from avocado_qemu import exec_command_and_wait_for_pattern + +from avocado.utils import network +from avocado.utils import wait + +class VfioUser(QemuSystemTest): + """ + :avocado: tags=vfiouser + """ + KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' + timeout = 20 + + @staticmethod + def migration_finished(vm): + res = vm.command('query-migrate') + if 'status' in res: + return res['status'] in ('completed', 'failed') + else: + return False + + def _get_free_port(self): + port = network.find_free_port() + if port is None: + self.cancel('Failed to find a free port') + return port + + def validate_vm_launch(self, vm): + wait_for_console_pattern(self, 'as init process', + 'Kernel panic - not syncing', vm=vm) + exec_command(self, 'mount -t sysfs sysfs /sys', vm=vm) + exec_command_and_wait_for_pattern(self, + 'cat /sys/bus/pci/devices/*/uevent', + 'PCI_ID=1000:0012', vm=vm) + + def launch_server_startup(self, socket, *opts): + server_vm = self.get_vm() + server_vm.add_args('-machine', 'x-remote') + server_vm.add_args('-nodefaults') + server_vm.add_args('-device', 'lsi53c895a,id=lsi1') + server_vm.add_args('-object', 'x-vfio-user-server,id=vfioobj1,' + 'type=unix,path='+socket+',device=lsi1') + for opt in opts: + server_vm.add_args(opt) + server_vm.launch() + return server_vm + + def launch_server_hotplug(self, socket): + server_vm = self.get_vm() + server_vm.add_args('-machine', 'x-remote') + server_vm.add_args('-nodefaults') + server_vm.add_args('-device', 'lsi53c895a,id=lsi1') + server_vm.launch() + server_vm.command('human-monitor-command', + command_line='object_add x-vfio-user-server,' + 'id=vfioobj,socket.type=unix,' + 'socket.path='+socket+',device=lsi1') + return server_vm + + def launch_client(self, kernel_path, initrd_path, kernel_command_line, + machine_type, socket, *opts): + client_vm = self.get_vm() + client_vm.set_console() + client_vm.add_args('-machine', machine_type) + client_vm.add_args('-accel', 'kvm') + client_vm.add_args('-cpu', 'host') + client_vm.add_args('-object', + 'memory-backend-memfd,id=sysmem-file,size=2G') + client_vm.add_args('--numa', 'node,memdev=sysmem-file') + client_vm.add_args('-m', '2048') + client_vm.add_args('-kernel', kernel_path, + '-initrd', initrd_path, + '-append', kernel_command_line) + client_vm.add_args('-device', + 'vfio-user-pci,x-enable-migration=true,' + 'socket='+socket) + for opt in opts: + client_vm.add_args(opt) + client_vm.launch() + return client_vm + + def do_test_startup(self, kernel_url, initrd_url, kernel_command_line, + machine_type): + self.require_accelerator('kvm') + + kernel_path = self.fetch_asset(kernel_url) + initrd_path = self.fetch_asset(initrd_url) + socket = os.path.join('/tmp', str(uuid.uuid4())) + if os.path.exists(socket): + os.remove(socket) + self.launch_server_startup(socket) + client = self.launch_client(kernel_path, initrd_path, + kernel_command_line, machine_type, socket) + self.validate_vm_launch(client) + + def do_test_hotplug(self, kernel_url, initrd_url, kernel_command_line, + machine_type): + self.require_accelerator('kvm') + + kernel_path = self.fetch_asset(kernel_url) + initrd_path = self.fetch_asset(initrd_url) + socket = os.path.join('/tmp', str(uuid.uuid4())) + if os.path.exists(socket): + os.remove(socket) + self.launch_server_hotplug(socket) + client = self.launch_client(kernel_path, initrd_path, + kernel_command_line, machine_type, socket) + self.validate_vm_launch(client) + + def do_test_migrate(self, kernel_url, initrd_url, kernel_command_line, + machine_type): + self.require_accelerator('kvm') + + kernel_path = self.fetch_asset(kernel_url) + initrd_path = self.fetch_asset(initrd_url) + srv_socket = os.path.join('/tmp', str(uuid.uuid4())) + if os.path.exists(srv_socket): + os.remove(srv_socket) + dst_socket = os.path.join('/tmp', str(uuid.uuid4())) + if os.path.exists(dst_socket): + os.remove(dst_socket) + client_uri = 'tcp:localhost:%u' % self._get_free_port() + server_uri = 'tcp:localhost:%u' % self._get_free_port() + + """ Launch destination VM """ + self.launch_server_startup(dst_socket, '-incoming', server_uri) + dst_client = self.launch_client(kernel_path, initrd_path, + kernel_command_line, machine_type, + dst_socket, '-incoming', client_uri) + + """ Launch source VM """ + self.launch_server_startup(srv_socket) + src_client = self.launch_client(kernel_path, initrd_path, + kernel_command_line, machine_type, + srv_socket) + self.validate_vm_launch(src_client) + + """ Kick off migration """ + src_client.qmp('migrate', uri=client_uri) + + wait.wait_for(self.migration_finished, + timeout=self.timeout, + step=0.1, + args=(dst_client,)) + + def test_vfio_user_x86_64(self): + """ + :avocado: tags=arch:x86_64 + :avocado: tags=distro:centos + """ + kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/vmlinuz') + initrd_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/initrd.img') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 rdinit=/bin/bash') + machine_type = 'pc' + self.do_test_startup(kernel_url, initrd_url, kernel_command_line, + machine_type) + + def test_vfio_user_aarch64(self): + """ + :avocado: tags=arch:aarch64 + :avocado: tags=distro:ubuntu + """ + kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/aarch64/os/images' + '/pxeboot/vmlinuz') + initrd_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/aarch64/os/images' + '/pxeboot/initrd.img') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'rdinit=/bin/bash console=ttyAMA0') + machine_type = 'virt,gic-version=3' + self.do_test_startup(kernel_url, initrd_url, kernel_command_line, + machine_type) + + def test_vfio_user_hotplug_x86_64(self): + """ + :avocado: tags=arch:x86_64 + :avocado: tags=distro:centos + """ + kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/vmlinuz') + initrd_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/initrd.img') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 rdinit=/bin/bash') + machine_type = 'pc' + self.do_test_hotplug(kernel_url, initrd_url, kernel_command_line, + machine_type) + + def test_vfio_user_migrate_x86_64(self): + """ + :avocado: tags=arch:x86_64 + :avocado: tags=distro:centos + """ + kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/vmlinuz') + initrd_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/initrd.img') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 rdinit=/bin/bash') + machine_type = 'pc' + self.do_test_migrate(kernel_url, initrd_url, kernel_command_line, + machine_type) +