From patchwork Mon Jul 19 20:00:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9227C07E95 for ; Mon, 19 Jul 2021 20:06:57 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 23E31610FB for ; Mon, 19 Jul 2021 20:06:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 23E31610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43308 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZXU-0007M0-As for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:06:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47820) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRj-0006Kh-8r for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:59 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:10188) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRe-0005hl-43 for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:59 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuc5F002487; Mon, 19 Jul 2021 20:00:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=iCfktZ7AINHjdFcfSuSEq/+qqI6dWDIrfO4IAZ/GUXU=; b=d4ipzbzhzUbN1byk8JjckrJdDKF8YM9XsPuS8OcMxaYjcCsP3WvRegsE6YTMUOqSbEZa ZAozAKdraCSrgy0nL+7AvJIDl54g+NfGi6n6uTm9nYh78dRQ0NH7yRumrTmB9sE337gR Eyc+0RtY4JJpliGCG+rqmeL0ycWxzrjg3ShwIDvosaoYffKtKK2RrJ25y5wlLRygIqp4 d/uvbBIddKFCB6kkZ/LQtK84BFnO1ztqTNzkqpuSSEg0TQzQeoRXNIh6vCSABBSvJJOJ uNefi3Awpa40L39YPUI6TimKZyPQGC+mgxTRp6nMWdZcbnQeFZ05zCU6TUICHmDgydST nQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=iCfktZ7AINHjdFcfSuSEq/+qqI6dWDIrfO4IAZ/GUXU=; b=WvaceVhg0hW+NZaLVshtLBct+HIinXXN9kOM5zRMDPRRecR1XQknL8DaKVutRRXWx6nK cVtha7JlgvhU1ASCD0u7wrXdBV9/YY8dScAfg731Ody50j3KtWLWYqaTSSMiDOW0DyY8 RqTXkz6mdcJdJ6ADfZsNEwTvzW26oyc1/39lbHyeX7xBDqC28Q06YroqjR1ttRoDTDgj F4sSEmNsA1Kf46RPWayspDkPB1Ud5c6+rp7ZtUuzfzkY/NiEm8JXigf1PJegrqzi0Mjf kCsXl1K4IW5QdWAXLD+yydhidifooS+9Xq1vY/uPHSjxF1ce6cVxtBErZ/RF6s34xhCo 3w== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 39vrn5ja0x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:52 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JK0WYs127596; Mon, 19 Jul 2021 20:00:51 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2100.outbound.protection.outlook.com [104.47.55.100]) by userp3020.oracle.com with ESMTP id 39v8ytbghn-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LhqYjbIuUYGjRdkzoF0XCAZrVkHSGdSWN6VX4MPsc3tkWr7iKqHP1uYZ9cH2txRw/wdh4oTrepSMm+wIOwA49OBcLzXAUoPDJ7+Tz/WFW2Q1GkIS7rSHIz5qAbhsqH5F419TRh0ZeQbY2w+oZACPZkQE0pnb2MWtI9U84xXS5BhmBtFpsOToIqHnWib8BDQQsi0Oq/NVZ9+ncTGIvUdYLwwdGZmFglerZ3SM3Kx5ig8A8XiYj4yZALtvOrdCMd4YIh/2ngRJoHY4prDlfnm91tady2wvWtbIcZCAr1+9D4W+mW2VpHtjznTAIo3PLNfoMJzGvY+G7GypjTz2kgwHnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iCfktZ7AINHjdFcfSuSEq/+qqI6dWDIrfO4IAZ/GUXU=; b=GXDF9SseJmcnhK/2sinK2oeTMGUk+QCxoE1Vy31TKKIRaWlDZt13AylV+RmqWDYPuuPII/lNLEJ6YnPYQvmsIuuHxhyasoQp6P+qW3pRDhJloZX2XYtPvRLq1LPrLysiHP17ehPCiEWRDDBNZKV0SE4gOVftjZyVIO9+43fMbSgvJyBZ1DMUodO89ij+tEVKT1ChuokqZFEJv8bs0QQfSqtnizMLmPhP5sbxxSCSFg3fh0r5FzuJmtPNc8SdoTuEniRMzEjwO473OvNJcsC9zoE/8RJDsGf+l1o4BV7Vp/r+GbD2cO0SpF71cPKL9vn18vzOD1M9vW5A3Mr9d/RWxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iCfktZ7AINHjdFcfSuSEq/+qqI6dWDIrfO4IAZ/GUXU=; b=HdDj9ayJOzN+RrLpyoS+J3iQyfn32i/5yDsK/V5jUz0jreg3xaYVJLrJZR4+3k3fgNxQf7p/xyGyvEYSn8tIRs4egohVSAaa+ZqdtiI+daCriUW1u/qNIYhZFpnFHjwy9QOnKlTItFt0cARcthUQecv0jCF7g7e0NAnLmU7RNX4= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4240.namprd10.prod.outlook.com (2603:10b6:208:1d9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 20:00:18 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:18 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 01/11] vfio-user: build library Date: Mon, 19 Jul 2021 16:00:03 -0400 Message-Id: <5002d6a67cb6c429b2e3d21e106b468176e82a1f.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 20106847-7384-41a3-df05-08d94aefd55a X-MS-TrafficTypeDiagnostic: MN2PR10MB4240: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:741; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: u70lsgtFQ4T0f4SJ2xtegW8+Yh9Q8+wA80KBvSh7OMwcakf5bcNOmGQOTxpsZ94qLO1ZWB92KvwZ/xCExGVtT87els+T17QE7QHypxfrn1aBcImAtMOXE5fP+6z4alrcbjxxXzuMbaMVt1EFLdwAMh0ex1SrrXTOsomTzi/GI6eifSvz34b1mXI0Y26OvHFOSBdReP374yTpKb6PCYhRgZutRXh9J++9oep798oZElWnt6A4mO+haFK3a554+wxKGt76zgStQa5H9EEVYocYCwswFeFLzaQ6iEVlmHj5rMIhmUzK/rdPQJMOZqcmGoHTgvG+BmWilampDWIXzxyrGemnhTLeZPthoaMz4JdwaH/X+sm+tcZ5EKqt90nE5Do9wSmt88ym3o69Sc/sVcGX0ROlqckGZMzbixualwKtt+MdQcXyGh642cJbYXElCqMjgDZ7sfbuw/xQypbivBineBMnu/8sMw0UL+4KaCLYrgyEQqqELQkUGP6HHvGVQe0mr3aalA7qPUxKq0QFE7HFYi2wyyjdrbqh0dmPFzBX7ZXj5MWxHCFqjIlXd9JwLLJcEsEoV/W8Dzir7JMeVrZV9ST9L0GrQHX3JwNal/15pW8L0m5rCgm+ws7A0GgWsripcgEp9BWcPaJAbLgb1lf64dkS2zJUaZoKRSIAqp6Knw0JTAaFeTbm3wqqm9zbIPyvsIhJI7w+bmC0BgLZ83tQPwuGG4JVpevEQpvG1y7E1bZ2n05KE/+SYkU8Y/BcMf7+1CkUItvOspZFlioRORDGY7+UuzduwXQsf6QEpp31uELF8FNuAlA3eOmf5v+jceuE+8Xq9DWhoaToepkcaq3xaQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(396003)(366004)(346002)(136003)(376002)(66476007)(966005)(66556008)(8936002)(6486002)(66946007)(36756003)(478600001)(52116002)(38350700002)(7696005)(4326008)(38100700002)(316002)(107886003)(6666004)(6916009)(2906002)(5660300002)(186003)(26005)(8676002)(86362001)(956004)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dWno1BOs4/lI2wV/669UM1jpUAtwHjZRoCeZndkf1Q5SRxolwwG65ycCYna1ipmCMcsYFQ8r6l6T1gQqhdWpxnXUs8nezGuJB80AxLoYZ0j7tmYbhb12beVacrNOk1lVrcso1/lEJUW5MGS/hmPytLJcSis+JrYGY9o0S20ZpM8iIJx0S6+ZvJBoF8HQFx0bPE5vCbYKZkHwA8X8UQTfCmVB35tZlghAsBQZeMNvvBGAKofy0aPDKKeZsOtFuoqvbC8D/EyNn14PRI6EbQN9uxM0CDAlgmAoO/6ZkSQnwyVL7wj6PMzZVBeWdS3T/6331COGDjO2LQiXHqKf9RCmCRYhPfLQsA8cV3QriBu5uoHxihTBH2/YlK/TkUXG//Ca6qZFN0q3kXSxdeP2AGO5vMHdIMdvKvPBWqvWAD+I5zbf+kL0na7zYH8pORNRRc5E+pTfApD1poZSaAeP5UO3YuJ+qxMRsFgNYaf6N1Bg5O0ckI0hxXh5Imfp1HAcWErBZtmc6DRoMy+jhZyRudRbrd/lGg7SOdiD+KgJF1af6K+nNo+vg+ODCH6Uzyd8BLyniwdJB6qq8YECFMIvVW5Hhmfy1l74SH0GnyWXyXlntloV+B+oKeHAE5+zRc+TPhbhvvJdQwj5iOcSs07P3YsJMrmNJS7V8wjGDmmo0QWe/vE7zPGqcAWqdy5HuUNKbyS6CdySUB7alCngFsrsxA1XmmQj9dzp/JH4ALrFs7c3WDr2otxY8TJjpZCpkPi+Y/mV X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 20106847-7384-41a3-df05-08d94aefd55a X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:18.8546 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: o0bF/x/TsyBJ+kZZcS31BEAK84mQa/eRx5aB+HckhjWV5GsnonIb7EbA6uDdJ37FkN2Uf1Vt98EkAjVADsCzng== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4240 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190113 X-Proofpoint-GUID: zHCUygofHV4UmGj48DNk3P0fmPLxs_6G X-Proofpoint-ORIG-GUID: zHCUygofHV4UmGj48DNk3P0fmPLxs_6G Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" add the libvfio-user library as a submodule. build it as part of QEMU Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- configure | 11 +++++++++++ meson.build | 35 +++++++++++++++++++++++++++++++++++ .gitmodules | 3 +++ MAINTAINERS | 7 +++++++ hw/remote/meson.build | 2 ++ libvfio-user | 1 + 6 files changed, 59 insertions(+) create mode 160000 libvfio-user diff --git a/configure b/configure index 49b5481..bc1c961 100755 --- a/configure +++ b/configure @@ -4297,6 +4297,17 @@ but not implemented on your system" fi ########################################## +# check for multiprocess + +case "$multiprocess" in + auto | enabled ) + if test "$git_submodules_action" != "ignore"; then + git_submodules="${git_submodules} libvfio-user" + fi + ;; +esac + +########################################## # End of CC checks # After here, no more $cc or $ld runs diff --git a/meson.build b/meson.build index 6e4d2d8..f2f9f86 100644 --- a/meson.build +++ b/meson.build @@ -1894,6 +1894,41 @@ if get_option('cfi') and slirp_opt == 'system' + ' Please configure with --enable-slirp=git') endif +vfiouser = not_found +if have_system and multiprocess_allowed + have_internal = fs.exists(meson.current_source_dir() / 'libvfio-user/Makefile') + + if not have_internal + error('libvfio-user source not found - please pull git submodule') + endif + + vfiouser_files = [ + 'libvfio-user/lib/dma.c', + 'libvfio-user/lib/irq.c', + 'libvfio-user/lib/libvfio-user.c', + 'libvfio-user/lib/migration.c', + 'libvfio-user/lib/pci.c', + 'libvfio-user/lib/pci_caps.c', + 'libvfio-user/lib/tran_sock.c', + ] + + vfiouser_inc = include_directories('libvfio-user/include', 'libvfio-user/lib') + + json_c = dependency('json-c', required: false) + if not json_c.found() + json_c = dependency('libjson-c') + endif + + libvfiouser = static_library('vfiouser', + build_by_default: false, + sources: vfiouser_files, + dependencies: json_c, + include_directories: vfiouser_inc) + + vfiouser = declare_dependency(link_with: libvfiouser, + include_directories: vfiouser_inc) +endif + fdt = not_found fdt_opt = get_option('fdt') if have_system diff --git a/.gitmodules b/.gitmodules index 08b1b48..a583a39 100644 --- a/.gitmodules +++ b/.gitmodules @@ -64,3 +64,6 @@ [submodule "roms/vbootrom"] path = roms/vbootrom url = https://gitlab.com/qemu-project/vbootrom.git +[submodule "libvfio-user"] + path = libvfio-user + url = https://github.com/nutanix/libvfio-user.git diff --git a/MAINTAINERS b/MAINTAINERS index aa4df6c..99646e7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3350,6 +3350,13 @@ F: semihosting/ F: include/semihosting/ F: tests/tcg/multiarch/arm-compat-semi/ +libvfio-user Library +M: Thanos Makatos +M: John Levon +T: https://github.com/nutanix/libvfio-user.git +S: Maintained +F: libvfio-user/* + Multi-process QEMU M: Elena Ufimtseva M: Jagannathan Raman diff --git a/hw/remote/meson.build b/hw/remote/meson.build index e6a5574..fb35fb8 100644 --- a/hw/remote/meson.build +++ b/hw/remote/meson.build @@ -7,6 +7,8 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c')) +remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: vfiouser) + specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('memory.c')) specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy-memory-listener.c')) diff --git a/libvfio-user b/libvfio-user new file mode 160000 index 0000000..2a0a929 --- /dev/null +++ b/libvfio-user @@ -0,0 +1 @@ +Subproject commit 2a0a92912d598de871ab47c034432c5fa6546dc4 From patchwork Mon Jul 19 20:00:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42526C07E95 for ; Mon, 19 Jul 2021 20:02:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B3293610FB for ; Mon, 19 Jul 2021 20:02:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3293610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33622 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZTd-0000u9-KB for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:02:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRl-0006QT-7m for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:01 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:21036) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRi-0005kG-SP for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:00 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuV6E015843; Mon, 19 Jul 2021 20:00:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=3Bsyd2HEbENxiOohTS0Bfm+QZAsTYYNrMTL9j9LWH68=; b=B9mGixpX0v5vQENU3Q/yUGfTGsaenSYoUyivsJnBG3fB79K/lze5r0FuWfMEKlTuMQdy yIWSs2Q+qO2vDEf8EHgRxGgYBoFwIa0s5JKBpANR59LApGYP+XIp5JCvE8+ZbqLNyDqh qU1HyAu6u43OtZJ5cropwgvjZSOHiamnJOppHDIdFcT1hz4FimeVzcQaj1HhBmtTzz0j pXizrasL5dsMTGR9DFOr4k7yCaxa5p1vWlmg0dNL02vLZC3Wc6VSJMxc0iJMhafDFeya H8Gvq21BSyabBdXyLktmv6lgyf7XiTJ2OZI2I+nZdO5HvKb3pzat8nEZj8foKCYpJZ/m 7Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=3Bsyd2HEbENxiOohTS0Bfm+QZAsTYYNrMTL9j9LWH68=; b=ZLop9p4LrtkYCBX1iIz8NAHlEJzrkocsZEI8sRy7eY2UaRlyRIcfC6m+vxkauKaZtDSF qcsGoU0QQukm3Q8n7pJ2peUx6RL8imhueR3Ekdl1cbv9ifSBZsaQjeXZJ2L0tiLIv76p aUTOoh0IlC6fIyHhVFEt9ebomWHdxchUU9P9RH7lgR/jd17mS/tDf5gZS1wq7VUDJKcI 2kHox+b1mnMuoBfW0lukCP00n+rbo54Ym8tqvi4J8L1CabZ4tj16PlmisYsRoCZ4FJgr pnW39FlcpuGmNysyCg60qlbkZtSsN3EkeU8ZyTgQySSiSKq0wtWUDitbXoakzhUr1Xkk AA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 39vqm9a98y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:57 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JK0WYx127596; Mon, 19 Jul 2021 20:00:54 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2100.outbound.protection.outlook.com [104.47.55.100]) by userp3020.oracle.com with ESMTP id 39v8ytbghn-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AmoyvhZaAW+IvtFI49t+2RHuOzx1u4d95Aip4kRy4pfF6QzAoGVcnDZb1MKPC3mqvZamkKd49ssGvqafe7Dg7viM13yy/1vnUXTkyrRGFNAtrLydqRPPlu25F0owz7HixbOTAfpOCPhwY4Xn6oizfsHFx+ILPuFbUOkOoboMq/jnSDGtEUQbm08S+HBZYk36J8HYasYyHHjSa4EQarFHNE2M+3hUmHcEfsyd2gjqjUZyQM4sbIE8ntxgtRrEE7iaWsRKuGdzEOUuWcq0twwDy7kDgv+qJbpXJlaxRhcu4HNAfaZLHXiIzPImjwQlmdGQ/mFZI9nNb1WCgIKjAhTioQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3Bsyd2HEbENxiOohTS0Bfm+QZAsTYYNrMTL9j9LWH68=; b=T+JjQQoKIt9d0Nzhmv8S4hGxS37KP5SH+sNp2HmUyseAs4ygnWVSL3TaHANt6g2y93gfojfX8Yn3KeHiDkuC/iFZQRC3NuMrJAnm0EYEYG3Doct2tyPNMfQ/bNq7nr5ULWWL8ABTk6TyB3QGmeinNVYxAXX53YL95dTUz0TYRgjUyTpPJ7cr/cltgDGEWiD426XjGY7CBzY7mTWl20YuiApAwa2RG6ww1QujDlUxMUlA0o5GNJNQ3/GCcJO4YlniepUQl1YEG2GKvRrT2bYLUQ8TIgwzyChmnTdIR4UfXviR2ubC1OHwgOdme6lHtuNzE2luhUlBgXhZJOFp5UA3+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3Bsyd2HEbENxiOohTS0Bfm+QZAsTYYNrMTL9j9LWH68=; b=HhB9ZhYEX+CvG2DIiYXHPV1NsZb5xqhHoFHykZXMdPWPf7i+ILPBB/w99LVAnKTtDtZX4+eo1sHo29Vf3eVwa4vx8SNRiUg7zyOxeKekkO3z5GfoDQKE4mZW9zmb+dfQbbnVwljLORUNCpdJInWFNQC9x3qL04or/t8G+weYblc= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB4240.namprd10.prod.outlook.com (2603:10b6:208:1d9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 20:00:20 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:20 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 02/11] vfio-user: define vfio-user object Date: Mon, 19 Jul 2021 16:00:04 -0400 Message-Id: <190ed3e058c4eb49ade982394b23853a9ad76552.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5610aafc-531b-4719-8695-08d94aefd627 X-MS-TrafficTypeDiagnostic: MN2PR10MB4240: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:421; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aip0fn18J5B23H8Uzmvc1ywSzEGu56ChuZsT+wF9y+GAadDE1Ul0GlfEs5qEjkm+p9zsid6CkU7Xs60uQz87LlrunAKC0RVQlHmrI63hFB1dziK2IrkL9q02iVz2A1qqDpNQe9JxDs+X1ecElYGyIbzZICXL2n4cSOQUYIURILODBMcgwaYG0Ml/Z+bjBDHRdAD1t7ihviBh0IMxs4w3iPdCAq+/u3AybiqRh4SXhGXm0s4MTsVgS5yXQghtwlmEeqx4a7Yf56X+T6KBx9YYz2PladT4lAMPazP35LxZqvgYjaaHar97TBQhpRm28J2hQX1ogHg5MKSIgbKBYcmv7w8P4d1Ep8w+nawbnSxt2mTSfCwmmtDLorzBjUpn8t3EnNgGS5N80swW3JFv+uDXLelO6fsPjry0XDefT14dp3P5B7fwklpdZZ4cEDJJaiDKfGh/thTmIF6b9invZXzpIExpz/SbmoFl6Z2H1zoPxfIMF57GMUp97TT6C7OY/AietJrLNtZLT87fFT0Krk/onv+Y10pJtxTcn/siuGpY3+uLcuQTy6yzPqjVypGtHKDwt/bnGayaX994B6l9L4XrrYs/bNrCLXSQt2mEDz8S8I1zZQi2Z/1JH+f5NF+BgVw8wdKYNHENiuhUhLmXMHfAAdv29qrR2BwRVNHoMYKQD4GG7LyNCfPE+0ERP6CMgKfy0PM/tpWfbdNedJBYAOgcOA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(396003)(366004)(346002)(136003)(376002)(66476007)(66556008)(8936002)(6486002)(66946007)(36756003)(478600001)(52116002)(38350700002)(7696005)(4326008)(38100700002)(316002)(107886003)(6666004)(6916009)(2906002)(5660300002)(186003)(26005)(83380400001)(8676002)(86362001)(956004)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 9qxem7G7BehYbxAMRUIW45lrql5X62OEULQ7zQ07qlYDMSVI+K8NKcysQHv0BE3N0nkMR9p9TSOPKxAxsHxyetUaKrZan/ha/jNDa+k6y48KbE6/kfiiB33Ki5uySAGumN96AAC6xH9p81xwJvtRKa6+W+IDjnCHyTaiIqfP7AX6K8ej7G2tX6XU8wjd3OTgRnwKR/Fe8kTHrne3lKNzzYNPcCtR6geBxAqG/cp39/slGyJsIW2OHYjWDr52930167Fagl9iXh5AtdIhDpbhOQtP1fqzMJaV6kSoSbK8Q0CHfGSplSrWCyAu7Ny8GOXTR7Z6Od0KDSYALd9TeSLwCKP6KcuaY/8OcLmKS1cUdtx0qk2VEWui8rUzJWiZWGFPgxT6HL14l9CcKNLk1xemRpCPMACHMUPQmmTt22vaBmi5BfWAknGLLJo6JpxyOY9JbQoZiTkHfaQ2OIXioY2ancEhICVn6TY/n/9ZuIDFVW9dMMmA/AgpsZFSCbRQtKypDhYjXLaXc9R7TvLhkKwJryqHxrNrBGRm9L8+4vhFaidj4+SaGHugzCBKlPT/V3pGfsVb1C5UvtFVBMfM/ssow2aTWX9O9jAFmI6oXtB7WZFUSPm8c0gfI6DvPfeWsZHxJjFMGw5pt7nfLy48fjc9h4trCNWAauQfFPiPQzmOqMf2dBoXDgzSnlZT4FOs+C8bflNMxNTSMoTXtLKyOsJXbS9oGfBtILUKb4kJsSWBUOuCyamX8IbR4a16gOEAx/y6 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5610aafc-531b-4719-8695-08d94aefd627 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:20.1938 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: l6n8gT2+PAZl9nSCL3NYHmPoaU0akMzThijJjn/se96U6u+QTxEqnxW7eb13y812PhggGFo07guQhq5T6Fnijg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4240 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190113 X-Proofpoint-GUID: vtFSPLejk8--e7GNXdnRG8nY_KSld1rb X-Proofpoint-ORIG-GUID: vtFSPLejk8--e7GNXdnRG8nY_KSld1rb Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define vfio-user object which is remote process server for QEMU. Setup object initialization functions and properties necessary to instantiate the object Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- qapi/qom.json | 20 ++++++- hw/remote/vfio-user-obj.c | 141 ++++++++++++++++++++++++++++++++++++++++++++++ MAINTAINERS | 1 + hw/remote/meson.build | 1 + hw/remote/trace-events | 3 + 5 files changed, 164 insertions(+), 2 deletions(-) create mode 100644 hw/remote/vfio-user-obj.c diff --git a/qapi/qom.json b/qapi/qom.json index 652be31..e0716d2 100644 --- a/qapi/qom.json +++ b/qapi/qom.json @@ -684,6 +684,20 @@ 'data': { 'fd': 'str', 'devid': 'str' } } ## +# @VfioUserProperties: +# +# Properties for vfio-user objects. +# +# @socket: path to be used as socket by the libvfiouser library +# +# @devid: the id of the device to be associated with the file descriptor +# +# Since: 6.0 +## +{ 'struct': 'VfioUserProperties', + 'data': { 'socket': 'str', 'devid': 'str' } } + +## # @RngProperties: # # Properties for objects of classes derived from rng. @@ -807,7 +821,8 @@ 'tls-creds-psk', 'tls-creds-x509', 'tls-cipher-suites', - 'x-remote-object' + 'x-remote-object', + 'vfio-user' ] } ## @@ -863,7 +878,8 @@ 'tls-creds-psk': 'TlsCredsPskProperties', 'tls-creds-x509': 'TlsCredsX509Properties', 'tls-cipher-suites': 'TlsCredsProperties', - 'x-remote-object': 'RemoteObjectProperties' + 'x-remote-object': 'RemoteObjectProperties', + 'vfio-user': 'VfioUserProperties' } } ## diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c new file mode 100644 index 0000000..5098169 --- /dev/null +++ b/hw/remote/vfio-user-obj.c @@ -0,0 +1,141 @@ +/* + * QEMU vfio-user server object + * + * Copyright © 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL-v2, version 2 or later. + * + * See the COPYING file in the top-level directory. + * + */ + +/** + * Usage: add options: + * -machine x-remote + * -device ,id= + * -object vfio-user,id=,socket=,devid= + * + * Note that vfio-user object must be used with x-remote machine only. This + * server could only support PCI devices for now. + * + * socket is path to a file. This file will be created by the server. It is + * a required option + * + * devid is the id of a PCI device on the server. It is also a required option. + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "qom/object.h" +#include "qom/object_interfaces.h" +#include "qemu/error-report.h" +#include "trace.h" +#include "sysemu/runstate.h" + +#define TYPE_VFU_OBJECT "vfio-user" +OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) + +struct VfuObjectClass { + ObjectClass parent_class; + + unsigned int nr_devs; + + /* Maximum number of devices the server could support*/ + unsigned int max_devs; +}; + +struct VfuObject { + /* private */ + Object parent; + + char *socket; + char *devid; +}; + +static void vfu_object_set_socket(Object *obj, const char *str, Error **errp) +{ + VfuObject *o = VFU_OBJECT(obj); + + g_free(o->socket); + + o->socket = g_strdup(str); + + trace_vfu_prop("socket", str); +} + +static void vfu_object_set_devid(Object *obj, const char *str, Error **errp) +{ + VfuObject *o = VFU_OBJECT(obj); + + g_free(o->devid); + + o->devid = g_strdup(str); + + trace_vfu_prop("devid", str); +} + +static void vfu_object_init(Object *obj) +{ + VfuObjectClass *k = VFU_OBJECT_GET_CLASS(obj); + + /* Add test for remote machine and PCI device */ + + if (k->nr_devs >= k->max_devs) { + error_report("Reached maximum number of vfio-user devices: %u", + k->max_devs); + return; + } + + k->nr_devs++; +} + +static void vfu_object_finalize(Object *obj) +{ + VfuObjectClass *k = VFU_OBJECT_GET_CLASS(obj); + VfuObject *o = VFU_OBJECT(obj); + + k->nr_devs--; + + g_free(o->socket); + g_free(o->devid); + + if (k->nr_devs == 0) { + qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN); + } +} + +static void vfu_object_class_init(ObjectClass *klass, void *data) +{ + VfuObjectClass *k = VFU_OBJECT_CLASS(klass); + + /* Limiting maximum number of devices to 1 until IOMMU support is added */ + k->max_devs = 1; + k->nr_devs = 0; + + object_class_property_add_str(klass, "socket", NULL, + vfu_object_set_socket); + object_class_property_add_str(klass, "devid", NULL, + vfu_object_set_devid); +} + +static const TypeInfo vfu_object_info = { + .name = TYPE_VFU_OBJECT, + .parent = TYPE_OBJECT, + .instance_size = sizeof(VfuObject), + .instance_init = vfu_object_init, + .instance_finalize = vfu_object_finalize, + .class_size = sizeof(VfuObjectClass), + .class_init = vfu_object_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_USER_CREATABLE }, + { } + } +}; + +static void vfu_register_types(void) +{ + type_register_static(&vfu_object_info); +} + +type_init(vfu_register_types); diff --git a/MAINTAINERS b/MAINTAINERS index 99646e7..46ab6b6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3380,6 +3380,7 @@ F: hw/remote/proxy-memory-listener.c F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h +F: hw/remote/vfio-user-obj.c EBPF: M: Jason Wang diff --git a/hw/remote/meson.build b/hw/remote/meson.build index fb35fb8..cd44dfc 100644 --- a/hw/remote/meson.build +++ b/hw/remote/meson.build @@ -6,6 +6,7 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('message.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c')) +remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('vfio-user-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: vfiouser) diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 0b23974..7da12f0 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -2,3 +2,6 @@ mpqemu_send_io_error(int cmd, int size, int nfds) "send command %d size %d, %d file descriptors to remote process" mpqemu_recv_io_error(int cmd, int size, int nfds) "failed to receive %d size %d, %d file descriptors to remote process" + +# vfio-user-obj.c +vfu_prop(const char *prop, const char *val) "vfu: setting %s as %s" From patchwork Mon Jul 19 20:00:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52CA7C07E9D for ; Mon, 19 Jul 2021 20:15:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C28A760FDC for ; Mon, 19 Jul 2021 20:15:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C28A760FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39930 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5Zg8-0007n8-TC for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:15:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRk-0006Og-NE for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:00 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:10066) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRe-0005hj-4F for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:00 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJudOH002502; Mon, 19 Jul 2021 20:00:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=eURXpfhGODLjNdZvfIWbtuwDtS5SdddqepiXcpaXyLc=; b=V6luWgqNeglukkKyfG9N7PtGu6ydSQyzy4+cs6Q/4HQ6GNxnqlaaFj0uxR56zpgHF+Xy VKkFZbyhtAYrOWHcHtd65hgpLfy0014MYJVi+aimG/ClZ5EUzUHOEvw8PqFUFR6s0I7A 6YvpLinfUgkDGDiuuoLmZbJpDq2A6iZQaZ4c0sRnYlBrPlqlyuRa8X1p4kzQ5zU28LE/ Wm58Jpk4viSGg+5pQ+OUMlkXTxqSA8j6sv1PZHPSYEvzN97qgzxCS6e9wV5vLBgDA74v 476Nx/SyXw1VJvPbZt7dSnYfK1LYGi5ApK2m9TkI6c0Yb+pbQ83oMHL8A8MxJ7fKn13J xg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=eURXpfhGODLjNdZvfIWbtuwDtS5SdddqepiXcpaXyLc=; b=rsjoiqY924pynpBY1ddEwZgNnE021pZScxmHDrOlW4ucf7z5qPUJeEsu2D8shKHabttZ 4bS9UR5C6B9ZiOaXd0PmogzOd/zRf3nqc4LjW+7HlFVGdUDdbuVoocim9F/lRyf7KRqs 9jD7FljWqEhcHlLol44UcOU9CdXvrzKlA5pUGBxirTzYa6/2Z+bc4MWcQjUO3VCmf4i+ D6rwLV0GNsRx97jp9FEicfmxGMJMDxvbS4BvCZIdQiKiC+4dndZKPdhFXEB1ToPP4oDP 4091UZHtR6W/0TNRvXjflBqivFp38h8/U4XlpM0VjFNkydEBSRoUYBYUdReAg0qSYJGT zg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 39vrn5ja0p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:51 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JK0ZpX127869; Mon, 19 Jul 2021 20:00:49 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2101.outbound.protection.outlook.com [104.47.55.101]) by userp3020.oracle.com with ESMTP id 39v8ytbgn6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S8gyk8t28a5qGVnUqPyogqFTwe51Np02ebXDeAtSzSrCSFariCJ2hUmP+dbeZxEHEO1fF60KxYJw5POcW7DThSm1/kENv6rMWfUs0Bhb/xJVxRKU3uS/olvwMCxJhywGIOWTQw6QizULHsM5vQ9iUFiCEKsHK4a9ywnDvPaOapOLHxWFS8UHaSaEcibwGqC8b50CyKqb6dGU9RQPdLGZ1JmwliiCwhYJRkGEOXFuMD2wTVhlXC3TGXwRB3R4kqwTcaezasy1Z4e+raAhA5g+x3AHcjBJusxnFdMQc/+W1DHlyt5VHW/XaGn6PxABC18P/aCSSIS80bX7Ctyf6H3pSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eURXpfhGODLjNdZvfIWbtuwDtS5SdddqepiXcpaXyLc=; b=V6zq3KVXGFKiVO3WuK6e00WvA6ryXP/YwVSeasWwvlgSyWTPIsImQbs3blkGuALr5ZD117TiKkFoyIJMuoqwLM7DuAUkG/3LzucnDV9UcPCWvZh9O2ztRvVnQN+2D44kMHlRJ8vuP0trZrsVrIbP2O0qFqLpnZAWV1xUZlcMzPdPqzEH3sD01fct7CkclMfufBuNw0cuoNBbOdGbXL6/3DGwweNexqlj0zcCWU7m3TW1HkrZMHDpQgV1uO0Xf3fppx7hgy8IET4lRpZk+2DzAns9aIbN4d64SyXb0ZtpqBwtT2NU0PPggH3+6JaimxCKT2j2tQT3l0x5tU8BlPh6hw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eURXpfhGODLjNdZvfIWbtuwDtS5SdddqepiXcpaXyLc=; b=h5sNnFbU6YSdomWzh07xhnIkIc9I3wqUgolOv3olju3d5DgQTiE/gndVs659XRwa5rBVAfFGmP2gadKpm+UBp8Pod3UbrmVSPjESeA/FSby0yXVqcp1X8+FKloHjuEEx4QvFx6tB+NnPJ4fItti5CqZOfxR7syYNUQFrCCTl3lY= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3792.namprd10.prod.outlook.com (2603:10b6:208:1bc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Mon, 19 Jul 2021 20:00:21 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:21 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 03/11] vfio-user: instantiate vfio-user context Date: Mon, 19 Jul 2021 16:00:05 -0400 Message-Id: <6d28f979cc237c7c02527de9cd92ca95763d984f.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:20 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9ca13f11-499d-43c3-2a53-08d94aefd6ed X-MS-TrafficTypeDiagnostic: MN2PR10MB3792: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:586; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Soac/hb3cF3PhcnPJ4fgmp0ZpeXxuuoNpm+9fSOKoLxzwo0YSM/tMbdjmWvXtrX7C/6ywxxBwfuKtH7A/17QwO95QD3IIbA8ozF+4RB3mzs9VQ128vF/+cdQMLmQfrRHcuyLu5NTDq57xpYABN/mjkwDu+ZAi+5mRMsDJz8BxCXNt5ARv8NSq7Nck4dOIrKw0gtWXQetR3xX7pJ9uZX3cvyJ9G7M6n85GnI1w5drzQiSgdNHd/z9bMOg369GeESgN5sUYvvhEpjefEfI68NHXpI8hWQZ8jFXOpFqQWaOScKyszn4wJ8PvqXS/MnZDGzwrfiFV6/jGcFM1BPngR32V/Uz9q1cnHyg7OzuSFgs3tArQU/S8k+WUApJqbn4ZLoz28BvHIRK9khmJZZJ6zJKZr18KGXNiiGXtS7Li/HTfnZBeROFu/da+iDYq73Ydwmt1SHNky/jDjtDm0567FHMF+woFxynDCDKqthg6KBkdOCXR4wUcaK2e9ZGddmpnEvF9aCzRn0u626KNP5rIkW0g44lIsHoiyz4KFFjJ7GHWVxxXEMZ4qzub4rxr7T1opdRZ0yowaHBgJTP5rI0vBn9oAuQiJkuK0VLdQm5MnuY2xIO3fsaoW6QFkCasMAOT8JrtBVkxeKOnPfu0NOFhCrUDiKXqGUHUMM/wozVAMz0iwRYeF5s6m8Pqwp9KJO86iUt2rJqhbrEcOsDiSC0TnGonw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(39860400002)(366004)(346002)(396003)(136003)(107886003)(8676002)(478600001)(36756003)(6666004)(8936002)(66946007)(86362001)(4326008)(6486002)(186003)(316002)(2906002)(5660300002)(52116002)(6916009)(7696005)(38100700002)(38350700002)(26005)(66556008)(66476007)(2616005)(956004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8jxVQoPzBazVqc2jlp2wsB/QD/ktpQk6M7XBus5IyNyVHQjSICd02t+LQXfJ/YAwDkxNw16ZRR8+o6nME0anEE1W/PDEHamMbNmNasSsbjLU1671P1vyKOLeYsSAM/gbgZ7ww7OaqWEe3uz+mK1/krTNvOAm6v+h7VBY4SnB4autGyLeRvmF+vP8UNqytwRaI51nkYAMvvmyXiANaWQDEDMCYzVevHZnp3VtZFohuMDrbD8HkeopAjbCatgw42Xng6219cN6RQJBv+rH6KHXYaJEP1cT4/FH+7oSPgNwaSaf97Axo4OBUanQtGGJvK3CgpCieabu5ilGL7sA7M3ByzmBLAI8TzFv+HgRuaf+tzKoiBOLIWYoPj0uC5QR4bCOVT3r3jhUQWUs9KDyjzDsBBNvsMpGn8+Kq6gPUH5rRAKxqfK4XVDvD7GzYZSQfyejPq3e4S5NQVAriLvexNJN5QYiRSKXLVA6LG9KocplCgWVR92bk2UNsfJHVXwBvxUt5p3WuotIHD4F7fTdeddBB1Q34BdxhoXF9j2n5608VgtawCZpp907XwpzJm3rbh28HofW8wPH3by+bi2WxQyjrm4kLZcgRtpt3bf9pKNKa+dfSskcDuz9wuTjdQcQ3sEXyWtMQgoyc+3j/q0EBFzJpnInRJ09znjuBxY5eqjubzpFLYOZvNc+1LpkkGKxv7eV7m55NS/8AEiXFEUv0ASfuhdHb3RQG63O02DIyfH94j/4x3A/1hXrce2S3N8trdgm X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9ca13f11-499d-43c3-2a53-08d94aefd6ed X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:21.4971 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C2Josf3yzx5glo9wVnwwYb0OCGGPqeOhTv2rZ6LCNRnxMagSih3ouX2f1egBBaRmP3OPuKmBi+aaXcPtXEivdQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3792 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190113 X-Proofpoint-GUID: 8A16x5UA-mE3xXrMP26de0sRfx8ZVX6q X-Proofpoint-ORIG-GUID: 8A16x5UA-mE3xXrMP26de0sRfx8ZVX6q Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" create a context with the vfio-user library for a device Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 5098169..adb3193 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -27,11 +27,18 @@ #include "qemu/osdep.h" #include "qemu-common.h" +#include + #include "qom/object.h" #include "qom/object_interfaces.h" #include "qemu/error-report.h" #include "trace.h" #include "sysemu/runstate.h" +#include "qemu/notify.h" +#include "qapi/error.h" +#include "sysemu/sysemu.h" + +#include "libvfio-user/include/libvfio-user.h" #define TYPE_VFU_OBJECT "vfio-user" OBJECT_DECLARE_TYPE(VfuObject, VfuObjectClass, VFU_OBJECT) @@ -51,6 +58,10 @@ struct VfuObject { char *socket; char *devid; + + Notifier machine_done; + + vfu_ctx_t *vfu_ctx; }; static void vfu_object_set_socket(Object *obj, const char *str, Error **errp) @@ -75,9 +86,23 @@ static void vfu_object_set_devid(Object *obj, const char *str, Error **errp) trace_vfu_prop("devid", str); } +static void vfu_object_machine_done(Notifier *notifier, void *data) +{ + VfuObject *o = container_of(notifier, VfuObject, machine_done); + + o->vfu_ctx = vfu_create_ctx(VFU_TRANS_SOCK, o->socket, 0, + o, VFU_DEV_TYPE_PCI); + if (o->vfu_ctx == NULL) { + error_setg(&error_abort, "vfu: Failed to create context - %s", + strerror(errno)); + return; + } +} + static void vfu_object_init(Object *obj) { VfuObjectClass *k = VFU_OBJECT_GET_CLASS(obj); + VfuObject *o = VFU_OBJECT(obj); /* Add test for remote machine and PCI device */ @@ -88,6 +113,9 @@ static void vfu_object_init(Object *obj) } k->nr_devs++; + + o->machine_done.notify = vfu_object_machine_done; + qemu_add_machine_init_done_notifier(&o->machine_done); } static void vfu_object_finalize(Object *obj) @@ -97,6 +125,8 @@ static void vfu_object_finalize(Object *obj) k->nr_devs--; + vfu_destroy_ctx(o->vfu_ctx); + g_free(o->socket); g_free(o->devid); From patchwork Mon Jul 19 20:00:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10BC1C07E95 for ; Mon, 19 Jul 2021 20:17:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9201B60FDC for ; Mon, 19 Jul 2021 20:17:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9201B60FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44284 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5Zhv-0002XC-Pd for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:17:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47860) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRl-0006RV-L4 for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:01 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:23190) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRj-0005kh-Jm for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:01 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJu6iQ017735; Mon, 19 Jul 2021 20:00:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=d8ctp07v7dtvx0lo7U5JYtOKCpaNBC1TURFpny+sErQ=; b=f2V1P+a5HhHZL9H7sIzQCKzejcZIYP7CDVEqmt1Z4YDpM9Bft09WZXd+9LPd4SFcp5YW q6vPOLJMSLwCfmPTiug41i3DGd7pXn4p1BS7gnzbLGG05rs1iSainGWxvO9KQZA+m23z CkCaLsz8xXeY3LbWiAYQtL3FA3Zi4Asohxj8thavMKsL4amVcgtV+/os/1EbsZCDb5b3 ChYZr0J5FCHyxY/ltYtV7kOw9MSP3RodGSj0XuUVHXleSVwCUl618xcW2RcmoxVngpWc GIsZbnmAAqPWim4oD6xZN+/8A1NETdtkMzi8D12/kYtfbekddd8Tk6oL+pFTEeUuXE6t Uw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=d8ctp07v7dtvx0lo7U5JYtOKCpaNBC1TURFpny+sErQ=; b=dJsI4EQBrL9dorx74XWj6lrGxdwtZRUh80KM86/4nLNAbJIDqOPqm4tcWd4FFZRubJpy HZgsLVILvLNUIyRp8JpdlKVNykIwDdiIVi2zpveU24f0h5HlE8v11aHrZao1wPQbxR9r tkuddlV41RUorFU1FQWWMSke8jfUGAZ3Tg0FfVJEScTlu2aTg/ZoXWLmknTcrktBYTF4 ZWbcjNOyzxMbOqEKjU+cYEoc1ytvTzJ6HL/MYJSjL2rqQc4sQ9mHp0hun1Im8ZqtKT51 sOQqWknz+ySQJB/qxVWgdWj3PFLBT9dcKaf1wr7pz9F7V2xvw+Kpufis8/oK65kjRcTm bw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 39w9hfs0d9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:57 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JK0Zpi127869; Mon, 19 Jul 2021 20:00:55 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2101.outbound.protection.outlook.com [104.47.55.101]) by userp3020.oracle.com with ESMTP id 39v8ytbgn6-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PGK/XRiPHl6H0xrd/ShnxRGwP79+ovA74gsyZNCFhe7Dw6t+iHv2lrjkpOuoe+hQp0NOXkjEUgB4mNgtVxBR4sdo5moDh75YHoAJVWxP0Ql6rp0YvtjaPeieRRZJ47lxb8E5tAqVebUwYqS/95ob4COLklnr54M1wN2rNmyjABQP9uau403UJ1BvWh6JJwyoY5Uk11pYPUhUnc3YTnY+9uZPyN++e3U6ZHAJuyXF+jbEbx9C63SkZximLB//6yuLnsEo0WaxZEtu3FrIDasb1jHgV6vKr17WcWscmyMTL+C+OjKScMQukyGL3TcmV7XCzwBZfYBrXabvS57697gjag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d8ctp07v7dtvx0lo7U5JYtOKCpaNBC1TURFpny+sErQ=; b=Br7omeyahj+qeJr3G4KVZimyytymBDh4En0EFCxd3SH1q8WDSL04gqkOdwmGW1AJp/SNdcFlAsztyeoIyrH2GG5mYnjDiwHs9z5kodQFyNGL9PqnOSd7/WnWsUJXtsVKbaKX7NjJFojGxq2ER4auqPiSKtSCpMklc12ribBrK4fTmLbTTUQ7VJFYR9to03ZxVUPSNx0jnIkhEfIjeRfw5nnZvrlJc+nXkwfxDri+Xvj2GklopBJfEexGHlp7YB/PHqLwcssT0ZKMcBl5phLB5ZBsIdN11t35UGPgnrv/rppsk++WXdP2ExFxNpscrLcrwu/u1JiEp1ANvMeB/B7ZIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d8ctp07v7dtvx0lo7U5JYtOKCpaNBC1TURFpny+sErQ=; b=SN/AMHCCGIWSC0hlk7sXznoKNfLYSExvHauQJEgvj4b95zQISwV3b3S5j04uL/LU19dG6J5kfHZu9erFJwkG0YRYam86NmRSF8G7cfN0Cd4/UqfCWtAdWpEiuFcm31GSgE2OWg0dpkP2+67Ctwd6DJjzeXXoLLGRsMMo/AMyd8Y= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3792.namprd10.prod.outlook.com (2603:10b6:208:1bc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Mon, 19 Jul 2021 20:00:22 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:22 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 04/11] vfio-user: find and init PCI device Date: Mon, 19 Jul 2021 16:00:06 -0400 Message-Id: <9c728a63a896ecf3de92a269668c3306cc3e6f5f.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d8cde697-e858-4aae-28e8-08d94aefd7b6 X-MS-TrafficTypeDiagnostic: MN2PR10MB3792: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:248; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qnesEXu0bKkUOP/ayEfAXlOVpAmRSC3eArqOiXr6YLrT/y0OvEYUC2BC9zlhexW5fynuNXUyV+DLf8SAMmGaZbvldJYCzpnQnjGfaYY9A7ZvSzAzVwrFRM/qiKLri0h/IOpt2grZMAEIcdwaJhlA02kCx4feF/QEnvm5ebVdsPb+fKUsSHJVErW6uGgdJIXnZH+Gk8jXtDoDsd58qpqHo5YtegN73xBFQJUpdDMq33Uddgq9UXVCtfze3koiqAnGsSMg9mN2wI94KDCcTPQZj8A3T1AOBAWIy4XGldkisJTq9acxbLPdqaZapGmDd4eiEy3tFLOHMDxSkwZBBN0tyG1ckgKYQInmP89y0YZM860mPGwc31FN+7UEnusGgrQ4JbxWE+JgUIfjwTIbPX16MholuZ2nFuEsRrlfRK8POpeNSHLss0hCiWV1AALKFjm+n352PNr9Tj6hjlzjnWcNhtucDOyzeQL4nf7gwu5wY3GLfMnsJqZCYMMrr6yvYeze+2zDSSM6g8NXYVN9YfoWc05Bz8DZKN/XpI3RlYKON/t75yY6NIwLfAcz4/7btQ1NdDkPczojHYQIO8Z+fWEyznRlOra6QFES5KOcKgbO0t42DAWbdnh1aI2B3oBTqT/Wd4PQmA5YluJ77Tfrcp0ScGoD9gdilo5CoTELFQETouvOVyrTwBQ5XVrc3WIM5hGsYOVkgqKlD6NYm2UQncLxcg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(39860400002)(366004)(346002)(396003)(136003)(107886003)(8676002)(478600001)(36756003)(6666004)(8936002)(66946007)(86362001)(4326008)(6486002)(186003)(316002)(2906002)(5660300002)(52116002)(6916009)(7696005)(38100700002)(38350700002)(26005)(66556008)(66476007)(2616005)(956004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: W434D5yjvzuF+yHDRf4eyYdcNvFYhjz/P0Q922jevDI9XzW/Oy34IDohTacuDDh4HtojeMBV79GMx9dotY2Zes3cwPu7zzHi/vmR5ON6HIDKtD+Cko/1/2AyzweYxQoei26iyXbAnuYd7JdK2D4ykIdIQACYYPpt3YMi/sD7NeLes7EBr7mndgL79bJsbaF7emIrP7FYjgZkfwi2qbjusy+WVjJXmb3R02oHvLzmWBBVlns2mQfGai2o4NaYtNJ2PPiDKHLbp3A7bzBYleFDeGvNXgfs72I0KWqY5yPdOhpIBqRkNwrf5y/tdmVTPIbR9W2mbpSBngL/t54LkqjWt9+52DY19keeAK4bsvrbgY5DY2MBgZ6H8lXPtoF5rldTmtSqbxZl1Es/zUQn5Pz1yBfl6+WK5x2jndW6QXcI2AKIUi7+6iTTfZhJB2wxZgnutJVnSuXxzbmKz+m4T1xeeekMGCnHyFVz1ascggmmE4fdbTPPr2YqugtJmedBFI49QrS/YohWDABdAik2w//vLdINOQ8YIp6mNwa1stGAKSLXRfsDZRwlFcNwyjD+RzM94SB2nvHMq7qSv7B5P7w6nEky4WT5iSNVr9E6jxfJTGrtOSfB0FN4OmrTzwlKKYezA29WbGgmUbUmrSgI+NxwD3ukdtvw3KlEiI1Im2Z/hNiy5ALb5fSTTN/coFJtsdLeV8Y4/K0WHq8XeVwue82KHzBci8YKvEr9qasdJuuvz+56JyktzsEhXYDuSN6JTz1+ X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d8cde697-e858-4aae-28e8-08d94aefd7b6 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:22.8393 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qjhVwYJx4yxkbGiGzm7FfLTwCJG04kl7NMzLqs5/2muQxMyCPXwAhr6iczaJdFB7AzrqHBJ8qZybmnPtKzN1YQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3792 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190113 X-Proofpoint-GUID: B5XgypIMVqbOXziw7U3aKHFQOLDxN8VX X-Proofpoint-ORIG-GUID: B5XgypIMVqbOXziw7U3aKHFQOLDxN8VX Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Find the PCI device with specified id. Initialize the device context with the QEMU PCI device Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index adb3193..e362709 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -37,6 +37,8 @@ #include "qemu/notify.h" #include "qapi/error.h" #include "sysemu/sysemu.h" +#include "hw/qdev-core.h" +#include "hw/pci/pci.h" #include "libvfio-user/include/libvfio-user.h" @@ -62,6 +64,8 @@ struct VfuObject { Notifier machine_done; vfu_ctx_t *vfu_ctx; + + PCIDevice *pci_dev; }; static void vfu_object_set_socket(Object *obj, const char *str, Error **errp) @@ -89,6 +93,8 @@ static void vfu_object_set_devid(Object *obj, const char *str, Error **errp) static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); + DeviceState *dev = NULL; + int ret; o->vfu_ctx = vfu_create_ctx(VFU_TRANS_SOCK, o->socket, 0, o, VFU_DEV_TYPE_PCI); @@ -97,6 +103,28 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) strerror(errno)); return; } + + dev = qdev_find_recursive(sysbus_get_default(), o->devid); + if (dev == NULL) { + error_setg(&error_abort, "vfu: Device %s not found", o->devid); + return; + } + o->pci_dev = PCI_DEVICE(dev); + + ret = vfu_pci_init(o->vfu_ctx, VFU_PCI_TYPE_CONVENTIONAL, + PCI_HEADER_TYPE_NORMAL, 0); + if (ret < 0) { + error_setg(&error_abort, + "vfu: Failed to attach PCI device %s to context - %s", + o->devid, strerror(errno)); + return; + } + + vfu_pci_set_id(o->vfu_ctx, + pci_get_word(o->pci_dev->config + PCI_VENDOR_ID), + pci_get_word(o->pci_dev->config + PCI_DEVICE_ID), + pci_get_word(o->pci_dev->config + PCI_SUBSYSTEM_VENDOR_ID), + pci_get_word(o->pci_dev->config + PCI_SUBSYSTEM_ID)); } static void vfu_object_init(Object *obj) From patchwork Mon Jul 19 20:00:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E3E2C07E95 for ; Mon, 19 Jul 2021 20:06:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DA02561029 for ; Mon, 19 Jul 2021 20:06:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA02561029 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZWk-0006XY-K0 for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:06:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47934) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRs-0006ox-TK for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:08 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:34852) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRo-0005oT-Ij for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:08 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuV6m015851; Mon, 19 Jul 2021 20:01:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=XFuAWPBjq97Pne6SqCqr++S7U0umgtMzFkqVnkMAFsk=; b=NEi7fypqfOPjAnDfn50zsMOPBxruZ4+LpQpX8g3z7gy0R0FWOq/W4WFxbNCYo7JSdUG1 RTgcw1Pr/e1P+2RhAwMexgouW/pgefCSkqccB4Vh9ErX1r3FJP5xCmBcQCuCye1nWD0u DDra2fhvskssPMAYVcmRiinm2aWsaQVkRtKQTpEZZSxizTb+etjuJE6vwX6lgBA2oeS6 Bf6yUGF6R7PfTxDryU8xQrytZvevM8FLxzgBsI9zfROZ1pLyT5wi960a1Vn0GA8D0jVd TPW6K1XZ2QWG1ujMFA6gqi689upQWjycSx8CjwsNgPv4G0ZLZYqomao93GZOnf7VYbaD rg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=XFuAWPBjq97Pne6SqCqr++S7U0umgtMzFkqVnkMAFsk=; b=JKBBZipm2FcpUnh2QGix2fskvno/nlIm6brsmMPtKyHouezzLu+Ifu/RdUO+8nIehhBY wMsAJoRjwvsxZbFdu1Jb/R5lkBXuFErrZMFHdHBwl9kjeQCiOWfkV30rh2+bI3llALHJ 7JPuuKypNuxkS5ZlHvAFyJaCovwAZmWhbNGZUu8ErQRryx6Uv3PkIEtRHH15F9lXGIOd +w0WYXlTGlKR5wo7XU8zWL/xUYMFVRiPu3LFC0v83qIpqNGiOPNRWeh86whVs/ahJHZO SL0hgsdIf71kF1OBdblXdrpZSxj/ddy0Ux6hkNhuw4rDUq4Y+ynH1zOsP/mqle/n9gmC jg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 39vqm9a99m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:01:00 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JK0Zpk127869; Mon, 19 Jul 2021 20:00:58 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2101.outbound.protection.outlook.com [104.47.55.101]) by userp3020.oracle.com with ESMTP id 39v8ytbgn6-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K5jt+F/uOEygx/u2ABT7DlXHqBysBtnuqYeYlLQYGzkz3jc1ILceO6mFNGfDCJxpcQY1CFmaXbUjuCQEqkSeqgCqYzL3hlRqHJ7Or+RGDal7WDoIN4hI8GX69B643G3/yGNGM+dZw3ybsq4pIB36wZWifDEXW47mYY4LDmdoh+27gNRH1WJGCNF7Rx8uNfm4Ov2vbuD8s6TZwtz/Fls31Yb1vFhOU78bhkJwd4Mu529Q9kW41Iuofk78AK9xreZSPjCIeYMFqyOpbE+11Nc6+QAA1wVufOUjFYfyJuvWgXzlWtW34ltQlKpIZH786gxZOaVOK3lo3hEftKZ4lM8IQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XFuAWPBjq97Pne6SqCqr++S7U0umgtMzFkqVnkMAFsk=; b=RlscfAYKxaI+HNgcS+kfQipG3LtJnSUqRoQY5zO6D9CHTMfalwj820xHDAWF8LY4FSLQx5bYEL4DUwJDfbELLAXzjWuuy7M1duc0ekXwrOpeN1nNhUr6HTUmVclYkl9JaaRSks266CHwNAKP7Jiasn7aKIZjyMX+1t0TaUKzCcsLuaJBh+i4/k1o2huoKtMqULMK30qp6ZdNWP3e4z3G3lCV0KXCfLDy3VgS4UCgAUJ3miPdpRycew15jOdv/IvRiz/Ut4oQzvP5QbO6E/NLqVfUamugMVujvwfmxPURO4Sp2elg36WvTq5vSnCUCutnnmANFjBbfYldsk8Fai0yqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XFuAWPBjq97Pne6SqCqr++S7U0umgtMzFkqVnkMAFsk=; b=X2tAk5k0fOH27HMefLNwrgxAK+bekYCt3H1lJnfV3l8jG+tZKzr69f2Y2JCPdsXGm1rqR/msSLkt0ZWYB4vK4LBxmC7DCkLyirTaNwV2wr8GG6QzHjeTaVl8tY4ED+cWMsq2UGHZxj8OlDC5n+dKFmTXi62/nH97OlFFu/eyYb0= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3792.namprd10.prod.outlook.com (2603:10b6:208:1bc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Mon, 19 Jul 2021 20:00:24 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:24 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 05/11] vfio-user: run vfio-user context Date: Mon, 19 Jul 2021 16:00:07 -0400 Message-Id: <9ea4e77a39aaff3dfac3e463d3de76b0e4e1844f.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:23 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 892f5012-e415-4a49-8ab5-08d94aefd887 X-MS-TrafficTypeDiagnostic: MN2PR10MB3792: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:962; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1DsNK7nrb9/olG/nhnpJ9NiFdpQNnJjoGW7Y3Vn9KpNTgiPL2GYr/2O7KbeEn3b/Rn0JqLTK9BwKRO1t9R39KPvU4r2ZArnvPFWmLuxjb4wwfmzc8Dtq2g0ixdaDfLX3C0eQnK6Vw/TlPg+q5RX5jguB5NRYbqVSOn5PzdoN/QP+NxZB2PYVhRMgoG6Ceftk5LxTzcrAljAKKEbOLRdU0itvARp+4mISl8HpjdGtBXqORI95ixQE9Q0hx7tUJ++wR+vBUYpLxHiK25XtYv3mNTFmfL4f10j6n8aaOHOl2uYb50xCl3o3usto1eaVs85B+rShG5dYKaQZYvcOSbxs9JPzQSFeFyUsTSxOetZeH+R2cJmvq40kEPBGepzJIShdesHqTcGRdixkNr0vKw3u5rx78kl9iy2++gktQ4I9oMTKGTOATtM4PevB2LNpRK7vz43lgNNiCpBxF6PDid7sUC4mlT8zX7Xo9jGp4O8FbsJhQVvEpclI3HdY4qZ2XR44aM4eWCTYp0duIagaHfC/3E55ZThRqyGzIFFmKE1keAHwUCt0o5FqWL+zXsCn2mynFAGlhxesKh/yJ0o0SUaGWnhWF8W9xXPcEZJNPydGf/zXBnoA/ndSBQZeWGkmzxpytvaP0Ys2eMScJgGbzxK7DUO/rPgSQ5O9a7cVaa4v4tYDxbf3DqD22OerRfJoTeVWsGMxiVLGFuhO/tZtN/wmHg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(39860400002)(366004)(346002)(396003)(136003)(107886003)(8676002)(478600001)(36756003)(6666004)(8936002)(66946007)(86362001)(4326008)(6486002)(186003)(316002)(2906002)(5660300002)(52116002)(6916009)(7696005)(38100700002)(38350700002)(26005)(66556008)(66476007)(2616005)(956004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7ND1694bkqS9LUEgWk8xNRX7ifqNvOywWbyoiKTvq+ziGxLxTeqWeX0pACNIEPgdP/QDGTZQc1Uv/5vXrhKAISTWnDfdxFdJi5aYRCQWOSGHPFKRTZU5B3AEYgg7myslj2kFNULDLVVEy7dLtMdzu0nLKjhKtdeko76RKcTaAfibqwx6xg94j3k2rZBvemU8Sk3M1W+mkVr3ZKIeuDkGITE4UgFHH5d7iV68YJgj957bcMZvICMqdbfHhnNl6W4461FGA0TquV+YI6bLVtHV7eHk3W2biu57QTl1B/6H5UlvP/2ieLR/Q1/s9MoEYGqYpsPiTZjJyZE2GUmC5AtEsaPVfzj1I9BoAwcqoe/ZmGBaG8IoifSkRvGQ1AY6Gvjs1kkjo1TFy2r7UU0aPttcQ5bw7Hsv0/ZZjKjwmUNgmJCF/x00DnjkeIsVf/34wZNjbbptZrTOrzkN0MwtTAJKSCAoN+tWtyqJYWse+P4YmVZrsEsG0zTA6SsfWMrfN0L5/YNd2m+LOyx/DcvB1YF+/aZt6+Z/+YjmynyALAbnU0mIji6RMimXLBet1JGur70J9CEG6BP9mWlR5WDNc0s1VZ8OqqTVMr4oe+fdJ0bP2mbE9SSmZMkik8uM7PEUvCHo0sXe5W+1ptQtWb3ccz4pJnYqADDQHlisrNz5oAc1kjwyneSLtlk0l7jPnjNDAiRDBuzWm8jGLt9F+2A8G0Owu+19Nj0DOHwD/5xF0gwMniXdMk7yo44IdFzZQLR5+FgB X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 892f5012-e415-4a49-8ab5-08d94aefd887 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:24.1685 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: etHEvzsTbCYGpTPZbjXjpaALhYCSa25UfIJuSchi/FVBUGcUGLpqeookjERto3zhiL/jnEKFMCpqgkC06cMQNQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3792 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190113 X-Proofpoint-GUID: vyi-30tiir--e90nFLnCYSBJvF0W1kdQ X-Proofpoint-ORIG-GUID: vyi-30tiir--e90nFLnCYSBJvF0W1kdQ Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -8 X-Spam_score: -0.9 X-Spam_bar: / X-Spam_report: (-0.9 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Setup a separate thread to run the vfio-user context. The thread acts as the main loop for the device. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index e362709..6a2d0f5 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -35,6 +35,7 @@ #include "trace.h" #include "sysemu/runstate.h" #include "qemu/notify.h" +#include "qemu/thread.h" #include "qapi/error.h" #include "sysemu/sysemu.h" #include "hw/qdev-core.h" @@ -66,6 +67,8 @@ struct VfuObject { vfu_ctx_t *vfu_ctx; PCIDevice *pci_dev; + + QemuThread vfu_ctx_thread; }; static void vfu_object_set_socket(Object *obj, const char *str, Error **errp) @@ -90,6 +93,44 @@ static void vfu_object_set_devid(Object *obj, const char *str, Error **errp) trace_vfu_prop("devid", str); } +static void *vfu_object_ctx_run(void *opaque) +{ + VfuObject *o = opaque; + int ret; + + ret = vfu_realize_ctx(o->vfu_ctx); + if (ret < 0) { + error_setg(&error_abort, "vfu: Failed to realize device %s- %s", + o->devid, strerror(errno)); + return NULL; + } + + ret = vfu_attach_ctx(o->vfu_ctx); + if (ret < 0) { + error_setg(&error_abort, + "vfu: Failed to attach device %s to context - %s", + o->devid, strerror(errno)); + return NULL; + } + + do { + ret = vfu_run_ctx(o->vfu_ctx); + if (ret < 0) { + if (errno == EINTR) { + ret = 0; + } else if (errno == ENOTCONN) { + object_unparent(OBJECT(o)); + break; + } else { + error_setg(&error_abort, "vfu: Failed to run device %s - %s", + o->devid, strerror(errno)); + } + } + } while (ret == 0); + + return NULL; +} + static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); @@ -125,6 +166,9 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) pci_get_word(o->pci_dev->config + PCI_DEVICE_ID), pci_get_word(o->pci_dev->config + PCI_SUBSYSTEM_VENDOR_ID), pci_get_word(o->pci_dev->config + PCI_SUBSYSTEM_ID)); + + qemu_thread_create(&o->vfu_ctx_thread, "VFU ctx runner", vfu_object_ctx_run, + o, QEMU_THREAD_JOINABLE); } static void vfu_object_init(Object *obj) From patchwork Mon Jul 19 20:00:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F7D3C07E95 for ; Mon, 19 Jul 2021 20:04:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F1BF61029 for ; Mon, 19 Jul 2021 20:04:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F1BF61029 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38382 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZUm-00041X-Aj for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:04:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47932) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRs-0006of-PL for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:08 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:35634) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRo-0005on-TY for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:01:08 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuFaE017810; Mon, 19 Jul 2021 20:01:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=buKXsZW1r1dXgNaXcJjUYsRNDpxKV1+CO7ZYDS7h1+E=; b=z2InRpSX5iGNCBFNS18odWQCRMt332d/ms7QafPhvLf2TFvY8GKlhiAIo9HJn6OuXX/z ZWSFBpM6ndE+19y+K+8X5BYwtXq7Q0CiNQHk7lFtEL39NySjupsf5aTAmiplRs0H5qnq l/Cx5fT8dLDUd4r8N8Aq6CpW3Wix9MBkXxC+uXzVLbgqZDJb3PcvWY4tkq2ZXsmsVK9T Cn5u85DCv91UH1XJ3AAvDi3cCKv513gouIJGPlDE3m25Nd5HdTplatOQsEpwE3Ya7jJi V5ImFtdbo6pLobP1MJMVQlxRIheg6sZXdka863MomS2l/pf5cMhD6Mpq+NynEgrWABls 8w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=buKXsZW1r1dXgNaXcJjUYsRNDpxKV1+CO7ZYDS7h1+E=; b=n0l4Y5VCGmMMhtD7War0U00cC83rffXI9LWlHUXzbBfNWdS9qBwOUpTtqJB/7kR6iApY dpEWiR7/MOc6wOtFAnwYqo5R5KpbCuKWTJ7er7zdV2nx/hs0zInvEAPwkj8GgwmFiBdc 2jiz7zdunhDpptMD3ID+MlOQoZv5N8QdzSsEWGR0PJJwJgSPYpa3F0JwjfzuK4bDYnqh xYHnDLbAJLqmmn2xM4O+qUB0RjJG6kh6AmmJvKQe8tDaUDavV+7VqQ3bKFzEzxrqSOBk IwXJPsKB8KRmSdKjBxGV1PVPB8yYcbhZlxQiz6uH7I0QmzA/a2avKVWWooOliUWs4qo8 Sw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 39w9hfs0e2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:01:03 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JK0Zpo127869; Mon, 19 Jul 2021 20:01:01 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2101.outbound.protection.outlook.com [104.47.55.101]) by userp3020.oracle.com with ESMTP id 39v8ytbgn6-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:01:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LsuSaoT2bQCX9MTFx5TUYxI1R8288RKgNQXvcSBUfAs5hn0nqhcAeFT1Mx+iTQyu9tbncLza+MYFQLQz+1gc1/9IzNpu0XnPCYJF6osDmYtaBIpeAK6hfbbuoME2b81cabXxiHjqzd9Cp5b2hCQNkiBGtjLTT5Kq6wUdkaL7Aj75CCwclmVzoFC8kL2Gk8tVCnWedREA7WGXLBVe/KKVoeP4/FMp+GV6jINK5yQIt/fiPo6XBvPIbH75Lz5UMvGDcLAqkJ6cRODEmcg48kk66eXdrx+D4MY6KfRuQQmHsSBmq7xjIliQi2DYKpY6BXAD7VORYAFwBuWl6Bc/Slo6qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=buKXsZW1r1dXgNaXcJjUYsRNDpxKV1+CO7ZYDS7h1+E=; b=gt77cTTIiGpLZ5vyNncYq+ExAwfZhw2k9TaPcXZacXsgTOfVSvcwnSEItNcox5qdVG8KxVThV/cGKBJwfZFxgeb6rmyAXYfYotFWUXGf9mhoSgcv+Axxz8fj5K4kfrQ5uXXJCUX50ioOTkqlwItl1ZYgE9dI9RZIxseJ9DhDYYVotpeg7n48Z1W1XAwrBxMcTxJ8EZaCLU5hkZSDsc7HOeU0LzXBzsMA7pqOnEw7kr7mV0Od+xY9mA+AkyDlh3ZK2IMzPcA3rrgEjtmXaDWILC0bc0dkAZD2x9Vd5vGXDbGhuPSvD7AgHktRo4OP857ix1dkw7n2crXvZ0WPOXDPxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=buKXsZW1r1dXgNaXcJjUYsRNDpxKV1+CO7ZYDS7h1+E=; b=ESJe1alE4SmndQiaGlWx6/BVhiIhND5hrrb/muEUVMQ64XwO1gS/qu8jxWVOdUu7l942wOUmJgcU7cyvJ86Q9PsoWMSp9nI0HyrQvnnoFXrSF9mBBVS1/hL+2/fiikG++8HRq9In6qiQ57UzhKTXlTHTFkrcvKNkxTQ0GOMt6Q4= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3792.namprd10.prod.outlook.com (2603:10b6:208:1bc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Mon, 19 Jul 2021 20:00:25 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:25 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 06/11] vfio-user: handle PCI config space accesses Date: Mon, 19 Jul 2021 16:00:08 -0400 Message-Id: <791f5f9a010b14844e5210c4390c8628ffefb80d.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:24 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 16ad9c22-8df1-468e-80fe-08d94aefd94b X-MS-TrafficTypeDiagnostic: MN2PR10MB3792: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:66; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 57POb5rLMcLrylcuG6a7EErXTnqkqI7whFOc3tiJYMn77DFKkgfg6uywgKTpOlk9jE2XSapRaRYirKol2yQTY9/NuyEt8O8EuV/hItrf8dzMir39MyLHAB28bwGtm4SVa15Q2aaqvkTAsyx78WwporRZx/UIz0GwDtF1C7dW+4cyzyZeWzWBabx6wKRvX8kDd1wN3n7o9RpYxYNeJpNaWOW3dGvUW3xPCzzP12TPq0Zd0sVpRM6yvKllb/LXKbZSXtECv8nrfZo3XFHZENy67Ejs2xNT4ADh+3bUZsaF8Pr8PfNy29Cf9QGsHJPa/RMBbzHV9H47DjQLnI2HhkDbopL+JvbomyGkM+0i6VkhXlyIRTwmvJ2XxwioOgKiLIwXTt7YHiWQLblJ4n6pctNhW2NaIppWlXhd0xvnXjDutr4nTFwlPs0Zxhu6RadnVeXxQ8TwSAc3XljgNzyjPSRa6B7MN/lXMVu5YJ07tTvUR69mg9QzbJ++1WMYQKcuFjQX5aHTcutmGOQxIMOHvUB007BV+PMJ3H+u2T7OmSo+KgP2LDn/W/YNZFWAbFjX29zM71pOZ2ntFg95T0KydQmDVD98EDVGz81U65UbSdMCCpcnzP/+NRrh7dyKeShWENXHskbgfcy5GJuIkgMyUK11nE5Yw2awq8jcLBXLo7Nrj21AhICMHwjE8ER7Vn4dkc+HKjswyBkkn9c0GwGQmFwxng== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(39860400002)(366004)(346002)(396003)(136003)(83380400001)(107886003)(8676002)(478600001)(36756003)(6666004)(8936002)(66946007)(86362001)(4326008)(6486002)(186003)(316002)(2906002)(5660300002)(52116002)(6916009)(7696005)(38100700002)(38350700002)(26005)(66556008)(66476007)(2616005)(956004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CNNaTZUfzU2nWfYRaal9eTYTB/scSCLZ7tIjTgX+fcMUY7HT5wkj/sHlr9oX9+37NC+hu0jFjZSyp7YjFox//v3NF9pQE+LKj19BiwITTKq72Fb8Mz4jME2VRgkWg+FsBVnXidFoey0Nb4j71Iag1WtHUQBo3Tj8nr9tJnVnWyUBNMH758esGAAD+MmeXRvmZF1BZcn3YjZV+d0uhZifR/rusdugF/3+5i55zH9zGCcJUbSivlnTDTo1HeBzBwg3ue3DmZTYyquY1lWbYaZJYYanD+BTjEEkQdAg7Zf2ou68efpSUMt3M9inmspe3XQ769FG2Iz9ya+N4LniBrOHefRMvdrq17rkuJngxP/18sr1spX/GxRAoYnY0Z2Avii4z2LN7SsEVtAppnFlDWE4VUIH/jzyx4bKuNfzoXMUdcyRkytZkNlP4wehBfWgMEIMouLO5dKZ32T9k0Sl9v0bIYkNbyAAcLH1ss2AIDAYWMI+BLkiu/7Sz7IsPmXMcMzDiZTNxmgbJq5YjjOxtluypv4Gl+kixtOF+ahdKxdhD9AT0r+q+NAXqWVAwCXV8GCcSU5rFovuRYzU0l/Nm2/MKzECjI78DyeeFR5bC2tOXXKe2F+Zy0qJiHRbeZXxKivyp3XsGlH1QxOgR3FqT3ZQKDRHQky1UVyY8N+vwdmZbMiUq2KIJyLKyDAlogh/OaXMKMEWIq3paLiOqPvu4pNrV+npSkVYSMJAOgok9ZYHn2YM3Go+ThcSdzLWx4LLitOW X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 16ad9c22-8df1-468e-80fe-08d94aefd94b X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:25.4887 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yi4T/9m0+5/cxTo8vmGCgXA7eVPasxZjdYbam26caQplDDYjvlrIN5MZgHO21u3g0naeV0EE4n+UUS9ke//dHw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3792 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190113 X-Proofpoint-GUID: 1IL7pzAVr9qCcCaYQ9J_wBOfJgtRA84v X-Proofpoint-ORIG-GUID: 1IL7pzAVr9qCcCaYQ9J_wBOfJgtRA84v Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define and register handlers for PCI config space accesses Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 41 +++++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 2 ++ 2 files changed, 43 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 6a2d0f5..60d9fa8 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -36,6 +36,7 @@ #include "sysemu/runstate.h" #include "qemu/notify.h" #include "qemu/thread.h" +#include "qemu/main-loop.h" #include "qapi/error.h" #include "sysemu/sysemu.h" #include "hw/qdev-core.h" @@ -131,6 +132,35 @@ static void *vfu_object_ctx_run(void *opaque) return NULL; } +static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf, + size_t count, loff_t offset, + const bool is_write) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + uint32_t val = 0; + int i; + + qemu_mutex_lock_iothread(); + + for (i = 0; i < count; i++) { + if (is_write) { + val = *((uint8_t *)(buf + i)); + trace_vfu_cfg_write((offset + i), val); + pci_default_write_config(PCI_DEVICE(o->pci_dev), + (offset + i), val, 1); + } else { + val = pci_default_read_config(PCI_DEVICE(o->pci_dev), + (offset + i), 1); + *((uint8_t *)(buf + i)) = (uint8_t)val; + trace_vfu_cfg_read((offset + i), val); + } + } + + qemu_mutex_unlock_iothread(); + + return count; +} + static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); @@ -167,6 +197,17 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) pci_get_word(o->pci_dev->config + PCI_SUBSYSTEM_VENDOR_ID), pci_get_word(o->pci_dev->config + PCI_SUBSYSTEM_ID)); + ret = vfu_setup_region(o->vfu_ctx, VFU_PCI_DEV_CFG_REGION_IDX, + pci_config_size(o->pci_dev), &vfu_object_cfg_access, + VFU_REGION_FLAG_RW | VFU_REGION_FLAG_ALWAYS_CB, + NULL, 0, -1, 0); + if (ret < 0) { + error_setg(&error_abort, + "vfu: Failed to setup config space handlers for %s- %s", + o->devid, strerror(errno)); + return; + } + qemu_thread_create(&o->vfu_ctx_thread, "VFU ctx runner", vfu_object_ctx_run, o, QEMU_THREAD_JOINABLE); } diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 7da12f0..2ef7884 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -5,3 +5,5 @@ mpqemu_recv_io_error(int cmd, int size, int nfds) "failed to receive %d size %d, # vfio-user-obj.c vfu_prop(const char *prop, const char *val) "vfu: setting %s as %s" +vfu_cfg_read(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u -> 0x%x" +vfu_cfg_write(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u <- 0x%x" From patchwork Mon Jul 19 20:00:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31A0BC07E95 for ; Mon, 19 Jul 2021 20:02:32 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C34561029 for ; Mon, 19 Jul 2021 20:02:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C34561029 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZTC-000064-OU for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:02:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRO-0005rd-Pw for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:40 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:23516) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRL-0005QR-Qd for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:38 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuFZl017810; Mon, 19 Jul 2021 20:00:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=Nf4UGqSD8Cux2WYhmD/1z0z+12sOdGzep4OdKuHLeE0=; b=V9FD3hMqkmFyGLTbj2lA/cfTphcwgLtdw4QG81MK0+xu2pBXpNzNlZpjamNQFCWciI5S rEDcNofe1LRaKq83fQCKVtA/yDMkgkX9C7qwfNqmAerwHod/QCUF2Gob1NyK29CMqoVp ++AEDZExFdPdgCbx4A2HGc5Uf1yytb2pTH5DRN6Uil6gv//Pqz9FabDKys9glsSkHUTu 1ctrgr6Z6xIgl0o/Tt/Vs4CoDx5Uu0Yivnk8XGSiY//rtlToO9K3SBQcgP1BQO7lFSFY LDz55NkG4SXmRHXQupu5huyKZjFExtRGn+Fm4etEt6EjA1Cy5O63kBbo5dXHgdUHIPzA 0w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=Nf4UGqSD8Cux2WYhmD/1z0z+12sOdGzep4OdKuHLeE0=; b=xEDnGT1UUQv7dRk4WuKbtqTfQlMUnAz1w8ePcJrUIE+Cr8ldSIpyM6yNgwbxl5Bhe3yb gaLz1BGHqOo7HOue0Zrcw9ckBdZpleoCl6zb1Zwt1rui1XSaBStjB+I46wOls/HkR/RW opXbPocUscBaHOfms14vj5OzEaBoRUIdKIZSbOBQ5tWd/hsLRNRFSxM5MLfNYE1vXz5t 0/pRFtmzOvKDjrDe0bHctlUrUl2ODVdk0fabRaMVQr4tY62VuhCEspWupYVvh1dNXcnC O5YiiTDvJlHL1j/RbccvV98t2F/zg6hxWv95L3tv7z9f+xYWOf6CBIJ+5mfoRlBAYYHj LQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 39w9hfs0as-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:31 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JJf5Iq033120; Mon, 19 Jul 2021 20:00:30 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 39uq15b6up-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kBVp3uMZ/Y/uwScKUe+JmVAeCFWhh+X91ld6/EA3fl3fV6flU2iSrM8MikYnDlHAnef9qD69OjNT/B7rzZ/PmQGDvyI+ODnbu8IzczOKg1BOFSPjnALP6RksdiVAV7Ay39GCdtlkeDjRTUrezh4TZpEPmKJhkRalxVwbLITI/PpYckrxcRMb00sJWdIQZjAuRx7Zgb0pCdWn7o/ejCdWRFOPSrHaH1PXR9s1yU/SyNHSZzvslKWniw6XAzCTs70kvx0+oBSo32t7vZBDw1TgdMHOtQ5iuXN0JkcSd4zTrGLMfPe1egPyHcwUxY2UOS9AcXicmHceaut/7AYeTtIeOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Nf4UGqSD8Cux2WYhmD/1z0z+12sOdGzep4OdKuHLeE0=; b=eiotXviqR4lqPKbiZifPVZ0GpHxqW+jJqMkE+4uHbStD4U/4jZ2/himqy8vBEI6tN4o3XicbDrKhUiOPAUBUSgvSaEEJz3R9hrfvo4rk1uffmaAHr6fr+qaJndTarBypR/8OeziTgn2iIS8U4njkOA/2l8Kg4tpHW4bxON+I8jAi/ulYPDy1vUld7cHpytCMnq1QxhueM5FL/YRIDUDr2Awci0+5AJUDVACsAm6dDdh+Ah3041ecU5zrDjCLzx6+xGf/GkNbdxB1lFzirpzfqOJOk15CpvE8+ATgAMoE72EKGJ8C/HpFSVZe6tTftc18+IyZ2O7NEoXsuSBw+RtxOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Nf4UGqSD8Cux2WYhmD/1z0z+12sOdGzep4OdKuHLeE0=; b=HJj2YimXoWYe7/o+DyaeRTLBIr6NVV5L0T87NCKzKD6oz3foaVCAF/N6LJG5Yxy8MIMuHU+9IDZlzBIxDkmmN+sey1QjXJGbiIyYB9EtU930JqCU02EmGU7EKmrt5LVdKs2imZ5KXHBATS1+BXyXYa7i7IKMLs0lwDoHGFumq/o= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3853.namprd10.prod.outlook.com (2603:10b6:208:180::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Mon, 19 Jul 2021 20:00:26 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:26 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 07/11] vfio-user: handle DMA mappings Date: Mon, 19 Jul 2021 16:00:09 -0400 Message-Id: <609dee10d839194f1dca5aac3064e75c408fbbdb.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cf56f81b-d985-4a61-5886-08d94aefda14 X-MS-TrafficTypeDiagnostic: MN2PR10MB3853: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:339; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AKF4cNprNUYPWoLBzZmf7OzB/73u3ovTDip6rFJG1Wr0Zp+0tkpfGxI589zDoAZvUdEHTKvxgMqxNHaIRUNUj7jDiBVrQMSwrGQTet8RwUAuCaC8f9y/rNXTlKHMs0sLqpgK5sr+yTy+06dmczbmpZTJRzO8jRcOv4HeT2TkhqCigirf10QihDms05TW5ZIIqZfMG7+de7NPPllkyHzftq7GIOuRwX0ON0Lkl4eLSMgY2t05vTC6T1kZuL0e3pXgU5AxvFV1a2QtTrT2BtzlbfleYOmvUBdM89oSYgYdRiUXQwYIFhTDqLyVDXIPuHn1UjGRIPpOwMSMRnz2o2CnZU6lEagdOBrBF/SVmcWaZ4vtOQx/vna/L16hBtELtqFukaEemZLQ/08VqVLqz4p1/vnLOernNvEmXEJ8dUdXWJRhDPLU39JPnTSlNIobc8FKx5IyuM9+57FoJAXK0H20pJwFJeJGzeDdLvgCtx17AvaalRGX5gaSZEnzUqFiK+qGn3tkEux/KfMG+tRUGnWA5Snb4dc4Z7hejQ90LzEcm+cXLLm04Ml1WpOlyCYplIj5fKo2Chwul46VvR/Nh50tCv2XU4zO1jUOsBHc6eySzW+2TO8x29uYUgayDdxSuS7MyEyvc5ETu5SqeEFVwQxMETLCXiJuc2mxqCbln0QuQPCZFkeCTFp92Yh8HU3Jip6H/q/PPKs0fW2l/sXV8wVbPg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(316002)(5660300002)(956004)(2616005)(107886003)(7696005)(26005)(6666004)(508600001)(38350700002)(38100700002)(52116002)(36756003)(86362001)(186003)(6486002)(4326008)(66556008)(66476007)(6916009)(66946007)(83380400001)(8676002)(2906002)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DuASCUykPWfQhfgzwj/tWgP2arBLRattFL3q9lMe0tFnkD2Iehcpt6HI7sA1Hv3oWrkn0JyBGgoXbLVSn45PKb8F3a/yxRspzoI4m4dC+lcJsrZ2WZPrq7rstzp/tz5Gix/AXOgrgFP5SrfLGTxnUiNR0JYdCczIpQ1ODZI8dqiYzLr26xNRd3spGCqBE/WPne5ZYFd0nLd7chOP/HC+l8x64sR+4p9k2hIy5nEJzH/Iab6MGjklX/dT5Fp8l6JKYAva/c2kmBew1kV1kNjIf1xEg85Jf796KxoLNE/b7BzmJ7hQz7xruZEgAtmciDYTHLfe+PEPuGhf39CoWJef1KXztkJ0yKOUiA61pWAYz708xV/dNxwdfRIgjkYWsBWzm5IhzMkaEyjlgqljSkODrMnPjqJuTQct3/bM3AFr2q9MoqlkK4r9k71noehUDWout8kyCWriprtba3+zJyZD+adYnOQXfCKxzbCmWvnPocEQgUqwup3vs5HtmYbRrfEGHXFdU9wg1FCsyIorg03FF4/MRhYVkxxAFGjdxvG7xdsaAe4vVn8TDc3sMo4xanBb5EjUOT5IwbicYNkgPq0kcum9yI03YLLpQ3c2FuqrM4fzE+lfoIeQcbiM3yZIc9qihNnIt4gWugcheQIWI3iDwXfiaKRGewnvMSbHfDXx37J9GFhp0xdd0k00iHO3pfE48lL9ma2ksDZvbX1M2jcKO0qHcPQXfYo3gpFmilbTxF3j4+HaGSG7a9UVsQmTbgpR X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: cf56f81b-d985-4a61-5886-08d94aefda14 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:26.7720 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Xq4AbkfzmDSz3qL6XTAOjeGTN3K5dk9XtSSxhHb4dPKiD2UjFVnwaRkcFZbDXl1Alu+0aSrjU8yN+7YpLBTxjg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3853 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190112 X-Proofpoint-GUID: QT0pqD2Pi00M6tWe9IzFKiPhpqTQsAp6 X-Proofpoint-ORIG-GUID: QT0pqD2Pi00M6tWe9IzFKiPhpqTQsAp6 Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define and register callbacks to manage the RAM regions used for device DMA Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 2 ++ 2 files changed, 60 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 60d9fa8..d158a7f 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -161,6 +161,57 @@ static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf, return count; } +static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) +{ + MemoryRegion *subregion = NULL; + g_autofree char *name = NULL; + static unsigned int suffix; + struct iovec *iov = &info->iova; + + if (!info->vaddr) { + return; + } + + name = g_strdup_printf("remote-mem-%u", suffix++); + + subregion = g_new0(MemoryRegion, 1); + + qemu_mutex_lock_iothread(); + + memory_region_init_ram_ptr(subregion, NULL, name, + iov->iov_len, info->vaddr); + + memory_region_add_subregion(get_system_memory(), (hwaddr)iov->iov_base, + subregion); + + qemu_mutex_unlock_iothread(); + + trace_vfu_dma_register((uint64_t)iov->iov_base, iov->iov_len); +} + +static int dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) +{ + MemoryRegion *mr = NULL; + ram_addr_t offset; + + mr = memory_region_from_host(info->vaddr, &offset); + if (!mr) { + return 0; + } + + qemu_mutex_lock_iothread(); + + memory_region_del_subregion(get_system_memory(), mr); + + object_unparent((OBJECT(mr))); + + qemu_mutex_unlock_iothread(); + + trace_vfu_dma_unregister((uint64_t)info->iova.iov_base); + + return 0; +} + static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); @@ -208,6 +259,13 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) return; } + ret = vfu_setup_device_dma(o->vfu_ctx, &dma_register, &dma_unregister); + if (ret < 0) { + error_setg(&error_abort, "vfu: Failed to setup DMA handlers for %s", + o->devid); + return; + } + qemu_thread_create(&o->vfu_ctx_thread, "VFU ctx runner", vfu_object_ctx_run, o, QEMU_THREAD_JOINABLE); } diff --git a/hw/remote/trace-events b/hw/remote/trace-events index 2ef7884..f945c7e 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -7,3 +7,5 @@ mpqemu_recv_io_error(int cmd, int size, int nfds) "failed to receive %d size %d, vfu_prop(const char *prop, const char *val) "vfu: setting %s as %s" vfu_cfg_read(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u -> 0x%x" vfu_cfg_write(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u <- 0x%x" +vfu_dma_register(uint64_t gpa, size_t len) "vfu: registering GPA 0x%"PRIx64", %zu bytes" +vfu_dma_unregister(uint64_t gpa) "vfu: unregistering GPA 0x%"PRIx64"" From patchwork Mon Jul 19 20:00:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2020FC07E95 for ; Mon, 19 Jul 2021 20:04:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9AF28610FB for ; Mon, 19 Jul 2021 20:04:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9AF28610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZVR-0005ho-QF for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:04:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47690) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRP-0005rh-5X for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:40 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:24166) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRL-0005Qk-0H for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:38 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJudNv002502; Mon, 19 Jul 2021 20:00:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=3SxDd3ZXIB+irqGn7Cyg1NVPw9m9wYjCsB52ZuqV+28=; b=nXjT6oqzonzvVXGqgXO/X5ugp17OQJGOw+J1ilOhMQ+RMPS0WkuqE1sPLESjZnPD276V oi2Tbp3MIjDmd2wo80lshSj1ixLYNo5I6FOkxTUKgii0WHmOuyeVlDI/nQMjM0djYuMd zk9P9tx/bO+IBKNNLhFFICLc9LB00Duzx8DCV+9S2dVh80lxK30AYIiAcotcS8kvbPec kvhSy2ZaC8fEHS8/3oPRrN0eWp3LSpYJ9MRiker6FZ0uMSLc8XJsWG9s6C3SfhRqdkiR xagMZFzC3/q5bbfvtqDAsVmVt2UfunGBxgaXiXDsmeHaTUulST50K+SXBCRblkiVIj2X yQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=3SxDd3ZXIB+irqGn7Cyg1NVPw9m9wYjCsB52ZuqV+28=; b=SLGIqR/EOCBKbTNdOHna3V2BJM3xi82OQ7+Q3F7mhjgC0Zu4/0nQTyZTRr1c/rCEL+qy RFwuwL+JVm/VJt6u8prv2soak8MK7PJxTkZLLYEnvCW+9zc2wGSI2IXk7D1JX/oXo+Io bjiEJuNtI5ZLyCBjkskA4E1lqmu6+d4u+crNSinqFpZpi3BcKl1nE2ZLazP4kchmrZrt 04KlYl7nSBO2jxvfrEYtkh7UAWWZ+2dzC5XOjeHUOBHW7X83PbCT2dA7JRkj4hnduomI H24cd3iVVHFQPZ+kgXaPRNtVjRVgFzDhXAIokC1OZkM4Qw7kbGR08Jv9cFwC0L7T0UMa ew== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 39vrn5j9xk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:32 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JJf5Is033120; Mon, 19 Jul 2021 20:00:31 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 39uq15b6up-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gCUBy2YSJaF0EqflcdjH7pRGoVW90Wy15FyQ60SsYBfvWWVOQ+dt/U+Uh5C1eOt0udeMxJc+KSJIak4NZsrubqtbFXJwIyYTssWSYrknSeQ0g8MYD1EtKa9QXLrtljvzd+ZvKTR0mXVvkBQZYLD2pw5DnTafPlTf6TYPYNeBvM4mq5rq0FAacW0Zr8r2gkht+QCW+Gvjb/74AvHJzARKLqL14uolI1EnU9nU2021eWX09GNdmvWhvmkIWVvTgx+6B1L10es1ISMbXVt30nSV+UspwNaiQgvMzeGdMNSy92PDbj7zzI5YblDGuKUXmUOa1zFmAN05IsFtA9HTXCTl6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3SxDd3ZXIB+irqGn7Cyg1NVPw9m9wYjCsB52ZuqV+28=; b=bzAortg6QHFj9AeySy6cZxhMclLaY7cOh0QDH4wdatfcI3MiUJO2+eq+Ay2Kq5TxZ6bH5CGRPhXDnNFup0vorTGSk3HmdiA69qMhwie+jjRv33Dn4KSN6NkYtAKw9WW84RnaYaNl3r9yrn4Hv/SnwkEzAdgxauFO7ku6LOKze9RqSCEkH5I6oc1f3NI7sgFcdt8yb6oU08r/x6Nd70cCa6MPV2isX2aB5Rgivmx7DfxMP5UYwSRHjxbkOSLsvUNMyDjMX0DOgZMOVNPLzysdSEVRyiksnmIyZ/1aYGAJMqLrhAXbt52ztk3lKuByd0K33R6+L+xQ+ZHvIJZjWHwyMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3SxDd3ZXIB+irqGn7Cyg1NVPw9m9wYjCsB52ZuqV+28=; b=xoJ2vcYzZuCpgXNdkGkmmfzo5fmDTdBKuU8ugbcIZAjKteisPwsOHgZUkWocCE3zJVaTx5ysCPiYDkIbUAVYhB8daMzuUhka1a5IPV6nQwCVClHfFmegjem6VKD9fIqoUGmbWhT2H3CsHc19FTxKzCG6dULKmrgCToDMv6/pBvQ= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3853.namprd10.prod.outlook.com (2603:10b6:208:180::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Mon, 19 Jul 2021 20:00:28 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:28 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 08/11] vfio-user: handle PCI BAR accesses Date: Mon, 19 Jul 2021 16:00:10 -0400 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:27 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8287826d-0bd2-4507-a0b9-08d94aefdadf X-MS-TrafficTypeDiagnostic: MN2PR10MB3853: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:346; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NZyIbrhj5Ans6ZRZNNHfmu8ReJ5q2cucOyKJ1evzleJD3Ili9sqJ0jjoWEDgpMfMeAJyVxUg7S8I8giZmn+xbyzEQW0wOPdhn8z3Dwd6ob3FenbsUbRchRb2bkWW+IVAOKyvOYGjITlV/NehY61So3JE2LYTDKFEphTYoajNWob+dcsUL0DpIySQ59yOTSBH6j/SBP3XrXxCXQB28loXAT9ekrnTW5edB7PUK/lHrWFGmc2TsRY1Gng3P05Aoe2nU1tkFsjIpU4J5V9rBbGsnEUfgrPNnroKIOGJHzAKCXjIqQvufVO2sjx2s3ZL1HZ03aZHdWe1wUTi4GMNXglThYGJsoDAQLw9JZvtI148tKQ3H+R9I5mhKBOntvPdoFCZs3S+t/nRiCYURtiHJnnHeMbT+sQa5XywwrcapFwTh8/SXL7KBwx6i/RBmC8LhJQkK50QQrMNDT5i+EBDfYcxxm7/pX7vTS3lYmcK8c5iWA+H82OyO2lS3tcrTrZL20x0vGoATyq9gcTiKIH8FQ3hvpAozrVVW11yNLJ0/0KCqRc4i4q5C3TmihyvKYO4B31xOwms4eaCcTDZ2lEXHaRvBNtvJpaBM96e1ZOAyBP0D01FJO2BcNCTwX97470QiScrX+VHH55TflgTMccSMmUqSH4l0ll4ciq1FUo2r4c8crucaRpu6ORi6yhKQez3+81ixfN0OuLmUCmMmLevkqiegw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(316002)(5660300002)(956004)(2616005)(107886003)(7696005)(26005)(6666004)(508600001)(38350700002)(38100700002)(52116002)(36756003)(86362001)(186003)(6486002)(4326008)(66556008)(66476007)(6916009)(66946007)(83380400001)(8676002)(2906002)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qxP+SHuYoUmw73SSR3WdhJLRoF7AWaqtvW0PWzNMDXdY+kKwO7jJODtK/oAiO1kOQ+rMpVBY4hHkWyKXcEONWiIpnSl0aIGrNGSxFyb9SUbVTfY7v5XWIUbaORLQpV3d+vrARFFXCmenrL5W15l7j4wtjiwMzsYhI2HEAHF+SLUYSHxtagIQWg8QYjVCH/abUNOTlLhw6kpPh4QucGsioE1aG33Jm/V3kDQoqbCXa7k/6j+MIVBJwRmytnb1OpmnoTdy9HiuXcQ/GVHvGY/K/S/GTHA6tAO2jiPSJw4bVuDoWJX2wX3LoqribFKDaYL1u5/l7REDPzkXC+HlvwmfnhgYkpGKqQeJGTwYtGPB4nsIjvEWRLkRh543wgRi7Rcs2zGHFWgZCRoqo3WrVzIIsj3A8wlXBjyrSgawA20sC4JCE0PrJE+PAomhKGQU2Np7H+u8um/d/pvswpkx6vjZem9lCo3rKekx9/J6pvyLBKL825lvprxZwecV6agHsqe6QWb9SoQ7+mOA8VZaCZDW6IRRqP/Etr3faXZgTrKzCInYgJwGTCP5ULLbRn7Y5wHikT+FZe9xyY3zR6uPehP8UdKW/nmtuv5k6ZZdaT1VL3+uljzp3pvwvmFhdDWuYLmUKUdzittzA5spG2mHaKeuoBLyvk6Vcx6VMMxEGmSqG/l4EDdNVJ4ZB2DSFWCUz0PV1gsTjeboDIp6qQ9/l0YTSh8E4Y5eHcc9RbbQnRNQ5GvcxY7297qNwwzezxusMIGE X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8287826d-0bd2-4507-a0b9-08d94aefdadf X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:28.0772 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YPxWVnYzdZDV2GmBPBjsKio9hGjPsXoMeSGVokmaURkelRPH/hLrdRVnQ9ogjoUGedXNB75Cn1C5ahjlwnW2FA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3853 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190112 X-Proofpoint-GUID: mXIv2UsOMw7AZ3fSLdm56u5Ikbz0CfK_ X-Proofpoint-ORIG-GUID: mXIv2UsOMw7AZ3fSLdm56u5Ikbz0CfK_ Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Determine the BARs used by the PCI device and register handlers to manage the access to the same. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/remote/vfio-user-obj.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++ hw/remote/trace-events | 2 + 2 files changed, 97 insertions(+) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index d158a7f..9853feb 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -212,6 +212,99 @@ static int dma_unregister(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) return 0; } +static ssize_t vfu_object_bar_rw(PCIDevice *pci_dev, hwaddr addr, size_t count, + char * const buf, const bool is_write, + uint8_t type) +{ + AddressSpace *as = NULL; + MemTxResult res; + + if (type == PCI_BASE_ADDRESS_SPACE_MEMORY) { + as = pci_device_iommu_address_space(pci_dev); + } else { + as = &address_space_io; + } + + trace_vfu_bar_rw_enter(is_write ? "Write" : "Read", (uint64_t)addr); + + res = address_space_rw(as, addr, MEMTXATTRS_UNSPECIFIED, (void *)buf, + (hwaddr)count, is_write); + if (res != MEMTX_OK) { + warn_report("vfu: failed to %s 0x%"PRIx64"", + is_write ? "write to" : "read from", + addr); + return -1; + } + + trace_vfu_bar_rw_exit(is_write ? "Write" : "Read", (uint64_t)addr); + + return count; +} + +/** + * VFU_OBJECT_BAR_HANDLER - macro for defining handlers for PCI BARs. + * + * To create handler for BAR number 2, VFU_OBJECT_BAR_HANDLER(2) would + * define vfu_object_bar2_handler + */ +#define VFU_OBJECT_BAR_HANDLER(BAR_NO) \ + static ssize_t vfu_object_bar##BAR_NO##_handler(vfu_ctx_t *vfu_ctx, \ + char * const buf, size_t count, \ + loff_t offset, const bool is_write) \ + { \ + VfuObject *o = vfu_get_private(vfu_ctx); \ + hwaddr addr = (hwaddr)(pci_get_long(o->pci_dev->config + \ + PCI_BASE_ADDRESS_0 + \ + (4 * BAR_NO)) + offset); \ + \ + return vfu_object_bar_rw(o->pci_dev, addr, count, buf, is_write, \ + o->pci_dev->io_regions[BAR_NO].type); \ + } \ + +VFU_OBJECT_BAR_HANDLER(0) +VFU_OBJECT_BAR_HANDLER(1) +VFU_OBJECT_BAR_HANDLER(2) +VFU_OBJECT_BAR_HANDLER(3) +VFU_OBJECT_BAR_HANDLER(4) +VFU_OBJECT_BAR_HANDLER(5) + +static vfu_region_access_cb_t *vfu_object_bar_handlers[PCI_NUM_REGIONS] = { + &vfu_object_bar0_handler, + &vfu_object_bar1_handler, + &vfu_object_bar2_handler, + &vfu_object_bar3_handler, + &vfu_object_bar4_handler, + &vfu_object_bar5_handler, +}; + +/** + * vfu_object_register_bars - Identify active BAR regions of pdev and setup + * callbacks to handle read/write accesses + */ +static void vfu_object_register_bars(vfu_ctx_t *vfu_ctx, PCIDevice *pdev) +{ + uint32_t orig_val, new_val; + int i, size; + + for (i = 0; i < PCI_NUM_REGIONS; i++) { + orig_val = pci_default_read_config(pdev, + PCI_BASE_ADDRESS_0 + (4 * i), 4); + new_val = 0xffffffff; + pci_default_write_config(pdev, + PCI_BASE_ADDRESS_0 + (4 * i), new_val, 4); + new_val = pci_default_read_config(pdev, + PCI_BASE_ADDRESS_0 + (4 * i), 4); + size = (~(new_val & 0xFFFFFFF0)) + 1; + pci_default_write_config(pdev, PCI_BASE_ADDRESS_0 + (4 * i), + orig_val, 4); + if (size) { + vfu_setup_region(vfu_ctx, VFU_PCI_DEV_BAR0_REGION_IDX + i, size, + vfu_object_bar_handlers[i], VFU_REGION_FLAG_RW, + NULL, 0, -1, 0); + } + } +} + static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); @@ -266,6 +359,8 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) return; } + vfu_object_register_bars(o->vfu_ctx, o->pci_dev); + qemu_thread_create(&o->vfu_ctx_thread, "VFU ctx runner", vfu_object_ctx_run, o, QEMU_THREAD_JOINABLE); } diff --git a/hw/remote/trace-events b/hw/remote/trace-events index f945c7e..f3f65e2 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -9,3 +9,5 @@ vfu_cfg_read(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u -> 0x%x" vfu_cfg_write(uint32_t offset, uint32_t val) "vfu: cfg: 0x%u <- 0x%x" vfu_dma_register(uint64_t gpa, size_t len) "vfu: registering GPA 0x%"PRIx64", %zu bytes" vfu_dma_unregister(uint64_t gpa) "vfu: unregistering GPA 0x%"PRIx64"" +vfu_bar_rw_enter(const char *op, uint64_t addr) "vfu: %s request for BAR address 0x%"PRIx64"" +vfu_bar_rw_exit(const char *op, uint64_t addr) "vfu: Finished %s of BAR address 0x%"PRIx64"" From patchwork Mon Jul 19 20:00:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC4D2C07E95 for ; Mon, 19 Jul 2021 20:08:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 68AC0610FB for ; Mon, 19 Jul 2021 20:08:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 68AC0610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48316 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZZH-0002Rz-In for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:08:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47688) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRO-0005rg-UR for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:40 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:25602) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRL-0005RT-Qj for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:38 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuc52002487; Mon, 19 Jul 2021 20:00:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=qQj4Bq9NpnX0KNnVOcQuojQUPtcfwYcVGA2xcMzVhgA=; b=NBK0YNnKhZltA4zdDVLLwRk7ROdFOUMXWHvU3at6EqI/bbvN5wU3qcXdrBv4JyV3jKn0 4uoRdXP4SLXshFUXfhZn+ZHwDA3U/HQn8xMrBvfpeG1tqx8Cs+Raai+x78e7BxAAqsjV tOyChtlhJ5akhGvQQCibRzAzZ+kuEL1dwMxVXjDXetyMbiO8dh2TrUPbeDXYRsVd/jHw RimuKSNUF5f53tk59672IIOmh6hpn0q4AVycfo9AxoYhQ5W5cZC1O3PKT6nlMbDfU7lJ zh5OPas6QFxw0dHLyfFZobXVV5N8liGik26erNnTK78c0yYi0ZdOMJ2uZYVTF8oRCAlu Ew== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=qQj4Bq9NpnX0KNnVOcQuojQUPtcfwYcVGA2xcMzVhgA=; b=pQfS/ywts5YlO5KstAREDAAh0qsPEUzpKl6UWtJbMMOkgVkkC38HODeUxDKHzQb0JwCi ls5RgfgMoYLFm9Izs3EGncwiiKhKFYfyNtjFVeXZatQFKRpP5RiUcOk9Pkl00lgmwYEq YDSLw/a/hkknNVBvyLWTIjr2/jEox1fkb8L7nUCeeEGDEotJ/9afE7JZPj2rR822icQ9 ccz7CubBbLP3PbsX8c8J8BOmtuVdwbsSmrkPwmWExwJzqYENunPKf5KD0Wvyp013+7IX P1/9iy4vr76J0gTNqWopSWk25heCGBWiFo75050YmIWzUTojwBwh+CwY4wOouSoCXu9k ZQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 39vrn5j9xs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:32 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JJf5It033120; Mon, 19 Jul 2021 20:00:31 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 39uq15b6up-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:31 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NF6ZCIwPWjeTQIGmys0zHridWIH5nRmfjXVdbpYUwQBX49gKfqRXkWZuBt5h61GJgBzuD1m0ZyUTgN/72Hn0Xw9bS6QkQ/xe97+A2z5GCdm4af6WFouQYQan4j79QgH4shWjrYnHaCllJoLTsi7zsu8uJYaNGM1bpXjzFGAHPY/SKRQFjXGcKPYN0saPrlFCeomvO8qKITBXYcVh6y615tYBK3w3BghSHW6OujY7RM76eHiUC/NOW7GYOsheNAIKbyxcxqio/yR+cRRlLAZNNXIasbVqATomAkn/1foGD1XSLzSi2A8qHcCK4QJiT0VE9jQM7ulMkpHuKuU6duw+cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qQj4Bq9NpnX0KNnVOcQuojQUPtcfwYcVGA2xcMzVhgA=; b=Dblc1/R3vB3HD2S6iYItkNenP+nOC2f7fyN7UZ0/lT2w8v7kQaiqGP435nF3wng1VLa+bYiE0ov/8tENKo4USXlyNddFMvGSs2KpDXAHWlJOfm1z9juXnpSCBaWzasPBr/vdhYnzB5sDSJumd5S1PYknHCMqEcTnyFS3RQi9NRjTn0CXyXPveVD8QJiXCml+7bB0umUjSq/L6j/o7eujqgCQuup6+JqSfSx8ug0jGXI5az8Qmf+whkmydJ3KbP0ceu084KgHl1Ig3ddvhZcvmhRL3gpvOSKIV0d0EvSlgcNzt+/KqCM5i/NbJekgE9efaCowYkFM37+AvkkNAiup7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qQj4Bq9NpnX0KNnVOcQuojQUPtcfwYcVGA2xcMzVhgA=; b=SznNEG/r+7pU1laZJkzJrKZg07tn1vmKte1PaDQyMMXP4eycfGwOFlvywPTtSC4izRsxboPbpuzbGtrfL4gO8Xt15JbbM8Q53RumZiwpcUCPR8NMZlZKJMb+OpEnwbMwptzP6ijLOD75tcbRH+o3qNLHeXkeQz34Li0gXZgTG6w= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3853.namprd10.prod.outlook.com (2603:10b6:208:180::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Mon, 19 Jul 2021 20:00:29 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:29 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 09/11] vfio-user: handle device interrupts Date: Mon, 19 Jul 2021 16:00:11 -0400 Message-Id: <8da299d8adc92a9f6bba76931db90c286cfe920d.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:28 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7cd4c57d-0f14-47d7-447a-08d94aefdba5 X-MS-TrafficTypeDiagnostic: MN2PR10MB3853: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:462; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zRdZjxSmMW6FS5+N/vNVkd+nj3RRcz3SyRy/XdzjvAVLDprGWJQugeWXIRi8zZdE5N7z0MpdeXdIBS+ubIEwTt9vFD2kxS8NY46PIjgihJqtwucNS7tE/fUNegaG/loym2kcmbTjEz9J6e6n8RFMezON6rPtedssgPYiHoxnlsN+sGw/NoG6aypMkRYHxiPmdatRFC9C3TLAf0jnvywcu94aWs557k0sLHcTo3UCtKe4hxuzGo6wYPzKGHLIoDSsfN2Aj7I8wcrUZPk90CpBpv0KSCoQzC4UOeBmVqFy8uq0/VJC3JsiE6kBniWiigH+F6Vd1gerxxbDc/CVGAWDANxYgB0mKB0HE3aQYfdOOdmYBe6Q6tppo12XSUU4qYD2tNPWwmx38j3/FNOFXxnXMmBcRrvIuKkeE13XM3Iah/cWWDT4auXRYzqAeXNzpSzilzXHjO8aHdyovolRqt4bJMFeYzS4UYquiX5u85zspHK960vS3nJGwnl/dyPlR9mKRSsiDo/3jm/skYX8B/1Z2a3oq8OBQEHPeN3Z2gERfw6n7cWwcSbkgwre9i3j5LShq88RQefOc4ChO0pyxe+nKTD5sIEgnMhxy+3GA3Uklchh54/ts2wLOIOMgU+2bF4QmblCt/M0Ql60d/tCwTMkWWefvb2ibwWzTyta5pXroi8bwcbL5BwyHS3+xcHqJCyiPVRtGpcNFj+s5BzbuCHESA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(316002)(5660300002)(956004)(2616005)(107886003)(7696005)(26005)(6666004)(508600001)(38350700002)(38100700002)(52116002)(36756003)(86362001)(186003)(6486002)(4326008)(66556008)(66476007)(6916009)(66946007)(83380400001)(8676002)(2906002)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zNeVwvj9S/2RoVHTx2O8uoOOc2aG9SjWEBG1MExvTNFN9mQ9SDCSbQRsl2JKpyqtP/8DqTv/FeclMuwOuvHHTiDu7R6GrGMzX8jmKXTuXRCayyaYN2Pi+rpwR2yMtEDzxVwjK7BJlq3NMBcJD/IunF51vsVKw1amQGL9O6re/vpGQSAB9d28TuzXC2XwZMClm46KH5o4JttYg4VV/T/tGWTj32Zv1JFikbKVXi86uTg1Ld+5K4K8SHr4EgGCh+SNHUgiNTyFT/s3bGJWsnsgoSdkpdA73Uc+fGFW/NTwqBxdxiO5dVQfYMbkDyN+yPmHmNSns4q/95Ut9jxzFWzfIs1ecCwuLSoiiNr9h3AkNebmPRqXgzzMayWderg9w+vIn03phFiKovKxZ0arKHAd0nYxtgeBgwEu5Z8Hq+xnZx2cxaJs79kOp8t7TlK7DjEtw+InwwB1/lRCdGNZ2DqpujRUmDy1rxvAIKaP+r4al5+uB5n1QUpz2uNTyqPJmM0A7OobvYGVIGqs/tb6bdZM0q+gVp57zKTYySKOwQswhMSA7y95OR/rRK7CuEQBeHj7myCB5E2cJwwr65Y3I2bsNTeYx+we7kQm5BZRkIUU76Ep1fU77p8ml4GpZjPWkPmjjnsHFwRqxss/X+cBTTK92n7zsTUNbR+kHSenlEOPQptsGAp8AZHcYJpEorjFWvg+gZ6/Xff0fV9nmmleW6SGAUF5+pfhf5JIKMfpToqTjgalkqt+v9G7ZNLEiwbO7Akg X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7cd4c57d-0f14-47d7-447a-08d94aefdba5 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:29.4255 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Dq3C1bdHPU1pX+15DdagX/exErS3vLIkUX/k/1xNtj2unYtGGtgCTO+iIohzvB/Xsd+ybjqtFD9hmBwus9KhDw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3853 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190112 X-Proofpoint-GUID: YI2mpn6o-4iLRjh2vxIWhGIipXS0GZY8 X-Proofpoint-ORIG-GUID: YI2mpn6o-4iLRjh2vxIWhGIipXS0GZY8 Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -8 X-Spam_score: -0.9 X-Spam_bar: / X-Spam_report: (-0.9 / 5.0 requ) DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Forward remote device's interrupts to the guest Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/hw/remote/iohub.h | 2 ++ hw/remote/iohub.c | 6 ++++++ hw/remote/vfio-user-obj.c | 30 ++++++++++++++++++++++++++++++ hw/remote/trace-events | 1 + 4 files changed, 39 insertions(+) diff --git a/include/hw/remote/iohub.h b/include/hw/remote/iohub.h index 0bf98e0..132f496 100644 --- a/include/hw/remote/iohub.h +++ b/include/hw/remote/iohub.h @@ -15,6 +15,7 @@ #include "qemu/event_notifier.h" #include "qemu/thread-posix.h" #include "hw/remote/mpqemu-link.h" +#include "libvfio-user/include/libvfio-user.h" #define REMOTE_IOHUB_NB_PIRQS PCI_DEVFN_MAX @@ -30,6 +31,7 @@ typedef struct RemoteIOHubState { unsigned int irq_level[REMOTE_IOHUB_NB_PIRQS]; ResampleToken token[REMOTE_IOHUB_NB_PIRQS]; QemuMutex irq_level_lock[REMOTE_IOHUB_NB_PIRQS]; + vfu_ctx_t *vfu_ctx[REMOTE_IOHUB_NB_PIRQS]; } RemoteIOHubState; int remote_iohub_map_irq(PCIDevice *pci_dev, int intx); diff --git a/hw/remote/iohub.c b/hw/remote/iohub.c index 547d597..241c8d7 100644 --- a/hw/remote/iohub.c +++ b/hw/remote/iohub.c @@ -18,6 +18,8 @@ #include "hw/remote/machine.h" #include "hw/remote/iohub.h" #include "qemu/main-loop.h" +#include "libvfio-user/include/libvfio-user.h" +#include "trace.h" void remote_iohub_init(RemoteIOHubState *iohub) { @@ -62,6 +64,10 @@ void remote_iohub_set_irq(void *opaque, int pirq, int level) QEMU_LOCK_GUARD(&iohub->irq_level_lock[pirq]); if (level) { + if (iohub->vfu_ctx[pirq]) { + trace_vfu_interrupt(pirq); + vfu_irq_trigger(iohub->vfu_ctx[pirq], 0); + } if (++iohub->irq_level[pirq] == 1) { event_notifier_set(&iohub->irqfds[pirq]); } diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 9853feb..d2a2e51 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -41,6 +41,9 @@ #include "sysemu/sysemu.h" #include "hw/qdev-core.h" #include "hw/pci/pci.h" +#include "hw/boards.h" +#include "hw/remote/iohub.h" +#include "hw/remote/machine.h" #include "libvfio-user/include/libvfio-user.h" @@ -305,6 +308,26 @@ static void vfu_object_register_bars(vfu_ctx_t *vfu_ctx, PCIDevice *pdev) } } +static int vfu_object_setup_irqs(vfu_ctx_t *vfu_ctx, PCIDevice *pci_dev) +{ + RemoteMachineState *machine = REMOTE_MACHINE(current_machine); + RemoteIOHubState *iohub = &machine->iohub; + int pirq, intx, ret; + + ret = vfu_setup_device_nr_irqs(vfu_ctx, VFU_DEV_INTX_IRQ, 1); + if (ret < 0) { + return ret; + } + + intx = pci_get_byte(pci_dev->config + PCI_INTERRUPT_PIN) - 1; + + pirq = remote_iohub_map_irq(pci_dev, intx); + + iohub->vfu_ctx[pirq] = vfu_ctx; + + return 0; +} + static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); @@ -361,6 +384,13 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) vfu_object_register_bars(o->vfu_ctx, o->pci_dev); + ret = vfu_object_setup_irqs(o->vfu_ctx, o->pci_dev); + if (ret < 0) { + error_setg(&error_abort, "vfu: Failed to setup interrupts for %s", + o->devid); + return; + } + qemu_thread_create(&o->vfu_ctx_thread, "VFU ctx runner", vfu_object_ctx_run, o, QEMU_THREAD_JOINABLE); } diff --git a/hw/remote/trace-events b/hw/remote/trace-events index f3f65e2..b419d6f 100644 --- a/hw/remote/trace-events +++ b/hw/remote/trace-events @@ -11,3 +11,4 @@ vfu_dma_register(uint64_t gpa, size_t len) "vfu: registering GPA 0x%"PRIx64", %z vfu_dma_unregister(uint64_t gpa) "vfu: unregistering GPA 0x%"PRIx64"" vfu_bar_rw_enter(const char *op, uint64_t addr) "vfu: %s request for BAR address 0x%"PRIx64"" vfu_bar_rw_exit(const char *op, uint64_t addr) "vfu: Finished %s of BAR address 0x%"PRIx64"" +vfu_interrupt(int pirq) "vfu: sending interrupt to device - PIRQ %d" From patchwork Mon Jul 19 20:00:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFD8BC07E95 for ; Mon, 19 Jul 2021 20:03:19 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6F3B561029 for ; Mon, 19 Jul 2021 20:03:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F3B561029 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34776 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5ZTy-0001ds-BG for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:03:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47698) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRP-0005ri-KL for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:40 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:33152) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRM-0005TM-UU for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:39 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJubNU002464; Mon, 19 Jul 2021 20:00:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=ZhMA06ReX6eFP2Y6fGK8Tjmg4wulVd+ZwemZA5aU0T0=; b=Zxyaqn6eYCcbYzk5CiLKami5P4RtC0MfjfY44ctyIKZpCyJn8HjSdLbJA5qMOCfh4khW C63NQyrMHccOQmWuCOrtNIQbph0uwA/yxkwcZJ7cb8LTT+qGibbUG/9HqBeXiyjv42ug oXAVWk0bAImX7L5x89HV4NJmwnwH45XD3S3Y5YQyGeXx4bmK53ZsOkFRNNFFcXVVIFP3 O4OzA8FzP64VlUwp9l0+2SnpG2tfRPGClgw/Z9hxt1hJem1Jvpm5/JgNLOwXi5szvNSJ T+hjvtWzqgZMfXR+4kvxGAPlm8eilG93jamycUCwbjzL7ljknc+HbJIso07mJvSdzYer 8A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=ZhMA06ReX6eFP2Y6fGK8Tjmg4wulVd+ZwemZA5aU0T0=; b=oOkb2ldkphtJ7vnCiQvKPm+ItPMZiDgqgEwrQc8nVTbzI5tz5Y0xsPgKl2JESyxNApSI IR4u6ojMcCdMt0tateM+/iawccKqu0jese0s1RT7m75Pi/IhZCP44ZEdj/5abFNx+JGP dTKATrJgyiBUnjhhburJYE9br6uxvmOxmjDiBjedvVqMp07sygkOf8wLVYSoeBEPYYwx tv5BH2mT1WHMXo2WDhE0V2n7ojwFINv0STwBUYU3fkSsmn4u9ena9rBw3m4z3LLJ8OXQ mPOcDcpfu8ujwmP6+oam9Kekvw0Sglbf0gQFUhQHNRdgex9NQBK7rpM3JX1XUk+Qfep4 Aw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 39vrn5j9xy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:35 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JJf5Iw033120; Mon, 19 Jul 2021 20:00:34 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 39uq15b6up-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Fu9a254D5fZJjCP+gCqX6OhisNM6YV2vCZXT0ELxmU/uBOd/IYhC+k+yLiolJNQgvM86nDFPU+xUccMBg/KmzmBinmU5cnRao/cpjm3rw9iYTLMx67iNu33z30S+bbK39RRVPDXXPdhmZwEtLXLvGjH1SPuIzGOQ4q6BnVt68Z7qxgt0X0fIehYasHOqzb0xYmLQOE2jQU2gI3Jvm/Q9UeliB22nJXFWY7NO9o7qXLkEKVBXKG1nDsxtF3plCVqR8glUApktj1Dahl14m7SbFYcKUenxSGUjeIIaKuY4z05q2ycKAmHsCAPaVQ4fJBYWiC1uCbovDt0xmza3Rd7FNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZhMA06ReX6eFP2Y6fGK8Tjmg4wulVd+ZwemZA5aU0T0=; b=eqjeh/O1MZ0ac1L9Iw9Z8c2LtbWXaC4SsGPi3SP88uM4q33YrVa92jIr3nbJfTu7Qwu+eL1Gabk54N2Bv55zSwpm3S3U9xDGjxh1dtVUmvrn3NfkYhIBVjKS859AgoHR/MVRIJmKTf2V7QW/G4mGSKN6tUu2bAco3vZOfCYaDGYL42BZxRG78A0ywqxmqTQKEkAC7BEPMgjuULb0aZuRB589p5lsoQFv/vMiLxRlDB6Px9Wp8bm1dPjaKpdb5vkkDLvuuFGJ+khC5ijkf99bY+t41CrlKseWO/ii1cwOPmHA1GY31cb5IaLRzXgoVFsTmAeN+t6HkGYEERELhXZvIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZhMA06ReX6eFP2Y6fGK8Tjmg4wulVd+ZwemZA5aU0T0=; b=V3wMqgoGVwbADbC2v59g1wrwZS2Iwiaw3GAHIrjz6TUFqQCLBY76W8FEd+APkz7CW2RLkapxNiHi3xdlEmz2XPahMjlqMFbrKLarlLsvq9xtp/gh5qTgXSZpL+uvX3lD6ctoziZ83KEFz8vbOJ2AvKU80btweklW07tNkZe9HUI= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3853.namprd10.prod.outlook.com (2603:10b6:208:180::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Mon, 19 Jul 2021 20:00:30 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:30 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 10/11] vfio-user: register handlers to facilitate migration Date: Mon, 19 Jul 2021 16:00:12 -0400 Message-Id: <2b9b94e7b5e1bcf77c0b900bc0a5488db4910833.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:29 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 517014c9-e38c-411d-582e-08d94aefdc76 X-MS-TrafficTypeDiagnostic: MN2PR10MB3853: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2150; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BshJVpo3rTkkMWQ+0Y4qpf4bz+rFbl7yb44t1G7U3ov+6VTywnrPR97j/R0zq8wCqs+V7i89xPYMbPtfEahvWq3IxKjbQzpMr5c2DI7e74rX8vxRckdarEq7wFasQk3DYjqAjYYLSdBFOBoYnH+fdLr6jTczqiCjMqpFVDNbX815zcWLGFEkfdGWhL/7OnG1YFwAlMb1/bhb7bPJkXOeKTp9Zdvso0IodPtu63XsEtvZn/hVeX6cBe/GiOiy6rL6irAdY2g3oQ7ivn2bf88rqPm/e1zGtsBwcG48jvZTX6DjHhyNT0SjM4JPezseDpU+ATPvyJOnwg3a5BwamoV7saMhbJSVimIhxqIihu5kLkbsQqJxcSnvHQffff/3lbQWv7ZXWzUb8l75mXUWIyvSFh156ioY1W7GXFrE4fCFnuAA56ucfOANa0/o6aDphxVwz2xcadKAYtyAg/WWCmKw4AXTLhHnp5/NDy2m/4HPAWxytWXgr7kjUnoZ9tPO0C1f49ONu870MNo9JKXg9nOfqWBL3dqz3v2pd3QsXnXUwS88jf3T6gTWxDA329llHtwrYDZ57D2PA0jR9wTk+VADt1kd+/sxnsSn7nCceeRUUYT5mdUShVXNL9AX3HpESeA8ZXfcTaa5U9ljZmHOdnHkuOHWz4okcenuQ03lpBFFjd3RVKaHWCaK8ZdcRJ9OP8KxIFqlR4v/oNm7lPFMbDR+rw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(30864003)(316002)(5660300002)(956004)(2616005)(107886003)(7696005)(26005)(6666004)(508600001)(38350700002)(38100700002)(52116002)(36756003)(86362001)(186003)(6486002)(4326008)(66556008)(66476007)(6916009)(66946007)(83380400001)(8676002)(2906002)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: soS9eVxHFydHoOOrDhQ8glFtRJdEmOs71TPSyZMm0hNGEw4sLDEP0euYRGaBfR8UlB8+g3Xacvf9oI5lY6ZQTEh66DELdgGqXJZWXYZxJTdxa6LaICxd3owphEHvdaWnrj8ryrzLZ4b4yb8ekK/283YI3+fiPNCL0/s+PM3shNzUjIVRtQoV2VPO95Si9KLpxWci5SwqNCgBN4tubtrJX1hiEGA0F1Z5HslShatAPJ7HIehYhsOciQfnrPD9jG3QBbPQbBtsfdC6j6K8sZ3YPPq/ZOG9V45YY6O9IJPff2g4xZ1xCltwgCAdzyo0k0fH2f8CJX1BxQmPHw3lGFX1L3u2+8pDUS44xwh5iJpJdhg0PZG5tA1OXfnYgxPwfN6CphY29Tj1VeDBwkkvcJR4XZUKarS8npUo60kjVPOJHQhihDclGwHdZ1W1FyPD/NRZTt0l8ZRlHE+bViC2iP2MofcH/l7yNleGhDy9fIRZH1vLVqlshFELKAZ4ntmpqtimY2lvS+xkSRtGtS6jV/UpCkTgxtLOcs7brtsWL8OL2TZZCSDYQp9aPYhhgZPGzPgtvjmlayX5tA4layOp0N6JKjnCk0poHWl0liztrL1JiplokKm+oq5I1JfISMX7T1WLSUlaI4ucBAIgSlk8drHTwf/6Fn2BoglsQU7i0G7EZSh2Uf03F2UQgt+1mo0RuVMth8EgLIQOcydFXszU7q0m8NUSpkkOsE6Ify8i8az5nm2P+x00LVg/xMHYEA4aQatf X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 517014c9-e38c-411d-582e-08d94aefdc76 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:30.7647 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eJz5f2xn9EKpp6oWqB6ThDZ7xtsNNbyCo7bYs45npgIEfnk2BzNDO1cIT4pQ+SGVN8zOWcXDoSJY0/Q5t6PjBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3853 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190112 X-Proofpoint-GUID: hM4ZWnflXbxRw1Cb9HA_jXB46AM-Aww1 X-Proofpoint-ORIG-GUID: hM4ZWnflXbxRw1Cb9HA_jXB46AM-Aww1 Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Store and load the device's state using handlers for live migration Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- migration/savevm.h | 2 + hw/remote/vfio-user-obj.c | 287 ++++++++++++++++++++++++++++++++++++++++++++++ migration/savevm.c | 63 ++++++++++ 3 files changed, 352 insertions(+) diff --git a/migration/savevm.h b/migration/savevm.h index 6461342..71d1733 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -67,5 +67,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis); int qemu_load_device_state(QEMUFile *f); int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, bool in_postcopy, bool inactivate_disks); +int qemu_remote_savevm(QEMUFile *f); +int qemu_remote_loadvm(QEMUFile *f); #endif diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index d2a2e51..5948576 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -44,6 +44,10 @@ #include "hw/boards.h" #include "hw/remote/iohub.h" #include "hw/remote/machine.h" +#include "migration/qemu-file.h" +#include "migration/savevm.h" +#include "migration/global_state.h" +#include "block/block.h" #include "libvfio-user/include/libvfio-user.h" @@ -73,6 +77,31 @@ struct VfuObject { PCIDevice *pci_dev; QemuThread vfu_ctx_thread; + + /* + * vfu_mig_buf holds the migration data. In the remote process, this + * buffer replaces the role of an IO channel which links the source + * and the destination. + * + * Whenever the client QEMU process initiates migration, the libvfio-user + * library notifies that to this server. The remote/server QEMU sets up a + * QEMUFile object using this buffer as backend. The remote passes this + * object to its migration subsystem, and it slirps the VMSDs of all its + * devices and stores them in this buffer. + * + * libvfio-user library subsequetly asks the remote for any data that needs + * to be moved over to the destination using its vfu_migration_callbacks_t + * APIs. The remote hands over this buffer as data at this time. + * + * A reverse of this process happens at the destination. + */ + uint8_t *vfu_mig_buf; + + uint64_t vfu_mig_buf_size; + + uint64_t vfu_mig_buf_pending; + + QEMUFile *vfu_mig_file; }; static void vfu_object_set_socket(Object *obj, const char *str, Error **errp) @@ -97,6 +126,226 @@ static void vfu_object_set_devid(Object *obj, const char *str, Error **errp) trace_vfu_prop("devid", str); } +/** + * Migration helper functions + * + * vfu_mig_buf_read & vfu_mig_buf_write are used by QEMU's migration + * subsystem - qemu_remote_savevm & qemu_remote_loadvm. savevm/loadvm + * call these functions via QEMUFileOps to save/load the VMSD of all + * the devices into vfu_mig_buf + * + */ +static ssize_t vfu_mig_buf_read(void *opaque, uint8_t *buf, int64_t pos, + size_t size, Error **errp) +{ + VfuObject *o = opaque; + + if (pos > o->vfu_mig_buf_size) { + size = 0; + } else if ((pos + size) > o->vfu_mig_buf_size) { + size = o->vfu_mig_buf_size; + } + + memcpy(buf, (o->vfu_mig_buf + pos), size); + + o->vfu_mig_buf_size -= size; + + return size; +} + +static ssize_t vfu_mig_buf_write(void *opaque, struct iovec *iov, int iovcnt, + int64_t pos, Error **errp) +{ + VfuObject *o = opaque; + uint64_t end = pos + iov_size(iov, iovcnt); + int i; + + if (end > o->vfu_mig_buf_size) { + o->vfu_mig_buf = g_realloc(o->vfu_mig_buf, end); + } + + for (i = 0; i < iovcnt; i++) { + memcpy((o->vfu_mig_buf + o->vfu_mig_buf_size), iov[i].iov_base, + iov[i].iov_len); + o->vfu_mig_buf_size += iov[i].iov_len; + o->vfu_mig_buf_pending += iov[i].iov_len; + } + + return iov_size(iov, iovcnt); +} + +static int vfu_mig_buf_shutdown(void *opaque, bool rd, bool wr, Error **errp) +{ + VfuObject *o = opaque; + + o->vfu_mig_buf_size = 0; + + g_free(o->vfu_mig_buf); + + return 0; +} + +static const QEMUFileOps vfu_mig_fops_save = { + .writev_buffer = vfu_mig_buf_write, + .shut_down = vfu_mig_buf_shutdown, +}; + +static const QEMUFileOps vfu_mig_fops_load = { + .get_buffer = vfu_mig_buf_read, + .shut_down = vfu_mig_buf_shutdown, +}; + +/** + * handlers for vfu_migration_callbacks_t + * + * The libvfio-user library accesses these handlers to drive the migration + * at the remote end, and also to transport the data stored in vfu_mig_buf + * + */ +static void vfu_mig_state_precopy(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + int ret; + + if (!o->vfu_mig_file) { + o->vfu_mig_file = qemu_fopen_ops(o, &vfu_mig_fops_save); + } + + global_state_store(); + + ret = qemu_remote_savevm(o->vfu_mig_file); + if (ret) { + qemu_file_shutdown(o->vfu_mig_file); + return; + } + + qemu_fflush(o->vfu_mig_file); + + bdrv_inactivate_all(); +} + +static void vfu_mig_state_running(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + Error *local_err = NULL; + int ret; + + ret = qemu_remote_loadvm(o->vfu_mig_file); + if (ret) { + error_setg(&error_abort, "vfu: failed to restore device state"); + return; + } + + bdrv_invalidate_cache_all(&local_err); + if (local_err) { + error_report_err(local_err); + return; + } + + vm_start(); +} + +static int vfu_mig_transition(vfu_ctx_t *vfu_ctx, vfu_migr_state_t state) +{ + switch (state) { + case VFU_MIGR_STATE_RESUME: + case VFU_MIGR_STATE_STOP_AND_COPY: + case VFU_MIGR_STATE_STOP: + break; + case VFU_MIGR_STATE_PRE_COPY: + vfu_mig_state_precopy(vfu_ctx); + break; + case VFU_MIGR_STATE_RUNNING: + if (!runstate_is_running()) { + vfu_mig_state_running(vfu_ctx); + } + break; + default: + warn_report("vfu: Unknown migration state %d", state); + } + + return 0; +} + +static uint64_t vfu_mig_get_pending_bytes(vfu_ctx_t *vfu_ctx) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + return o->vfu_mig_buf_pending; +} + +static int vfu_mig_prepare_data(vfu_ctx_t *vfu_ctx, uint64_t *offset, + uint64_t *size) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + if (offset) { + *offset = 0; + } + + if (size) { + *size = o->vfu_mig_buf_size; + } + + return 0; +} + +static ssize_t vfu_mig_read_data(vfu_ctx_t *vfu_ctx, void *buf, + uint64_t size, uint64_t offset) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + + if (offset > o->vfu_mig_buf_size) { + return -1; + } + + if ((offset + size) > o->vfu_mig_buf_size) { + warn_report("vfu: buffer overflow - check pending_bytes"); + size = o->vfu_mig_buf_size - offset; + } + + memcpy(buf, (o->vfu_mig_buf + offset), size); + + o->vfu_mig_buf_pending -= size; + + return size; +} + +static ssize_t vfu_mig_write_data(vfu_ctx_t *vfu_ctx, void *data, + uint64_t size, uint64_t offset) +{ + VfuObject *o = vfu_get_private(vfu_ctx); + uint64_t end = offset + size; + + if (end > o->vfu_mig_buf_size) { + o->vfu_mig_buf = g_realloc(o->vfu_mig_buf, end); + o->vfu_mig_buf_size = end; + } + + memcpy((o->vfu_mig_buf + offset), data, size); + + if (!o->vfu_mig_file) { + o->vfu_mig_file = qemu_fopen_ops(o, &vfu_mig_fops_load); + } + + return size; +} + +static int vfu_mig_data_written(vfu_ctx_t *vfu_ctx, uint64_t count) +{ + return 0; +} + +static const vfu_migration_callbacks_t vfu_mig_cbs = { + .version = VFU_MIGR_CALLBACKS_VERS, + .transition = &vfu_mig_transition, + .get_pending_bytes = &vfu_mig_get_pending_bytes, + .prepare_data = &vfu_mig_prepare_data, + .read_data = &vfu_mig_read_data, + .data_written = &vfu_mig_data_written, + .write_data = &vfu_mig_write_data, +}; + static void *vfu_object_ctx_run(void *opaque) { VfuObject *o = opaque; @@ -332,6 +581,7 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) { VfuObject *o = container_of(notifier, VfuObject, machine_done); DeviceState *dev = NULL; + size_t migr_area_size; int ret; o->vfu_ctx = vfu_create_ctx(VFU_TRANS_SOCK, o->socket, 0, @@ -391,6 +641,35 @@ static void vfu_object_machine_done(Notifier *notifier, void *data) return; } + /* + * TODO: The 0x20000 number used below is a temporary. We are working on + * a cleaner fix for this. + * + * The libvfio-user library assumes that the remote knows the size of + * the data to be migrated at boot time, but that is not the case with + * VMSDs, as it can contain a variable-size buffer. 0x20000 is used + * as a sufficiently large buffer to demonstrate migration, but that + * cannot be used as a solution. + * + */ + ret = vfu_setup_region(o->vfu_ctx, VFU_PCI_DEV_MIGR_REGION_IDX, + 0x20000, NULL, + VFU_REGION_FLAG_RW, NULL, 0, -1, 0); + if (ret < 0) { + error_setg(&error_abort, "vfu: Failed to register migration BAR %s- %s", + o->devid, strerror(errno)); + return; + } + + migr_area_size = vfu_get_migr_register_area_size(); + ret = vfu_setup_device_migration_callbacks(o->vfu_ctx, &vfu_mig_cbs, + migr_area_size); + if (ret < 0) { + error_setg(&error_abort, "vfu: Failed to setup migration %s- %s", + o->devid, strerror(errno)); + return; + } + qemu_thread_create(&o->vfu_ctx_thread, "VFU ctx runner", vfu_object_ctx_run, o, QEMU_THREAD_JOINABLE); } @@ -412,6 +691,14 @@ static void vfu_object_init(Object *obj) o->machine_done.notify = vfu_object_machine_done; qemu_add_machine_init_done_notifier(&o->machine_done); + + o->vfu_mig_file = NULL; + + o->vfu_mig_buf = NULL; + + o->vfu_mig_buf_size = 0; + + o->vfu_mig_buf_pending = 0; } static void vfu_object_finalize(Object *obj) diff --git a/migration/savevm.c b/migration/savevm.c index 72848b9..c2279af 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1603,6 +1603,33 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) return ret; } +int qemu_remote_savevm(QEMUFile *f) +{ + SaveStateEntry *se; + int ret; + + QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { + if (!se->vmsd || !vmstate_save_needed(se->vmsd, se->opaque)) { + continue; + } + + save_section_header(f, se, QEMU_VM_SECTION_FULL); + + ret = vmstate_save(f, se, NULL); + if (ret) { + qemu_file_set_error(f, ret); + return ret; + } + + save_section_footer(f, se); + } + + qemu_put_byte(f, QEMU_VM_EOF); + qemu_fflush(f); + + return 0; +} + void qemu_savevm_live_state(QEMUFile *f) { /* save QEMU_VM_SECTION_END section */ @@ -2443,6 +2470,42 @@ qemu_loadvm_section_start_full(QEMUFile *f, MigrationIncomingState *mis) return 0; } +int qemu_remote_loadvm(QEMUFile *f) +{ + uint8_t section_type; + int ret = 0; + + qemu_mutex_lock_iothread(); + + while (true) { + section_type = qemu_get_byte(f); + + if (qemu_file_get_error(f)) { + ret = qemu_file_get_error(f); + break; + } + + switch (section_type) { + case QEMU_VM_SECTION_FULL: + ret = qemu_loadvm_section_start_full(f, NULL); + if (ret < 0) { + break; + } + break; + case QEMU_VM_EOF: + goto out; + default: + ret = -EINVAL; + goto out; + } + } + +out: + qemu_mutex_unlock_iothread(); + + return ret; +} + static int qemu_loadvm_section_part_end(QEMUFile *f, MigrationIncomingState *mis) { From patchwork Mon Jul 19 20:00:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12386699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87D37C07E95 for ; Mon, 19 Jul 2021 20:11:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 308B06113B for ; Mon, 19 Jul 2021 20:11:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 308B06113B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55362 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5Zbl-0007OE-C6 for qemu-devel@archiver.kernel.org; Mon, 19 Jul 2021 16:11:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRQ-0005sW-EK for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:40 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:35868) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5ZRO-0005Tp-1i for qemu-devel@nongnu.org; Mon, 19 Jul 2021 16:00:40 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16JJuV6H015851; Mon, 19 Jul 2021 20:00:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=9CzF7tsHv/R5f3eDR1pzZDOtP8hvcfcUrVHAr1lwATo=; b=fjctjeHfTEh76HrOoN2rksU/fstFE1BLiImQhI6tC5k5a5fy1FW6Ko4HjQBuq5MgIHWj Eh2xL9lj4DPT4uUZLP+TWpFM7BUPdmMP2j8ZFeMFaA8G8HRdFC9d9aq1tGERC3VislPO /sbmdBM7R/ayLsn6Ecr4Wp5opajBsrKh+d95alsuxEIIIP/ip7cRcUJ0TCOvtk+txFF9 w08GNZKYrJ63LdhfZsJ3SWQg+bHjYoogRLLbEqZztuZo2uzozyeJ0Nsk+PA7CdNnYXwZ MYXiOnX8FNfLEB3PPEhL1jSliftgCiaWh596HEgMLKMGLS4IVjH7pjmYFqbOc1Q8vRjZ ow== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=9CzF7tsHv/R5f3eDR1pzZDOtP8hvcfcUrVHAr1lwATo=; b=wzOy26vpIWGejJKi3ce+gMm6G3V9Hdm2Qeyq7BEz1V15aPtADdPkQHulcO5uuN4UIR9I bJPDZ0uyqplq7ibvILQKaVgQuB5yiVY9s6+YwDXrz6X721ZeyHD4uIgYPtbol5erXlsU oK6wAGlBuLw5F/Qtn/Etz7D6wr5DaWMlgkhzomKMWGqnbXBde6RXX1okFxBD/NjsrITc MeXN+RpP6zloo75tcZ/vWFZl6cLxzwU5ZewR0AtPWCyxLk6x/aPTALxJ99cAmZ27L7eK fNvMVo3fOpQ91RQmcjqUzKz4MVE77zATQe0v+CdVSEzVP1S6PfxXJwbnsipk66qLBNDL Hw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 39vqm9a96w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:36 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16JJf5Ix033120; Mon, 19 Jul 2021 20:00:35 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2103.outbound.protection.outlook.com [104.47.55.103]) by aserp3020.oracle.com with ESMTP id 39uq15b6up-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 19 Jul 2021 20:00:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oKbaLJsUS01j+M4fKUEVTQPQgBAR3vYQaAGLtBxR18Wwg51pjCJdsrgR3/CZmx1SaKzdnIlDCJyNWRCezs5+7VgeMR7x5vW/Q8rbwX3TCJaQQe1pktd/KgKD7/M+2ko/sQYNjjqaPyaoCxDcROONxvtwtRiNNwe5A776YbXhgDy2azsdqiyI9bHsqqJwNkzUC8ICdIoTCCG7/hpOq7yxCw493uZHJictdKrCy1y/7WUcV/uqhy7CioXarqe91GNFvi3pGonzqxFYP6EpgNRK6tP+UKHHRYfWx+LeGFnWQEUsWSvbaO6RtsbMRi+7YkWypyF0PTNIzC4RJNMPB7wupQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9CzF7tsHv/R5f3eDR1pzZDOtP8hvcfcUrVHAr1lwATo=; b=gjpP4kz4fFnaLeIkwoN7uo9aDTkc5d4V2hQL0VMGkBi36HCCmVz3QwYVsnfNnH/BbnnlGS9mh60rqMXtCea45kfUldab8gT/AHWF1BxnJz9cNAp5u/qUoWJinuOsoVRoufk5l/gBMYvSC9VZdRFwxrFzsVYogJWH6g83j7wAGWAAFoI80jLiwss4KtgUWD9odlbHJv083kcEPBEB2tkWiIpRreEWJrgRtsxdpLAQfkGO1Wv6sMtu1TjaXCkSEjjCW03Hbva2FBFgbhZWyF8gWT69fJQFtuX80ac09NYsytLfd8WXVwdLpqnSy1UXDAhlDixhckuTx0M+DZFRcTLJJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9CzF7tsHv/R5f3eDR1pzZDOtP8hvcfcUrVHAr1lwATo=; b=vN1RTFdohgxETCOzGR/e3+fBA60qQziuNYQOO1bXRlMDoJc7byaPRWvYC7UfovnYbJo4SRglU6JA3pPuwmN0U2OrrEr3Y2LF+fLHCh8bGi36GlUdvGgiplsO2mxoB9JpNO6tjgw7vBQ0veSqnsVTNVswH+NcfLgLxG7ccJ/yqcA= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by MN2PR10MB3853.namprd10.prod.outlook.com (2603:10b6:208:180::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.26; Mon, 19 Jul 2021 20:00:32 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::e935:82a8:3c37:8aa4%4]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 20:00:32 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH RFC server 11/11] vfio-user: acceptance test Date: Mon, 19 Jul 2021 16:00:13 -0400 Message-Id: <0ed2315f59965380ffbbfb01556e18075fc654b2.1626722742.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-ClientProxiedBy: SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.44) by SA0PR11CA0040.namprd11.prod.outlook.com (2603:10b6:806:d0::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 20:00:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 11878326-cc9e-4a08-824e-08d94aefdd3d X-MS-TrafficTypeDiagnostic: MN2PR10MB3853: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: v6cOMCC3y12A9kIK+UmLlE5K8Ctgz0bhqs+XSdlFhiH542h6jSZY3fUDsFBAMe/jSwGexisn2ER+1zvRvNzo3klwVsvjbNDy5k93kTMeaJ8O7V6c2kXIKMsin0UuVOV+GaHBGPpov70sQOtnDKWUjjIM31xY2Td8nhF83PWJN1WHospn75kbMMS/e5T7jqtaro0rzqzcpxTATXgoKKLd+9N7C6YKrkJn51UjZ0KsDmFxKiLwXkke+/QSKpTzZN42/ojl/rAHLSiuqNhOPpojZgNM7qinBT9n54izl8ytZoeIpvvrjmDjOvwojv7SM5A/AG59jiaMS9pp/G9IKyKkL4ede7/903Gtd8QVf97qS8DHxnzzF7ScVNgqsQF85hmuDe5UTI1sYTUZX7C9I/yFD6UOnDxrQFkprxPjmnDloyopg9C1GoumAqa3yv0OIx0m8JYdR0LgvgsZJbAqvxvKqVskdrgJ6oj++ojcSuEKuGMw8L5rOUwjpUzet/OzyHlnipxMB9QHDdta7IzQRSPqFdO7BP2mTZae0oh1uNYjXe247fNtGpIhxLV/je5jRNvqDW1JD78pgKwAfiPTNqkZCa2j4lJDJztxfbHdhrf0R0KXc+ca8j2SQmRbB5B5NVrKXa9HjhuqyxwmGab8JThrhADED8CqvrAHkiAdtshaFK4cWxEU8034SNW0iS9+46QMFd/XAbUWp4aAkKtcBDUcu1W/kZGDrt7TS+GGAoFoejcap8SEc+iO08y0W58PfJ6gESb2L9myCKshAoDxKieebQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(316002)(5660300002)(956004)(2616005)(107886003)(7696005)(26005)(6666004)(508600001)(38350700002)(38100700002)(52116002)(36756003)(86362001)(186003)(6486002)(4326008)(66556008)(66476007)(6916009)(66946007)(83380400001)(8676002)(2906002)(8936002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FII1FzQfrQTx/6z7gdBLS9DsXScqWf/XHALrnnJeaOqG1Sc0Bj+HR0iDjsqGl5/jv57zCqjAkYr7jTvG7LvyM/lW+GxKxDqPzqags0jNfQ+W+hn6FuibajIhVosxzKiPpDNpJN6oTuXDnRr8Rb7Xylc9mSTUQf1Uoz+QWXDUa33YkF/Ej1mIFP2XS1wwEn4ZKkHuHeGMw3sQU1iRdlQefC9DIqre6toWAYKO/Ixrl3EYReWJdQrBLl2uTurDMvM3I63nj55uLfEihayle1/uKp7d1wtBlgVhQoMi7pA0dc4pqGujeK1EZRTvuyxmMniuGUE8Z+smdC2HFNOglxXoUbY8XRjWL+fJLT25uChMmXtRz9jlnqgjQenqHAdEjrE26457RLQ82wWBXsibZEZy89d35wtrNHhboKwXBK68LAdTaRPCclMmVV64ksuVHlqIaqtJ+zD3+wfg6Xou1u8DxL7lri3Rl5anjVq1MiGTyiaB35z+6Jg3UhxxwCuy4NQSWuiQsLFCUrBQosbgjLqTxmeRFCrm3a7PF9qIG0PNbpwiqEf+6InKo9DGaHri9lsSfI6yHHGiTghsxOvC0/KIzIZi6MB86xW91AG9vHoG3dPRRffruD4PHC7Zfi4/D02FrG6NpPJLwdEeGYBbCHEij/iTV2fYqls1uHA/MtCBH3d3p9r9QDK2DXYDoz5PpGlBIFNjKWlCn9CNWXYazKUuQTvc/gAZrnFBlUrbFUxx01SFtt0lM1ZGlOuGCQC81Yox X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11878326-cc9e-4a08-824e-08d94aefdd3d X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 20:00:32.0619 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Kt+TA6nCi6eHKZ6vmjXlBYjftiuUPoP4fbguT+wxpwwGPKPrUMT+a5RLoc/hmztbh243pr8/oaOr7AFlToC2ng== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3853 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10050 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107190112 X-Proofpoint-GUID: 2Ru-f8qgt1LIBpucFDuifJBkSZdeUcfI X-Proofpoint-ORIG-GUID: 2Ru-f8qgt1LIBpucFDuifJBkSZdeUcfI Received-SPF: pass client-ip=205.220.165.32; envelope-from=jag.raman@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, jag.raman@oracle.com, swapnil.ingle@nutanix.com, john.levon@nutanix.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Acceptance test for libvfio-user in QEMU Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- MAINTAINERS | 1 + tests/acceptance/vfio-user.py | 94 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+) create mode 100644 tests/acceptance/vfio-user.py diff --git a/MAINTAINERS b/MAINTAINERS index 46ab6b6..644bd35 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3381,6 +3381,7 @@ F: include/hw/remote/proxy-memory-listener.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h F: hw/remote/vfio-user-obj.c +F: tests/acceptance/vfio-user.py EBPF: M: Jason Wang diff --git a/tests/acceptance/vfio-user.py b/tests/acceptance/vfio-user.py new file mode 100644 index 0000000..ef318d9 --- /dev/null +++ b/tests/acceptance/vfio-user.py @@ -0,0 +1,94 @@ +# vfio-user protocol sanity test +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + + +import os +import socket +import uuid + +from avocado_qemu import Test +from avocado_qemu import wait_for_console_pattern +from avocado_qemu import exec_command +from avocado_qemu import exec_command_and_wait_for_pattern + +class VfioUser(Test): + """ + :avocado: tags=vfiouser + """ + KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 ' + + def do_test(self, kernel_url, initrd_url, kernel_command_line, + machine_type): + """Main test method""" + self.require_accelerator('kvm') + + kernel_path = self.fetch_asset(kernel_url) + initrd_path = self.fetch_asset(initrd_url) + + socket = os.path.join('/tmp', str(uuid.uuid4())) + if os.path.exists(socket): + os.remove(socket) + + # Create remote process + remote_vm = self.get_vm() + remote_vm.add_args('-machine', 'x-remote') + remote_vm.add_args('-nodefaults') + remote_vm.add_args('-device', 'lsi53c895a,id=lsi1') + remote_vm.add_args('-object', 'vfio-user,id=vfioobj1,' + 'devid=lsi1,socket='+socket) + remote_vm.launch() + + # Create proxy process + self.vm.set_console() + self.vm.add_args('-machine', machine_type) + self.vm.add_args('-accel', 'kvm') + self.vm.add_args('-cpu', 'host') + self.vm.add_args('-object', + 'memory-backend-memfd,id=sysmem-file,size=2G') + self.vm.add_args('--numa', 'node,memdev=sysmem-file') + self.vm.add_args('-m', '2048') + self.vm.add_args('-kernel', kernel_path, + '-initrd', initrd_path, + '-append', kernel_command_line) + self.vm.add_args('-device', + 'vfio-user-pci,' + 'socket='+socket) + self.vm.launch() + wait_for_console_pattern(self, 'as init process', + 'Kernel panic - not syncing') + exec_command(self, 'mount -t sysfs sysfs /sys') + exec_command_and_wait_for_pattern(self, + 'cat /sys/bus/pci/devices/*/uevent', + 'PCI_ID=1000:0012') + + def test_multiprocess_x86_64(self): + """ + :avocado: tags=arch:x86_64 + """ + kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/vmlinuz') + initrd_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/x86_64/os/images' + '/pxeboot/initrd.img') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 rdinit=/bin/bash') + machine_type = 'pc' + self.do_test(kernel_url, initrd_url, kernel_command_line, machine_type) + + def test_multiprocess_aarch64(self): + """ + :avocado: tags=arch:aarch64 + """ + kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/aarch64/os/images' + '/pxeboot/vmlinuz') + initrd_url = ('https://archives.fedoraproject.org/pub/archive/fedora' + '/linux/releases/31/Everything/aarch64/os/images' + '/pxeboot/initrd.img') + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'rdinit=/bin/bash console=ttyAMA0') + machine_type = 'virt,gic-version=3' + self.do_test(kernel_url, initrd_url, kernel_command_line, machine_type)