From patchwork Fri Jan 29 16:46:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12056075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17FB9C433DB for ; Fri, 29 Jan 2021 17:00:53 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 64D5864DEB for ; Fri, 29 Jan 2021 17:00:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 64D5864DEB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53970 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5X8d-0008KQ-B9 for qemu-devel@archiver.kernel.org; Fri, 29 Jan 2021 12:00:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42006) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5Wvl-0007HM-Ej for qemu-devel@nongnu.org; Fri, 29 Jan 2021 11:47:34 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:60042) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5Wvb-0005nI-Fp for qemu-devel@nongnu.org; Fri, 29 Jan 2021 11:47:33 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10TGOiHi121677; Fri, 29 Jan 2021 16:47:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=bSc7FL5nFi0+xGcsWTkkqW8vF96TflCH6/DZRipj63Y=; b=qQEHMZqGm5n5VpwmKp23ZU9+s+Dm5DgbvSHzjaUxC7bd1Jv5zwPJvC8Rk6zr8edIIY4R Zl9Vjmh5bhbNrrpA/SVjFniqcdl2odFfBOjrZbDMp4j+yA5Zo2XcMdbMLsLS1W0twqp0 0IEIygDrClEKDdNb1f4U4JV0kHzPbM2mWCvKuP6z+BiUgYY+cZG3NDMaFtAkmnUzlwCD HED30OUIcLK29+tUlWOWrXImkB8clXYgFPV+ccGnU2vWBeMi4XDAgMIXlr2JhhNvkhpw vi2P6+EpO70IASFf7lDvvGURePTYc38bkh5v9s6WFut+aUKHCPb/R0MVE7hjuiI5MbJc Rg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 36cmf88j56-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jan 2021 16:47:15 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10TGPtEo089699; Fri, 29 Jan 2021 16:47:14 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2102.outbound.protection.outlook.com [104.47.58.102]) by userp3020.oracle.com with ESMTP id 36ceugw3rs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jan 2021 16:47:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YJ1M71lKJGgbfzHXOoe3RweGmN0MzQ3aGEgHm5gLKKh4KMQCsQ4IX3k9olzXigmVstx8xJQ1f47vfhNIPAC8ZiBh0/ZmKhCCS6rds7jdz2tISYAAjXgKzFcGPQRpLUgdmJBOisICRFR2nDJsq2yufErWJYvNhYv+3OW2cH2gqwS+5SbSMH8ty/tCmiLkLSsl/f6ODwRkGbr4iUkk68bjqpA3FsDDnDi9uUFoEUkgbF18Uqwb/Jwx/3i9U+Q3RRIG1dT1itIkwySoR0tnthD+83hBoGzCwnEowyHPTxrNB+B8ylbASk91Jt4wI4p/JNdYakZTl+GcFI+YPsEYUvURiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bSc7FL5nFi0+xGcsWTkkqW8vF96TflCH6/DZRipj63Y=; b=Muf3yk3mD5sqeoeMAkroqDD61PvzAMm1vd1/Dl+BDwYKKyugKIgkW6HBn/MarEuFaZOWYAmrc91r17bHgRTmElV4L6WlRO41g342jOEK3awzli3ZGmnxM15S7Pa6PiWvlFm95WO0SBQKdulQzeoC4JyRabkoO7aE5Iq9aGAPg3ufsLq+0GeehtBKitEnTqiOS4k6Chawd1jxu0YGmBFvfo/RHUWIUXJwq1qMW28s0VO61ubKltpoedLN4C6wqZ7wYAGHFY5ql3a//CzkR68jEO34rW+LgMBVDzKlmhLJoue2ic3KY+9BPShkcYS1WLHWuXq3+//Pg3uzTmBySIhl3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bSc7FL5nFi0+xGcsWTkkqW8vF96TflCH6/DZRipj63Y=; b=RaBSG+PguyXh3LQKT2VmHR1Sz6w0iPCzUhv0QVjQej25v3VWLvV9dWKGCRwh+TfwtLvpcZFho/u82ST5en8kPC1d79PoRtAZ1nJqcrnG9xvHl4zhNf4fam7eAsBbq3QyZLrZSI4frENlFotxLawXmlcbwe7NQOas3QPBWvCaTRw= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB2744.namprd10.prod.outlook.com (2603:10b6:a02:ae::18) by SJ0PR10MB4671.namprd10.prod.outlook.com (2603:10b6:a03:2d5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.19; Fri, 29 Jan 2021 16:47:11 +0000 Received: from BYAPR10MB2744.namprd10.prod.outlook.com ([fe80::74a8:e88d:b89e:bf2d]) by BYAPR10MB2744.namprd10.prod.outlook.com ([fe80::74a8:e88d:b89e:bf2d%4]) with mapi id 15.20.3805.020; Fri, 29 Jan 2021 16:47:11 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v21 17/20] multi-process: Synchronize remote memory Date: Fri, 29 Jan 2021 11:46:18 -0500 Message-Id: <04fe4e6a9ca90d4f11ab6f59be7652f5b086a071.1611938319.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: X-Originating-IP: [209.17.40.45] X-ClientProxiedBy: BYAPR05CA0053.namprd05.prod.outlook.com (2603:10b6:a03:74::30) To BYAPR10MB2744.namprd10.prod.outlook.com (2603:10b6:a02:ae::18) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from jaraman-bur-1.us.oracle.com (209.17.40.45) by BYAPR05CA0053.namprd05.prod.outlook.com (2603:10b6:a03:74::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.8 via Frontend Transport; Fri, 29 Jan 2021 16:47:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3b75d5f5-7916-4e75-9f1e-08d8c475860d X-MS-TrafficTypeDiagnostic: SJ0PR10MB4671: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3bbGkHsMk0lytpRTSH0iGrBTSfFPeAN3Rwr4SHVFkLyayIm3YEkAzCTXQhQreKIucbycSpRm9jjVd2bzFZRtdmW5lkPOTre7WvOtrtVuCly+ufjubZIls1P4BhMxWB35MbGZB8soEFc5v2J99b/d7ffHFL73gBJQGONV4p1Qliu/U2MmJb9Gg1mihKCbeAjhjPCEw+T8rg5ECyuU1IcPM9RfTvN/EFY22It8gwDmA8xLmww8//bqbWAuA98x+XvweUP2vp/8884VF3moVHkHAERssFGWS1pMnECTKz6iUXtT+rYPT6q7ziRn3ERemG69p5M5AEbFEdXZxFXM1+mVRtUHoLsBkCwFHTv3UxLpq/IoqzKDoMhO/vURviwi4RSz+Y/Jy/wIwhl0lT7q9JeESP0NQIehFeDoyUl75oDnryAX9SGtRFfjMy4dGWMYWI4ojuRcxmBD01Ma6bKfpN1++KgJ5yCP8rs0mA36/zrATHHmYyZ0DE2QY1lPyVNKLwcei3BNatLRCC7aK78bNJJ4YQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR10MB2744.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(136003)(366004)(376002)(39860400002)(396003)(346002)(956004)(2616005)(4326008)(8676002)(30864003)(6666004)(66556008)(478600001)(86362001)(36756003)(7416002)(107886003)(316002)(6916009)(2906002)(66476007)(16526019)(66946007)(7696005)(52116002)(6486002)(186003)(8936002)(5660300002)(83380400001)(26005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?GJviyP2YWOG1NGOy+yTnxs+ZF3j+te?= =?utf-8?q?1/z0bZhon1WIX1xCdsGlLJq9V7tCQAxJZDUg/4vK3cZsLkIJZy9MOC4itlebkJkjP?= =?utf-8?q?ki6IxN4CTE9ePeaMMV+0GNx2ny4IMHbPmxwkc2oh/SMg5fNnyCjt0WHOvB+6H0/EV?= =?utf-8?q?WOKNrvuTsod3XM3lQyAryfHBrWQGn0JKNLfXM8NUiYGqntW2mgIsv6gH1Jc0vg2c2?= =?utf-8?q?upLXP3h7Qu51QH3fKCLY17H7rwB9PKIuJKesq9XH+0U6jucTahXqhuT8CX9j+CUW/?= =?utf-8?q?cZvdOuZ+eCge3bLDHPOgIOd3Kj80Ky0lwu0q9tQ4k5u2qceRg+QTPZm4EUcs/xEve?= =?utf-8?q?yYMdbQ19seZ9VgylGIQSbmS0GXy3esc3yv/3A3OQbzF5U7BI+B+wT013jowhT1X7m?= =?utf-8?q?pHiOhWQ3J+yljHvPbuMOKu7WjHAQ2eyaj1pKue/CO0ehNY883on6JVLUhw0fCSATS?= =?utf-8?q?anRhgp2CV6Wp0vZDyOkNTkUQhwnF5w6fFwCme0PNhzlCS0J/l/gh2DXOQvA3R3XKH?= =?utf-8?q?WH97JbrLCs608umESFZZPz3bX1s/soucjcerFuGzoV3k+h0lOxR3BaHpFouqHAfq3?= =?utf-8?q?2aTWl8Q5OeDxbPk45w59dcTAsP23xLRsHeYiV17oIx0Wt+Him1Nmjprhb32LHUzlD?= =?utf-8?q?8++gdWDB7VIWLpzKJXjszQJUcWPsx3+3X0hJZ93TSaPjlFFbRnVTfeREBEflOHCnM?= =?utf-8?q?dJ0YrSFBa5j5/3c/QNg88lR1b2rsK8ZJoBYQCzkzbRZgL2vQz8C76gJuWrWBDtlCr?= =?utf-8?q?/iGAwHvdA/VU/pmcAe/57xUsWiLYZiTBMTR/56VBAa1k76jdoTlE/TupDNnYMqM52?= =?utf-8?q?ZZd/YJ1t3GiVxktAeIjfVawjusHFp9KS9ddQKnCIDQA4lvSc+UZokrU9dxXq1LqRm?= =?utf-8?q?0H0njZ6CsP9tq4ce87I1lQJhox/s91Q0gX9dpIrAA59CvuIshXQlOAP5UZ0yA8dLN?= =?utf-8?q?VJ+VvPxCowXsZWFN7nWN2XnL9XA72amNjUO+SzD92j2JAqLndlYvjUZU2dQX13JmH?= =?utf-8?q?2xb8fYFtZap8dRnZj+?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b75d5f5-7916-4e75-9f1e-08d8c475860d X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2744.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2021 16:47:11.4890 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fK4njiFBnSXFIhsEUR+VJNCcgoStGX2A4lWjgUjl/17/aBE1D7AEwP5tQfvS3T1my6eLCIL2XF1o25BRuf4ebw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4671 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9878 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 bulkscore=0 suspectscore=0 mlxscore=0 spamscore=0 phishscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101290081 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9878 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 lowpriorityscore=0 spamscore=0 clxscore=1015 adultscore=0 priorityscore=1501 impostorscore=0 phishscore=0 mlxscore=0 malwarescore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101290081 Received-SPF: pass client-ip=141.146.126.78; envelope-from=jag.raman@oracle.com; helo=aserp2120.oracle.com X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add ProxyMemoryListener object which is used to keep the view of the RAM in sync between QEMU and remote process. A MemoryListener is registered for system-memory AddressSpace. The listener sends SYNC_SYSMEM message to the remote process when memory listener commits the changes to memory, the remote process receives the message and processes it in the handler for SYNC_SYSMEM message. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Reviewed-by: Stefan Hajnoczi --- include/hw/remote/proxy-memory-listener.h | 28 ++++ include/hw/remote/proxy.h | 2 + hw/remote/message.c | 4 + hw/remote/proxy-memory-listener.c | 227 ++++++++++++++++++++++++++++++ hw/remote/proxy.c | 6 + MAINTAINERS | 2 + hw/remote/meson.build | 1 + 7 files changed, 270 insertions(+) create mode 100644 include/hw/remote/proxy-memory-listener.h create mode 100644 hw/remote/proxy-memory-listener.c diff --git a/include/hw/remote/proxy-memory-listener.h b/include/hw/remote/proxy-memory-listener.h new file mode 100644 index 0000000..c4f3efb --- /dev/null +++ b/include/hw/remote/proxy-memory-listener.h @@ -0,0 +1,28 @@ +/* + * Copyright © 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef PROXY_MEMORY_LISTENER_H +#define PROXY_MEMORY_LISTENER_H + +#include "exec/memory.h" +#include "io/channel.h" + +typedef struct ProxyMemoryListener { + MemoryListener listener; + + int n_mr_sections; + MemoryRegionSection *mr_sections; + + QIOChannel *ioc; +} ProxyMemoryListener; + +void proxy_memory_listener_configure(ProxyMemoryListener *proxy_listener, + QIOChannel *ioc); +void proxy_memory_listener_deconfigure(ProxyMemoryListener *proxy_listener); + +#endif diff --git a/include/hw/remote/proxy.h b/include/hw/remote/proxy.h index ea7fa4f..12888b4 100644 --- a/include/hw/remote/proxy.h +++ b/include/hw/remote/proxy.h @@ -11,6 +11,7 @@ #include "hw/pci/pci.h" #include "io/channel.h" +#include "hw/remote/proxy-memory-listener.h" #define TYPE_PCI_PROXY_DEV "x-pci-proxy-dev" OBJECT_DECLARE_SIMPLE_TYPE(PCIProxyDev, PCI_PROXY_DEV) @@ -36,6 +37,7 @@ struct PCIProxyDev { QemuMutex io_mutex; QIOChannel *ioc; Error *migration_blocker; + ProxyMemoryListener proxy_listener; ProxyMemoryRegion region[PCI_NUM_REGIONS]; }; diff --git a/hw/remote/message.c b/hw/remote/message.c index f2e8445..25341d8 100644 --- a/hw/remote/message.c +++ b/hw/remote/message.c @@ -17,6 +17,7 @@ #include "sysemu/runstate.h" #include "hw/pci/pci.h" #include "exec/memattrs.h" +#include "hw/remote/memory.h" static void process_config_write(QIOChannel *ioc, PCIDevice *dev, MPQemuMsg *msg, Error **errp); @@ -61,6 +62,9 @@ void coroutine_fn mpqemu_remote_msg_loop_co(void *data) case MPQEMU_CMD_BAR_READ: process_bar_read(com->ioc, &msg, &local_err); break; + case MPQEMU_CMD_SYNC_SYSMEM: + remote_sysmem_reconfig(&msg, &local_err); + break; default: error_setg(&local_err, "Unknown command (%d) received for device %s" diff --git a/hw/remote/proxy-memory-listener.c b/hw/remote/proxy-memory-listener.c new file mode 100644 index 0000000..af1fa6f --- /dev/null +++ b/hw/remote/proxy-memory-listener.c @@ -0,0 +1,227 @@ +/* + * Copyright © 2018, 2021 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "qemu/compiler.h" +#include "qemu/int128.h" +#include "qemu/range.h" +#include "exec/memory.h" +#include "exec/cpu-common.h" +#include "cpu.h" +#include "exec/ram_addr.h" +#include "exec/address-spaces.h" +#include "qapi/error.h" +#include "hw/remote/mpqemu-link.h" +#include "hw/remote/proxy-memory-listener.h" + +/* + * TODO: get_fd_from_hostaddr(), proxy_mrs_can_merge() and + * proxy_memory_listener_commit() defined below perform tasks similar to the + * functions defined in vhost-user.c. These functions are good candidates + * for refactoring. + * + */ + +static void proxy_memory_listener_reset(MemoryListener *listener) +{ + ProxyMemoryListener *proxy_listener = container_of(listener, + ProxyMemoryListener, + listener); + int mrs; + + for (mrs = 0; mrs < proxy_listener->n_mr_sections; mrs++) { + memory_region_unref(proxy_listener->mr_sections[mrs].mr); + } + + g_free(proxy_listener->mr_sections); + proxy_listener->mr_sections = NULL; + proxy_listener->n_mr_sections = 0; +} + +static int get_fd_from_hostaddr(uint64_t host, ram_addr_t *offset) +{ + MemoryRegion *mr; + ram_addr_t off; + + /** + * Assumes that the host address is a valid address as it's + * coming from the MemoryListener system. In the case host + * address is not valid, the following call would return + * the default subregion of "system_memory" region, and + * not NULL. So it's not possible to check for NULL here. + */ + mr = memory_region_from_host((void *)(uintptr_t)host, &off); + + if (offset) { + *offset = off; + } + + return memory_region_get_fd(mr); +} + +static bool proxy_mrs_can_merge(uint64_t host, uint64_t prev_host, size_t size) +{ + if (((prev_host + size) != host)) { + return false; + } + + if (get_fd_from_hostaddr(host, NULL) != + get_fd_from_hostaddr(prev_host, NULL)) { + return false; + } + + return true; +} + +static bool try_merge(ProxyMemoryListener *proxy_listener, + MemoryRegionSection *section) +{ + uint64_t mrs_size, mrs_gpa, mrs_page; + MemoryRegionSection *prev_sec; + bool merged = false; + uintptr_t mrs_host; + RAMBlock *mrs_rb; + + if (!proxy_listener->n_mr_sections) { + return false; + } + + mrs_rb = section->mr->ram_block; + mrs_page = (uint64_t)qemu_ram_pagesize(mrs_rb); + mrs_size = int128_get64(section->size); + mrs_gpa = section->offset_within_address_space; + mrs_host = (uintptr_t)memory_region_get_ram_ptr(section->mr) + + section->offset_within_region; + + if (get_fd_from_hostaddr(mrs_host, NULL) < 0) { + return true; + } + + mrs_host = mrs_host & ~(mrs_page - 1); + mrs_gpa = mrs_gpa & ~(mrs_page - 1); + mrs_size = ROUND_UP(mrs_size, mrs_page); + + prev_sec = proxy_listener->mr_sections + + (proxy_listener->n_mr_sections - 1); + uint64_t prev_gpa_start = prev_sec->offset_within_address_space; + uint64_t prev_size = int128_get64(prev_sec->size); + uint64_t prev_gpa_end = range_get_last(prev_gpa_start, prev_size); + uint64_t prev_host_start = + (uintptr_t)memory_region_get_ram_ptr(prev_sec->mr) + + prev_sec->offset_within_region; + uint64_t prev_host_end = range_get_last(prev_host_start, prev_size); + + if (mrs_gpa <= (prev_gpa_end + 1)) { + g_assert(mrs_gpa > prev_gpa_start); + + if ((section->mr == prev_sec->mr) && + proxy_mrs_can_merge(mrs_host, prev_host_start, + (mrs_gpa - prev_gpa_start))) { + uint64_t max_end = MAX(prev_host_end, mrs_host + mrs_size); + merged = true; + prev_sec->offset_within_address_space = + MIN(prev_gpa_start, mrs_gpa); + prev_sec->offset_within_region = + MIN(prev_host_start, mrs_host) - + (uintptr_t)memory_region_get_ram_ptr(prev_sec->mr); + prev_sec->size = int128_make64(max_end - MIN(prev_host_start, + mrs_host)); + } + } + + return merged; +} + +static void proxy_memory_listener_region_addnop(MemoryListener *listener, + MemoryRegionSection *section) +{ + ProxyMemoryListener *proxy_listener = container_of(listener, + ProxyMemoryListener, + listener); + + if (!memory_region_is_ram(section->mr) || + memory_region_is_rom(section->mr)) { + return; + } + + if (try_merge(proxy_listener, section)) { + return; + } + + ++proxy_listener->n_mr_sections; + proxy_listener->mr_sections = g_renew(MemoryRegionSection, + proxy_listener->mr_sections, + proxy_listener->n_mr_sections); + proxy_listener->mr_sections[proxy_listener->n_mr_sections - 1] = *section; + proxy_listener->mr_sections[proxy_listener->n_mr_sections - 1].fv = NULL; + memory_region_ref(section->mr); +} + +static void proxy_memory_listener_commit(MemoryListener *listener) +{ + ProxyMemoryListener *proxy_listener = container_of(listener, + ProxyMemoryListener, + listener); + MPQemuMsg msg; + MemoryRegionSection *section; + ram_addr_t offset; + uintptr_t host_addr; + int region; + Error *local_err = NULL; + + memset(&msg, 0, sizeof(MPQemuMsg)); + + msg.cmd = MPQEMU_CMD_SYNC_SYSMEM; + msg.num_fds = proxy_listener->n_mr_sections; + msg.size = sizeof(SyncSysmemMsg); + if (msg.num_fds > REMOTE_MAX_FDS) { + error_report("Number of fds is more than %d", REMOTE_MAX_FDS); + return; + } + + for (region = 0; region < proxy_listener->n_mr_sections; region++) { + section = &proxy_listener->mr_sections[region]; + msg.data.sync_sysmem.gpas[region] = + section->offset_within_address_space; + msg.data.sync_sysmem.sizes[region] = int128_get64(section->size); + host_addr = (uintptr_t)memory_region_get_ram_ptr(section->mr) + + section->offset_within_region; + msg.fds[region] = get_fd_from_hostaddr(host_addr, &offset); + msg.data.sync_sysmem.offsets[region] = offset; + } + if (!mpqemu_msg_send(&msg, proxy_listener->ioc, &local_err)) { + error_report_err(local_err); + } +} + +void proxy_memory_listener_deconfigure(ProxyMemoryListener *proxy_listener) +{ + memory_listener_unregister(&proxy_listener->listener); + + proxy_memory_listener_reset(&proxy_listener->listener); +} + +void proxy_memory_listener_configure(ProxyMemoryListener *proxy_listener, + QIOChannel *ioc) +{ + proxy_listener->n_mr_sections = 0; + proxy_listener->mr_sections = NULL; + + proxy_listener->ioc = ioc; + + proxy_listener->listener.begin = proxy_memory_listener_reset; + proxy_listener->listener.commit = proxy_memory_listener_commit; + proxy_listener->listener.region_add = proxy_memory_listener_region_addnop; + proxy_listener->listener.region_nop = proxy_memory_listener_region_addnop; + proxy_listener->listener.priority = 10; + + memory_listener_register(&proxy_listener->listener, + &address_space_memory); +} diff --git a/hw/remote/proxy.c b/hw/remote/proxy.c index 22eb422..472b2df 100644 --- a/hw/remote/proxy.c +++ b/hw/remote/proxy.c @@ -19,6 +19,8 @@ #include "qemu/sockets.h" #include "hw/remote/mpqemu-link.h" #include "qemu/error-report.h" +#include "hw/remote/proxy-memory-listener.h" +#include "qom/object.h" static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) { @@ -52,6 +54,8 @@ static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) qemu_mutex_init(&dev->io_mutex); qio_channel_set_blocking(dev->ioc, true, NULL); + + proxy_memory_listener_configure(&dev->proxy_listener, dev->ioc); } static void pci_proxy_dev_exit(PCIDevice *pdev) @@ -65,6 +69,8 @@ static void pci_proxy_dev_exit(PCIDevice *pdev) migrate_del_blocker(dev->migration_blocker); error_free(dev->migration_blocker); + + proxy_memory_listener_deconfigure(&dev->proxy_listener); } static void config_op_send(PCIProxyDev *pdev, uint32_t addr, uint32_t *val, diff --git a/MAINTAINERS b/MAINTAINERS index c60c333..b8b24d6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3200,6 +3200,8 @@ F: include/hw/remote/memory.h F: hw/remote/memory.c F: hw/remote/proxy.c F: include/hw/remote/proxy.h +F: hw/remote/proxy-memory-listener.c +F: include/hw/remote/proxy-memory-listener.h Build and test automation ------------------------- diff --git a/hw/remote/meson.build b/hw/remote/meson.build index 569cd20..7f11be4 100644 --- a/hw/remote/meson.build +++ b/hw/remote/meson.build @@ -7,5 +7,6 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c')) remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c')) specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('memory.c')) +specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy-memory-listener.c')) softmmu_ss.add_all(when: 'CONFIG_MULTIPROCESS', if_true: remote_ss)