From patchwork Tue Jul 4 08:06:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattias Nissler X-Patchwork-Id: 13300759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1F511EB64D9 for ; Tue, 4 Jul 2023 08:07:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qGb3n-0006nC-13; Tue, 04 Jul 2023 04:06:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGb3m-0006mz-4U for qemu-devel@nongnu.org; Tue, 04 Jul 2023 04:06:54 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qGb3g-0000sN-H9 for qemu-devel@nongnu.org; Tue, 04 Jul 2023 04:06:53 -0400 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-666e64e97e2so2716277b3a.1 for ; Tue, 04 Jul 2023 01:06:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20221208.gappssmtp.com; s=20221208; t=1688458007; x=1691050007; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gQ690QrMYdqdPeWSwhBzqou+wm7X9S8MDsUYTaTIvYY=; b=LwdnSD8EynBFBd/Q7iRdMxtbRk0YykA3puw+U3Ngj4LmWtkuawzQlKPyv9aLJrTTR/ T1U2KHSuJEHP9NESrYmSvknRMwc0/ev8Y+zdkMQWNllaKPaLbs5KMbAdibxlCQVOpLMP luXmux2lrlErRWcj1bBhjNjmfmd0PoC8UInZ1sZI5R5iDmb42LwaHpLvgLQrWKK98+2u 3VA2Ugi1wwpTKyySleWEojVgczeGzD/psiefFY8wkVd1r0MQfsMy7/X22L6uDO+LV8lH 93c88EsLtz/qa0tKMPJkfl3itFcGLnv1Qmf7CjKMEteNh3b3TNkS6YFp1iS0kB63ttgo bbpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688458007; x=1691050007; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gQ690QrMYdqdPeWSwhBzqou+wm7X9S8MDsUYTaTIvYY=; b=ZFivwvgxs/wN6Pld0Ysuxo+7tFPMA7fQMM/GpgMKqFBBzbgFPiEX42HbfuXpW6AdPy 5O5uG0peGVl4roY7QXMcdnL8morHswCl6MTNPVycbO2kBJQPCSIu+cZEpa331EStHLfe GXj/qwj46rgDlRjgd+pALID5jEH8HbLPDcWpFeR72O6nbxLdk6TyIcl7mAYZsc1uVVNq BVEWAVWuZdEQebJxWrH/Bai5a0ocevhwYer2vFoikw5W4VcFJsHsE0StVvnyaE7KDbQP KN6tG1K7HCgjyNiWhyFF0hfZIgt+fq2rRP0+rmVp4kvDl1nbRmvby+9UH4hIumpMO269 JR5Q== X-Gm-Message-State: ABy/qLZu11ymUtpvIp3GZiv1/bkDL9udIT+PZDFmRqZQAxOopMf0Vcn6 kMrwYKCUa/xRRWdnhdPZc3+L4M84QJon4hpeyZ4= X-Google-Smtp-Source: APBJJlFuFRFsA9QE3oilriDfAesYiBkU4Ve/gyqnZHn8gphzu0EZtN0GU2qNiYdZP3CnbKeXoOV/+A== X-Received: by 2002:a05:6a00:2389:b0:668:681f:ca98 with SMTP id f9-20020a056a00238900b00668681fca98mr14549431pfc.27.1688458006794; Tue, 04 Jul 2023 01:06:46 -0700 (PDT) Received: from mnissler.ba.rivosinc.com ([66.220.2.162]) by smtp.gmail.com with ESMTPSA id u1-20020aa78381000000b00662c4ca18ebsm15113101pfm.128.2023.07.04.01.06.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jul 2023 01:06:46 -0700 (PDT) From: Mattias Nissler To: qemu-devel@nongnu.org Cc: Elena Ufimtseva , Jagannathan Raman , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Paolo Bonzini , Peter Xu , stefanha@redhat.com, David Hildenbrand , john.levon@nutanix.com, Mattias Nissler Subject: [PATCH 1/3] softmmu: Support concurrent bounce buffers Date: Tue, 4 Jul 2023 01:06:25 -0700 Message-Id: <20230704080628.852525-2-mnissler@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230704080628.852525-1-mnissler@rivosinc.com> References: <20230704080628.852525-1-mnissler@rivosinc.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=mnissler@rivosinc.com; helo=mail-pf1-x435.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org It is not uncommon for device models to request mapping of several DMA regions at the same time. An example is igb (and probably other net devices as well) when a packet is spread across multiple descriptors. In order to support this when indirect DMA is used, as is the case when running the device model in a vfio-server process without mmap()-ed DMA, this change allocates DMA bounce buffers dynamically instead of supporting only a single buffer. Signed-off-by: Mattias Nissler --- softmmu/physmem.c | 74 ++++++++++++++++++++++------------------------- 1 file changed, 35 insertions(+), 39 deletions(-) diff --git a/softmmu/physmem.c b/softmmu/physmem.c index bda475a719..56130b5a1d 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -2904,13 +2904,11 @@ void cpu_flush_icache_range(hwaddr start, hwaddr len) typedef struct { MemoryRegion *mr; - void *buffer; hwaddr addr; - hwaddr len; - bool in_use; + uint8_t buffer[]; } BounceBuffer; -static BounceBuffer bounce; +static size_t bounce_buffers_in_use; typedef struct MapClient { QEMUBH *bh; @@ -2947,7 +2945,7 @@ void cpu_register_map_client(QEMUBH *bh) QLIST_INSERT_HEAD(&map_client_list, client, link); /* Write map_client_list before reading in_use. */ smp_mb(); - if (!qatomic_read(&bounce.in_use)) { + if (qatomic_read(&bounce_buffers_in_use)) { cpu_notify_map_clients_locked(); } qemu_mutex_unlock(&map_client_list_lock); @@ -3076,31 +3074,24 @@ void *address_space_map(AddressSpace *as, RCU_READ_LOCK_GUARD(); fv = address_space_to_flatview(as); mr = flatview_translate(fv, addr, &xlat, &l, is_write, attrs); + memory_region_ref(mr); if (!memory_access_is_direct(mr, is_write)) { - if (qatomic_xchg(&bounce.in_use, true)) { - *plen = 0; - return NULL; - } - /* Avoid unbounded allocations */ - l = MIN(l, TARGET_PAGE_SIZE); - bounce.buffer = qemu_memalign(TARGET_PAGE_SIZE, l); - bounce.addr = addr; - bounce.len = l; - - memory_region_ref(mr); - bounce.mr = mr; + qatomic_inc_fetch(&bounce_buffers_in_use); + + BounceBuffer *bounce = g_malloc(l + sizeof(BounceBuffer)); + bounce->addr = addr; + bounce->mr = mr; + if (!is_write) { flatview_read(fv, addr, MEMTXATTRS_UNSPECIFIED, - bounce.buffer, l); + bounce->buffer, l); } *plen = l; - return bounce.buffer; + return bounce->buffer; } - - memory_region_ref(mr); *plen = flatview_extend_translation(fv, addr, len, mr, xlat, l, is_write, attrs); fuzz_dma_read_cb(addr, *plen, mr); @@ -3114,31 +3105,36 @@ void *address_space_map(AddressSpace *as, void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len, bool is_write, hwaddr access_len) { - if (buffer != bounce.buffer) { - MemoryRegion *mr; - ram_addr_t addr1; + MemoryRegion *mr; + ram_addr_t addr1; + + mr = memory_region_from_host(buffer, &addr1); + if (mr == NULL) { + /* + * Must be a bounce buffer (unless the caller passed a pointer which + * wasn't returned by address_space_map, which is illegal). + */ + BounceBuffer *bounce = container_of(buffer, BounceBuffer, buffer); - mr = memory_region_from_host(buffer, &addr1); - assert(mr != NULL); if (is_write) { - invalidate_and_set_dirty(mr, addr1, access_len); + address_space_write(as, bounce->addr, MEMTXATTRS_UNSPECIFIED, + bounce->buffer, access_len); } - if (xen_enabled()) { - xen_invalidate_map_cache_entry(buffer); + memory_region_unref(bounce->mr); + g_free(bounce); + + if (qatomic_dec_fetch(&bounce_buffers_in_use) == 1) { + cpu_notify_map_clients(); } - memory_region_unref(mr); return; } + + if (xen_enabled()) { + xen_invalidate_map_cache_entry(buffer); + } if (is_write) { - address_space_write(as, bounce.addr, MEMTXATTRS_UNSPECIFIED, - bounce.buffer, access_len); - } - qemu_vfree(bounce.buffer); - bounce.buffer = NULL; - memory_region_unref(bounce.mr); - /* Clear in_use before reading map_client_list. */ - qatomic_set_mb(&bounce.in_use, false); - cpu_notify_map_clients(); + invalidate_and_set_dirty(mr, addr1, access_len); + } } void *cpu_physical_memory_map(hwaddr addr, From patchwork Tue Jul 4 08:06:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattias Nissler X-Patchwork-Id: 13300760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2BA4EB64DD for ; Tue, 4 Jul 2023 08:08:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qGb3l-0006mX-QS; Tue, 04 Jul 2023 04:06:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGb3k-0006lv-CQ for qemu-devel@nongnu.org; Tue, 04 Jul 2023 04:06:52 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qGb3h-0000ta-5u for qemu-devel@nongnu.org; Tue, 04 Jul 2023 04:06:52 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-6686708c986so4331631b3a.0 for ; Tue, 04 Jul 2023 01:06:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20221208.gappssmtp.com; s=20221208; t=1688458008; x=1691050008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iBKn+fLwG4WbDopaOGiZKodHGjHZmzgCsifZztYw2kw=; b=o0RJ+dkUAlH8Zj4CiDZJIFG1GXrDPaFjdX1698RMOvY2i8GsYa9ONGIIEzFd5eIJXE IH+u9Ztl6j1BrJr8bywr1IxX7bsKA7W1hYguJa8eOQMSbwJZRlq7kiQ8insSiT5hOksc eQfs1NqooJ6ZCyculb81OMsNFhIWwPF4piOCwFJ6UzHFTjbfWG9xFNPTSuI+jIqL0uuw hWi0UQTq2WPZIGX9rRyiLMieJ5zv2FMViy1NdyRIw/5awFiWGvY4snrzXRZPGT6ZeOQx VZRqTpWEONqWpd5Ft2Rt4a7NLO8sGMtjJG0dMstOsjXczgOXHX/+OwcOmKt/PkQoRQnT U8fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688458008; x=1691050008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iBKn+fLwG4WbDopaOGiZKodHGjHZmzgCsifZztYw2kw=; b=JpFJzcc+0+n9pdSQj+T/nQderxgFGcPzIN4KUFWC7aeBqwIucwbQ4u/Dc/4wqQWKWe 3tJXpneGwtI0vR8DDP1avDvSFPQMZRK7/LCh0hPG0ar/Jrpm8aiJQEFTcOc5y+JH5Z9K Yj6I4UjtB0bBYRm+BYbTg+p92J+/DIq/LHuBum3rG36o8b1mNb5gIZI3qhdCIKr0NTf+ qcItF3Ec6pfXAxDcnJjzKCsiUB4j3IMlrdpb96GlDHA+G70JNP7G2cTUmDyjlKQW161L wDgHhd0gfMZt6PnqF0Z65m+qJWQY8Ik6wXOhNV5Nf/8kPuuBfQpPlNESf014PQa95EHU wPTg== X-Gm-Message-State: ABy/qLYdL+pvtJFw+bM5AuBTBhDkF+eCe09BhDBkzBddzJY9QLUe55+/ fL60IKhInNel+ospJqtvvSdaLImyDPKvq9VX8WkDyw== X-Google-Smtp-Source: APBJJlG4AXASuhwYZGfhChvCqxS3dwQFnDX/u1UJRPXZ+J4UDYPYwghEvTGyFvafNxAUAb3L213XsQ== X-Received: by 2002:a05:6a00:2488:b0:681:142f:e8e3 with SMTP id c8-20020a056a00248800b00681142fe8e3mr17100221pfv.14.1688458007714; Tue, 04 Jul 2023 01:06:47 -0700 (PDT) Received: from mnissler.ba.rivosinc.com ([66.220.2.162]) by smtp.gmail.com with ESMTPSA id u1-20020aa78381000000b00662c4ca18ebsm15113101pfm.128.2023.07.04.01.06.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jul 2023 01:06:47 -0700 (PDT) From: Mattias Nissler To: qemu-devel@nongnu.org Cc: Elena Ufimtseva , Jagannathan Raman , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Paolo Bonzini , Peter Xu , stefanha@redhat.com, David Hildenbrand , john.levon@nutanix.com, Mattias Nissler Subject: [PATCH 2/3] softmmu: Remove DMA unmap notification callback Date: Tue, 4 Jul 2023 01:06:26 -0700 Message-Id: <20230704080628.852525-3-mnissler@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230704080628.852525-1-mnissler@rivosinc.com> References: <20230704080628.852525-1-mnissler@rivosinc.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=mnissler@rivosinc.com; helo=mail-pf1-x434.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org According to old commit messages, this was introduced to retry a DMA operation at a later point in case the single bounce buffer is found to be busy. This was never used widely - only the dma-helpers code made use of it, but there are other device models that use multiple DMA mappings (concurrently) and just failed. After the improvement to support multiple concurrent bounce buffers, the condition the notification callback allowed to work around no longer exists, so we can just remove the logic and simplify the code. Signed-off-by: Mattias Nissler --- softmmu/dma-helpers.c | 28 ----------------- softmmu/physmem.c | 71 ------------------------------------------- 2 files changed, 99 deletions(-) diff --git a/softmmu/dma-helpers.c b/softmmu/dma-helpers.c index 2463964805..d05d226f11 100644 --- a/softmmu/dma-helpers.c +++ b/softmmu/dma-helpers.c @@ -68,23 +68,10 @@ typedef struct { int sg_cur_index; dma_addr_t sg_cur_byte; QEMUIOVector iov; - QEMUBH *bh; DMAIOFunc *io_func; void *io_func_opaque; } DMAAIOCB; -static void dma_blk_cb(void *opaque, int ret); - -static void reschedule_dma(void *opaque) -{ - DMAAIOCB *dbs = (DMAAIOCB *)opaque; - - assert(!dbs->acb && dbs->bh); - qemu_bh_delete(dbs->bh); - dbs->bh = NULL; - dma_blk_cb(dbs, 0); -} - static void dma_blk_unmap(DMAAIOCB *dbs) { int i; @@ -101,7 +88,6 @@ static void dma_complete(DMAAIOCB *dbs, int ret) { trace_dma_complete(dbs, ret, dbs->common.cb); - assert(!dbs->acb && !dbs->bh); dma_blk_unmap(dbs); if (dbs->common.cb) { dbs->common.cb(dbs->common.opaque, ret); @@ -164,13 +150,6 @@ static void dma_blk_cb(void *opaque, int ret) } } - if (dbs->iov.size == 0) { - trace_dma_map_wait(dbs); - dbs->bh = aio_bh_new(ctx, reschedule_dma, dbs); - cpu_register_map_client(dbs->bh); - goto out; - } - if (!QEMU_IS_ALIGNED(dbs->iov.size, dbs->align)) { qemu_iovec_discard_back(&dbs->iov, QEMU_ALIGN_DOWN(dbs->iov.size, dbs->align)); @@ -189,18 +168,12 @@ static void dma_aio_cancel(BlockAIOCB *acb) trace_dma_aio_cancel(dbs); - assert(!(dbs->acb && dbs->bh)); if (dbs->acb) { /* This will invoke dma_blk_cb. */ blk_aio_cancel_async(dbs->acb); return; } - if (dbs->bh) { - cpu_unregister_map_client(dbs->bh); - qemu_bh_delete(dbs->bh); - dbs->bh = NULL; - } if (dbs->common.cb) { dbs->common.cb(dbs->common.opaque, -ECANCELED); } @@ -239,7 +212,6 @@ BlockAIOCB *dma_blk_io(AioContext *ctx, dbs->dir = dir; dbs->io_func = io_func; dbs->io_func_opaque = io_func_opaque; - dbs->bh = NULL; qemu_iovec_init(&dbs->iov, sg->nsg); dma_blk_cb(dbs, 0); return &dbs->common; diff --git a/softmmu/physmem.c b/softmmu/physmem.c index 56130b5a1d..2b4123c127 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -2908,49 +2908,6 @@ typedef struct { uint8_t buffer[]; } BounceBuffer; -static size_t bounce_buffers_in_use; - -typedef struct MapClient { - QEMUBH *bh; - QLIST_ENTRY(MapClient) link; -} MapClient; - -QemuMutex map_client_list_lock; -static QLIST_HEAD(, MapClient) map_client_list - = QLIST_HEAD_INITIALIZER(map_client_list); - -static void cpu_unregister_map_client_do(MapClient *client) -{ - QLIST_REMOVE(client, link); - g_free(client); -} - -static void cpu_notify_map_clients_locked(void) -{ - MapClient *client; - - while (!QLIST_EMPTY(&map_client_list)) { - client = QLIST_FIRST(&map_client_list); - qemu_bh_schedule(client->bh); - cpu_unregister_map_client_do(client); - } -} - -void cpu_register_map_client(QEMUBH *bh) -{ - MapClient *client = g_malloc(sizeof(*client)); - - qemu_mutex_lock(&map_client_list_lock); - client->bh = bh; - QLIST_INSERT_HEAD(&map_client_list, client, link); - /* Write map_client_list before reading in_use. */ - smp_mb(); - if (qatomic_read(&bounce_buffers_in_use)) { - cpu_notify_map_clients_locked(); - } - qemu_mutex_unlock(&map_client_list_lock); -} - void cpu_exec_init_all(void) { qemu_mutex_init(&ram_list.mutex); @@ -2964,28 +2921,6 @@ void cpu_exec_init_all(void) finalize_target_page_bits(); io_mem_init(); memory_map_init(); - qemu_mutex_init(&map_client_list_lock); -} - -void cpu_unregister_map_client(QEMUBH *bh) -{ - MapClient *client; - - qemu_mutex_lock(&map_client_list_lock); - QLIST_FOREACH(client, &map_client_list, link) { - if (client->bh == bh) { - cpu_unregister_map_client_do(client); - break; - } - } - qemu_mutex_unlock(&map_client_list_lock); -} - -static void cpu_notify_map_clients(void) -{ - qemu_mutex_lock(&map_client_list_lock); - cpu_notify_map_clients_locked(); - qemu_mutex_unlock(&map_client_list_lock); } static bool flatview_access_valid(FlatView *fv, hwaddr addr, hwaddr len, @@ -3077,8 +3012,6 @@ void *address_space_map(AddressSpace *as, memory_region_ref(mr); if (!memory_access_is_direct(mr, is_write)) { - qatomic_inc_fetch(&bounce_buffers_in_use); - BounceBuffer *bounce = g_malloc(l + sizeof(BounceBuffer)); bounce->addr = addr; bounce->mr = mr; @@ -3122,10 +3055,6 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len, } memory_region_unref(bounce->mr); g_free(bounce); - - if (qatomic_dec_fetch(&bounce_buffers_in_use) == 1) { - cpu_notify_map_clients(); - } return; } From patchwork Tue Jul 4 08:06:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mattias Nissler X-Patchwork-Id: 13300758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5E647EB64D9 for ; Tue, 4 Jul 2023 08:07:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qGb3l-0006mT-Mq; Tue, 04 Jul 2023 04:06:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qGb3k-0006lw-Db for qemu-devel@nongnu.org; Tue, 04 Jul 2023 04:06:52 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qGb3h-0000vm-TZ for qemu-devel@nongnu.org; Tue, 04 Jul 2023 04:06:52 -0400 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-66869feb7d1so2695492b3a.3 for ; Tue, 04 Jul 2023 01:06:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20221208.gappssmtp.com; s=20221208; t=1688458008; x=1691050008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HFNNCtPqWtjxwt+qHGQAYUqPKlbWW72Qgk4NOXVFZYs=; b=eTaoSz2JvUQBoObqpxPhXdllw0/5dtohOVJgEYDXlmPmVinyMJJ6FCRDHaPJwiBWom y+ERZPfp4ATyBLKp7kG8BJyrZ3n4HKPWj/GfZcWG0XnLWoVm7xMKOP4nSX23fbLjsfQE J6+cQkqDWdHdqdAy1sCqDrsRVln/6m5FhaN6mlUpzfgY5q3olxs4vL0yL1Yhw0+gYpyk kOoUe2VVKagDNMJ0gTlA3szD8tE62mPfwsHR5ZYs5gjwodGFvqsDcwemYlgySMlwnr7H o+1H7psjPle4d4fZ/ur70nZCFGM6AaNbZ+JtfymairvpDx7vl+tb5AmBfzJe4I1Yg1LI DcEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688458008; x=1691050008; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HFNNCtPqWtjxwt+qHGQAYUqPKlbWW72Qgk4NOXVFZYs=; b=iAUHhUpCWgSZAclyNxJQahgg0HcnozkuwVf/bIXMt4TVqvN/lFKi2JzC276sxnbX/H IAb4xeTEaRZtsPRowIsiYFbXze0y2/PR/xKEmdvtsPByqm3GmfrL/e8TekyN+LdNdWna RCuQSeNwXdc6KjW7jEErwjbRmONX8P0nOUKeC/sZgzhhMkChG0vUzDRIWTvU69bujBDI RzbDaZd6cq9CWQf9d8geulBvyG92z1nC366EnrTWlcJD8QfHos5U3ZNducEJyUrfve1k tlnIgf45Jb+FnwRdx++TUy+do0ER3z6Ldk4MqAYe5dLzr5qWzAPbsT3YJ/mqdXmgA4+V A1fA== X-Gm-Message-State: ABy/qLbg6Ec1Bz93Td+KDsT8zbVSRKUe3epUAufvXCvwmxbXzaqE71RF lAXXdxQsJnM+qa0oySzzh/BCVf+YGYw7H6HVDwafdQ== X-Google-Smtp-Source: APBJJlGtzmlW93GdkFzk6DhoIma8WvJ/stKAWNA4/HrG7+vMFWpI8sy868TGWeoWVvH3cxmMVzeLVQ== X-Received: by 2002:a05:6a00:1910:b0:668:7744:10f5 with SMTP id y16-20020a056a00191000b00668774410f5mr13299584pfi.10.1688458008534; Tue, 04 Jul 2023 01:06:48 -0700 (PDT) Received: from mnissler.ba.rivosinc.com ([66.220.2.162]) by smtp.gmail.com with ESMTPSA id u1-20020aa78381000000b00662c4ca18ebsm15113101pfm.128.2023.07.04.01.06.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Jul 2023 01:06:48 -0700 (PDT) From: Mattias Nissler To: qemu-devel@nongnu.org Cc: Elena Ufimtseva , Jagannathan Raman , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Paolo Bonzini , Peter Xu , stefanha@redhat.com, David Hildenbrand , john.levon@nutanix.com, Mattias Nissler Subject: [PATCH 3/3] vfio-user: Message-based DMA support Date: Tue, 4 Jul 2023 01:06:27 -0700 Message-Id: <20230704080628.852525-4-mnissler@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230704080628.852525-1-mnissler@rivosinc.com> References: <20230704080628.852525-1-mnissler@rivosinc.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=mnissler@rivosinc.com; helo=mail-pf1-x436.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Wire up support for DMA for the case where the vfio-user client does not provide mmap()-able file descriptors, but DMA requests must be performed via the VFIO-user protocol. This installs an indirect memory region, which already works for pci_dma_{read,write}, and pci_dma_map works thanks to the existing DMA bounce buffering support. Note that while simple scenarios work with this patch, there's a known race condition in libvfio-user that will mess up the communication channel: https://github.com/nutanix/libvfio-user/issues/279 I intend to contribute a fix for this problem, see discussion on the github issue for more details. Signed-off-by: Mattias Nissler --- hw/remote/vfio-user-obj.c | 62 ++++++++++++++++++++++++++++++++++----- 1 file changed, 55 insertions(+), 7 deletions(-) diff --git a/hw/remote/vfio-user-obj.c b/hw/remote/vfio-user-obj.c index 8b10c32a3c..9799580c77 100644 --- a/hw/remote/vfio-user-obj.c +++ b/hw/remote/vfio-user-obj.c @@ -300,6 +300,53 @@ static ssize_t vfu_object_cfg_access(vfu_ctx_t *vfu_ctx, char * const buf, return count; } +static MemTxResult vfu_dma_read(void *opaque, hwaddr addr, uint64_t *val, + unsigned size, MemTxAttrs attrs) +{ + MemoryRegion *region = opaque; + VfuObject *o = VFU_OBJECT(region->owner); + + dma_sg_t *sg = alloca(dma_sg_size()); + vfu_dma_addr_t vfu_addr = (vfu_dma_addr_t)(region->addr + addr); + if (vfu_addr_to_sgl(o->vfu_ctx, vfu_addr, size, sg, 1, PROT_READ) < 0 || + vfu_sgl_read(o->vfu_ctx, sg, 1, val) != 0) { + return MEMTX_ERROR; + } + + return MEMTX_OK; +} + +static MemTxResult vfu_dma_write(void *opaque, hwaddr addr, uint64_t val, + unsigned size, MemTxAttrs attrs) +{ + MemoryRegion *region = opaque; + VfuObject *o = VFU_OBJECT(region->owner); + + dma_sg_t *sg = alloca(dma_sg_size()); + vfu_dma_addr_t vfu_addr = (vfu_dma_addr_t)(region->addr + addr); + if (vfu_addr_to_sgl(o->vfu_ctx, vfu_addr, size, sg, 1, PROT_WRITE) < 0 || + vfu_sgl_write(o->vfu_ctx, sg, 1, &val) != 0) { + return MEMTX_ERROR; + } + + return MEMTX_OK; +} + +static const MemoryRegionOps vfu_dma_ops = { + .read_with_attrs = vfu_dma_read, + .write_with_attrs = vfu_dma_write, + .endianness = DEVICE_NATIVE_ENDIAN, + .valid = { + .min_access_size = 1, + .max_access_size = 8, + .unaligned = true, + }, + .impl = { + .min_access_size = 1, + .max_access_size = 8, + }, +}; + static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) { VfuObject *o = vfu_get_private(vfu_ctx); @@ -308,17 +355,18 @@ static void dma_register(vfu_ctx_t *vfu_ctx, vfu_dma_info_t *info) g_autofree char *name = NULL; struct iovec *iov = &info->iova; - if (!info->vaddr) { - return; - } - name = g_strdup_printf("mem-%s-%"PRIx64"", o->device, - (uint64_t)info->vaddr); + (uint64_t)iov->iov_base); subregion = g_new0(MemoryRegion, 1); - memory_region_init_ram_ptr(subregion, NULL, name, - iov->iov_len, info->vaddr); + if (info->vaddr) { + memory_region_init_ram_ptr(subregion, OBJECT(o), name, + iov->iov_len, info->vaddr); + } else { + memory_region_init_io(subregion, OBJECT(o), &vfu_dma_ops, subregion, + name, iov->iov_len); + } dma_as = pci_device_iommu_address_space(o->pci_dev);