From patchwork Mon Mar 25 08:13:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Wiklander X-Patchwork-Id: 13601617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40E0BC54E64 for ; Mon, 25 Mar 2024 08:14:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=KVbaD9Kvvu6GCoXtg+JTleOSyqlvujiK05FcQcP3FaI=; b=LP5Zi7dVl/ENXm KSNDZV413sBvA9QP1dkL85KvZSnSIucujOusAvFX0l8vwmvFiqD/JfzScgZzVLeKa7IlFkS3Bolk/ 6ynvwT8Y7c57RAg8RwW4sgVWu23+QRSt/sBTGUkr+qKk33gfcOi8K0N/pwMSs6dMzhXF1oa0o5jtR UStomNnEYzVQDC1ZCjBrzp4YW/tRj6m9jbpw3o0aRARRZa30fciOMeSX7falF1jHgfKu2SagTHe0V BZjGtPF5BXvxwjKnKmgLNs7Y1Eo26fxyAdFoS5D9QWXJtd7AQsA0NaS7/v7WDFUjEUUueK+s379PH oubw+AJGyLrSr2k7u6VA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rofSs-0000000G64d-3WYW; Mon, 25 Mar 2024 08:13:54 +0000 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rofSo-0000000G62d-1FfZ for linux-arm-kernel@lists.infradead.org; Mon, 25 Mar 2024 08:13:52 +0000 Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a4751063318so119175966b.0 for ; Mon, 25 Mar 2024 01:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1711354427; x=1711959227; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=DhXWU9LA3T0RxQdB87MsBwVWbw2Pg7nCtgEYH/YmzjQ=; b=ecir2eYfr1BaozaREei4t2DIzqauCoNaGvshcIoTaNMIZ1jCdvZMdQ2Qmngw5i5kFz /0dOZ/WLwYVJUYbvsvAimKKVALvgGd9EDyggUcAWRYm7/02ogpJdh7kf1m4J8kAygNiS +ZSVJTGeI4CP8DGM9QdF6sHcgO+mB7u8a41T7Rk0igXxVFjCv+yT5sWq4lyKjKxGjkzl c2sB/8IT0Yw+t1N6bXoDHF8hx31X5EtqUDHj0Pb72KQXq+Pp7GrcSrsMJsOqFEV8cptz sx4H8IyGNiPQ4uwEW4Fy+X/ebYJXYegmZrjSiUIJElD1BIxlLBpU2mbgq/H+tku1p3Od X9pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711354427; x=1711959227; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DhXWU9LA3T0RxQdB87MsBwVWbw2Pg7nCtgEYH/YmzjQ=; b=bXWEZKj54rRA63NVpPD7hF3iCFRhxyuYmDxuwxYDwkcDOnsMKzJPSvXtDAUAExLbWB DC78tfWywnCWWqZsdzwSatpt3zvvG1CxEO4ekyQoGTv3Ybfla0YeURESRem4blkJLRcQ P/YYk0kK3Pzg/F69Bxxd7ORcJ11f8y9yG8DHnJrDjb2SiDltVIUzkdSdBKhkvtj/j7Mx XcYreCu9ILkS5YXztYskOBaUlqR1bNYKzTUAZ3Lbp5ajrOaWONRQ4/XhSfvPjwK48eqB /aRInGMSXYW+uY5Z5OyUbqKNLXbF8zCq2nxraE/ds7pzJx1n0DTMNpxg0LdedUNcJuVY W6sw== X-Forwarded-Encrypted: i=1; AJvYcCUpG4gaO/0MANap6MLGTp8SdVewLAIrj6zd3C7K+fXuYdt8TQBSFEqLUrseBIDFbAH/G0EfNvCfEFLNdUFLgBNvjkx92H9fWD/fDmFADTL7nTY051M= X-Gm-Message-State: AOJu0YyjXV41/XvAXDRBVTY5LO3PJ0CSXxFynARUhHi095Q6sAB5eqiX 5p6rEH1u4oTotgjoQYG75luZJhIP2MZZzOZOFH+cLY4diMRva7GCCYvERoEUoC8= X-Google-Smtp-Source: AGHT+IH9IzSEIibSx/cUg3GgoaiZtJ5TnwurXFuYszQrutP1U/mFc422e2ipla6NDZj535YIzmdoYQ== X-Received: by 2002:a17:906:1ec9:b0:a46:62c0:659f with SMTP id m9-20020a1709061ec900b00a4662c0659fmr3720752ejj.70.1711354427102; Mon, 25 Mar 2024 01:13:47 -0700 (PDT) Received: from rayden.urgonet (h-217-31-164-171.A175.priv.bahnhof.se. [217.31.164.171]) by smtp.gmail.com with ESMTPSA id dk5-20020a170907940500b00a4750b2e0f0sm1607036ejc.164.2024.03.25.01.13.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 01:13:46 -0700 (PDT) From: Jens Wiklander To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Sudeep Holla , Marc Bonnici , Olivier Deprez , Lorenzo Pieralisi , Bertrand Marquis , Jens Wiklander Subject: [PATCH v2] firmware: arm_ffa: support running as a guest in a vm Date: Mon, 25 Mar 2024 09:13:35 +0100 Message-Id: <20240325081335.2326979-1-jens.wiklander@linaro.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240325_011350_703679_A3F7F900 X-CRM114-Status: GOOD ( 23.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add support for running the driver in a guest to a hypervisor. The main difference is introducing notification pending interrupt and that FFA_NOTIFICATION_BITMAP_CREATE doesn't need to be called. The guest may need to use a notification pending interrupt instead of or in addition to the schedule receiver interrupt. FFA_FEAT_NOTIFICATION_PENDING_INT gives the interrupt the hypervisor has chosen to notify its guest of pending notifications. Signed-off-by: Jens Wiklander --- Hi, I've tested this with Xen with as hypervisor (Xen patches to be posted) and the NPI handler delivers notifications as expected using FFA_NOTIFICATION_GET. I've also tested without a hypervisor to see that the SRI handler still works as expected. Thanks, Jens v1->v2: * Adds a Notification Pending Interrupt handler to retrieve notifications using for the interrupted CPU. * The original Schedule Receiver Interrupt handler is initialized in parallel depending on whether FFA_FEAT_SCHEDULE_RECEIVER_INT is available. drivers/firmware/arm_ffa/driver.c | 117 +++++++++++++++++++++--------- 1 file changed, 82 insertions(+), 35 deletions(-) diff --git a/drivers/firmware/arm_ffa/driver.c b/drivers/firmware/arm_ffa/driver.c index f2556a8e9401..9732f0066c70 100644 --- a/drivers/firmware/arm_ffa/driver.c +++ b/drivers/firmware/arm_ffa/driver.c @@ -101,11 +101,12 @@ struct ffa_drv_info { bool bitmap_created; bool notif_enabled; unsigned int sched_recv_irq; + unsigned int notif_pend_irq; unsigned int cpuhp_state; struct ffa_pcpu_irq __percpu *irq_pcpu; struct workqueue_struct *notif_pcpu_wq; struct work_struct notif_pcpu_work; - struct work_struct irq_work; + struct work_struct sched_recv_irq_work; struct xarray partition_info; DECLARE_HASHTABLE(notifier_hash, ilog2(FFA_MAX_NOTIFICATIONS)); struct mutex notify_lock; /* lock to protect notifier hashtable */ @@ -1291,12 +1292,23 @@ static void ffa_partitions_cleanup(void) #define FFA_FEAT_SCHEDULE_RECEIVER_INT (2) #define FFA_FEAT_MANAGED_EXIT_INT (3) -static irqreturn_t irq_handler(int irq, void *irq_data) +static irqreturn_t sched_recv_irq_handler(int irq, void *irq_data) { struct ffa_pcpu_irq *pcpu = irq_data; struct ffa_drv_info *info = pcpu->info; - queue_work(info->notif_pcpu_wq, &info->irq_work); + queue_work(info->notif_pcpu_wq, &info->sched_recv_irq_work); + + return IRQ_HANDLED; +} + +static irqreturn_t notif_pend_irq_handler(int irq, void *irq_data) +{ + struct ffa_pcpu_irq *pcpu = irq_data; + struct ffa_drv_info *info = pcpu->info; + + queue_work_on(smp_processor_id(), info->notif_pcpu_wq, + &info->notif_pcpu_work); return IRQ_HANDLED; } @@ -1306,15 +1318,16 @@ static void ffa_sched_recv_irq_work_fn(struct work_struct *work) ffa_notification_info_get(); } -static int ffa_sched_recv_irq_map(void) +static int ffa_irq_map(u32 id) { - int ret, irq, sr_intid; + int ret, irq, intid; - /* The returned sr_intid is assumed to be SGI donated to NS world */ - ret = ffa_features(FFA_FEAT_SCHEDULE_RECEIVER_INT, 0, &sr_intid, NULL); + /* The returned intid is assumed to be SGI donated to NS world */ + ret = ffa_features(id, 0, &intid, NULL); if (ret < 0) { + pr_err("Failed to retrieve FF-A id %u interrupt\n", id); if (ret != -EOPNOTSUPP) - pr_err("Failed to retrieve scheduler Rx interrupt\n"); + pr_err("Failed to retrieve FF-A id %u interrupt\n", id); return ret; } @@ -1329,12 +1342,12 @@ static int ffa_sched_recv_irq_map(void) oirq.np = gic; oirq.args_count = 1; - oirq.args[0] = sr_intid; + oirq.args[0] = intid; irq = irq_create_of_mapping(&oirq); of_node_put(gic); #ifdef CONFIG_ACPI } else { - irq = acpi_register_gsi(NULL, sr_intid, ACPI_EDGE_SENSITIVE, + irq = acpi_register_gsi(NULL, intid, ACPI_EDGE_SENSITIVE, ACPI_ACTIVE_HIGH); #endif } @@ -1355,15 +1368,29 @@ static void ffa_sched_recv_irq_unmap(void) } } +static void ffa_notif_pend_irq_unmap(void) +{ + if (drv_info->notif_pend_irq) { + irq_dispose_mapping(drv_info->notif_pend_irq); + drv_info->notif_pend_irq = 0; + } +} + static int ffa_cpuhp_pcpu_irq_enable(unsigned int cpu) { - enable_percpu_irq(drv_info->sched_recv_irq, IRQ_TYPE_NONE); + if (drv_info->sched_recv_irq) + enable_percpu_irq(drv_info->sched_recv_irq, IRQ_TYPE_NONE); + if (drv_info->notif_pend_irq) + enable_percpu_irq(drv_info->notif_pend_irq, IRQ_TYPE_NONE); return 0; } static int ffa_cpuhp_pcpu_irq_disable(unsigned int cpu) { - disable_percpu_irq(drv_info->sched_recv_irq); + if (drv_info->sched_recv_irq) + disable_percpu_irq(drv_info->sched_recv_irq); + if (drv_info->notif_pend_irq) + disable_percpu_irq(drv_info->notif_pend_irq); return 0; } @@ -1382,13 +1409,16 @@ static void ffa_uninit_pcpu_irq(void) if (drv_info->sched_recv_irq) free_percpu_irq(drv_info->sched_recv_irq, drv_info->irq_pcpu); + if (drv_info->notif_pend_irq) + free_percpu_irq(drv_info->notif_pend_irq, drv_info->irq_pcpu); + if (drv_info->irq_pcpu) { free_percpu(drv_info->irq_pcpu); drv_info->irq_pcpu = NULL; } } -static int ffa_init_pcpu_irq(unsigned int irq) +static int ffa_init_pcpu_irq(void) { struct ffa_pcpu_irq __percpu *irq_pcpu; int ret, cpu; @@ -1402,13 +1432,29 @@ static int ffa_init_pcpu_irq(unsigned int irq) drv_info->irq_pcpu = irq_pcpu; - ret = request_percpu_irq(irq, irq_handler, "ARM-FFA", irq_pcpu); - if (ret) { - pr_err("Error registering notification IRQ %d: %d\n", irq, ret); - return ret; + if (drv_info->sched_recv_irq) { + ret = request_percpu_irq(drv_info->sched_recv_irq, + sched_recv_irq_handler, + "ARM-FFA-SR", irq_pcpu); + if (ret) { + pr_err("Error registering schedule receiver nIRQ %d: %d\n", + drv_info->sched_recv_irq, ret); + return ret; + } + } + + if (drv_info->notif_pend_irq) { + ret = request_percpu_irq(drv_info->notif_pend_irq, + notif_pend_irq_handler, + "ARM-FFA-NP", irq_pcpu); + if (ret) { + pr_err("Error registering notification pendig nIRQ %d: %d\n", + drv_info->notif_pend_irq, ret); + return ret; + } } - INIT_WORK(&drv_info->irq_work, ffa_sched_recv_irq_work_fn); + INIT_WORK(&drv_info->sched_recv_irq_work, ffa_sched_recv_irq_work_fn); INIT_WORK(&drv_info->notif_pcpu_work, notif_pcpu_irq_work_fn); drv_info->notif_pcpu_wq = create_workqueue("ffa_pcpu_irq_notification"); if (!drv_info->notif_pcpu_wq) @@ -1429,6 +1475,7 @@ static void ffa_notifications_cleanup(void) { ffa_uninit_pcpu_irq(); ffa_sched_recv_irq_unmap(); + ffa_notif_pend_irq_unmap(); if (drv_info->bitmap_created) { ffa_notification_bitmap_destroy(); @@ -1439,30 +1486,30 @@ static void ffa_notifications_cleanup(void) static void ffa_notifications_setup(void) { - int ret, irq; + int ret; ret = ffa_features(FFA_NOTIFICATION_BITMAP_CREATE, 0, NULL, NULL); - if (ret) { - pr_info("Notifications not supported, continuing with it ..\n"); - return; + if (!ret) { + ret = ffa_notification_bitmap_create(); + if (ret) { + pr_info("Notification bitmap create error %d\n", ret); + return; + } + drv_info->bitmap_created = true; } - ret = ffa_notification_bitmap_create(); - if (ret) { - pr_info("Notification bitmap create error %d\n", ret); - return; - } - drv_info->bitmap_created = true; + ret = ffa_irq_map(FFA_FEAT_SCHEDULE_RECEIVER_INT); + if (ret > 0) + drv_info->sched_recv_irq = ret; - irq = ffa_sched_recv_irq_map(); - if (irq <= 0) { - ret = irq; - goto cleanup; - } + ret = ffa_irq_map(FFA_FEAT_NOTIFICATION_PENDING_INT); + if (ret > 0) + drv_info->notif_pend_irq = ret; - drv_info->sched_recv_irq = irq; + if (!drv_info->sched_recv_irq && !drv_info->notif_pend_irq) + goto cleanup; - ret = ffa_init_pcpu_irq(irq); + ret = ffa_init_pcpu_irq(); if (ret) goto cleanup;