From patchwork Mon Feb 8 12:07:49 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 8249151 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id A2F2ABEEE5 for ; Mon, 8 Feb 2016 12:08:00 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id BC637202FF for ; Mon, 8 Feb 2016 12:07:59 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D4EE6202EC for ; Mon, 8 Feb 2016 12:07:58 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aSkaF-00007Y-9C; Mon, 08 Feb 2016 12:06:23 +0000 Received: from mail-pa0-x242.google.com ([2607:f8b0:400e:c03::242]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1aSkZK-0006er-8S for linux-arm-kernel@lists.infradead.org; Mon, 08 Feb 2016 12:05:29 +0000 Received: by mail-pa0-x242.google.com with SMTP id yy13so5630514pab.1 for ; Mon, 08 Feb 2016 04:05:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gTfc5I9fdK4fhZ9EVBMPTMy4VXfVoKy+8SuyA+Azr6k=; b=CV9Vf6idh7syYxepfP9Ty4zUgEBF5eUMUprV/jGktYZzjeuDPuTzL75wD3CV2BHm5M AJ253inR5Jzq/pQyBbL7/xBpDfDrip5vZxTUPEp+AnO2Gy8cX2ILcrjzBKrqv5YOscIQ q+hGCMYwvOQW9tN6MPAiT8xBItgU6aRpg7VvzJoaZaDIwjAhD65E4bAMMEX/+txE/ynw JrPz6sGEdQY35joWZU9bVCxYYWRZPtyMdDIGa/qCODNQITmLdWlUBSDgXw7Auzy50cAW EIfvSOntzFtGr245VMXRAzw7z38Mqc8U8KxfJIj5KikEFSEnv2FDILC12oK5iU1483LA C0QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gTfc5I9fdK4fhZ9EVBMPTMy4VXfVoKy+8SuyA+Azr6k=; b=e4sJ4BYAOD3aPgDB+TABsXX3UJiXgXAQyVpgjsos/onVqcCwk9iuGGCReE6qQ8XRr7 wF1+YmlUegV0auHe7qcZUoiUNusd7BcsawzwB/p8/tBKRUTMjmirIRQNKzCaLfZv96kY gXkp1HnSx1nyG1sTc2Jm8Naa8JEl8yix3EnLM1GJz1qD424ifU2IxNiadJtCdJbl1gKp I3Vi7PuPL+jHmGHt7Kc8FrEhO20HGrWdumNgKmIAPUSy/ueVg9Q7W4WJFS1HqOliqM8Z qCf4nlU/ZKMUJpYzJNvHaZN4DrB+SB04hHQd72V2w2ogiqw4+r2GWCqAKqH8hnH+Vhx3 S0Vw== X-Gm-Message-State: AG10YOQtL1axcjGVQSV1CsAgsNJ67wNHJfO7eYqNqme6l+ARXF5i0/APCeFQ3HZpzqiNsA== X-Received: by 10.67.2.10 with SMTP id bk10mr40983470pad.26.1454933105451; Mon, 08 Feb 2016 04:05:05 -0800 (PST) Received: from localhost.localdomain ([14.140.2.178]) by smtp.googlemail.com with ESMTPSA id 17sm43082945pfq.51.2016.02.08.04.05.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Feb 2016 04:05:04 -0800 (PST) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Subject: [PATCH 3/6] net: thunderx: Assign affinity hints to vf's interrupts Date: Mon, 8 Feb 2016 17:37:49 +0530 Message-Id: <1454933272-45005-4-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1454933272-45005-1-git-send-email-sunil.kovvuri@gmail.com> References: <1454933272-45005-1-git-send-email-sunil.kovvuri@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160208_040526_675920_40A21977 X-CRM114-Status: GOOD ( 15.91 ) X-Spam-Score: -2.7 (--) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: sgoutham@cavium.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, robert.richter@caviumnetworks.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sunil Goutham This affinity hint can be used by user space irqbalance tool to set preferred CPU mask for irqs registered by this VF. Irqbalance needs to be in 'exact' mode to set irq affinity same as indicated by affinity hint. Signed-off-by: Sunil Goutham --- drivers/net/ethernet/cavium/thunder/nic.h | 1 + drivers/net/ethernet/cavium/thunder/nicvf_main.c | 37 ++++++++++++++++++++- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nic.h b/drivers/net/ethernet/cavium/thunder/nic.h index 8af363a..00cc915 100644 --- a/drivers/net/ethernet/cavium/thunder/nic.h +++ b/drivers/net/ethernet/cavium/thunder/nic.h @@ -309,6 +309,7 @@ struct nicvf { struct msix_entry msix_entries[NIC_VF_MSIX_VECTORS]; char irq_name[NIC_VF_MSIX_VECTORS][20]; bool irq_allocated[NIC_VF_MSIX_VECTORS]; + cpumask_var_t affinity_mask[NIC_VF_MSIX_VECTORS]; /* VF <-> PF mailbox communication */ bool pf_acked; diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index c6f146c..90ce93e 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -899,6 +899,31 @@ static void nicvf_disable_msix(struct nicvf *nic) } } +static void nicvf_set_irq_affinity(struct nicvf *nic) +{ + int vec, cpu; + int irqnum; + + for (vec = 0; vec < nic->num_vec; vec++) { + if (!nic->irq_allocated[vec]) + continue; + + if (!zalloc_cpumask_var(&nic->affinity_mask[vec], GFP_KERNEL)) + return; + /* CQ interrupts */ + if (vec < NICVF_INTR_ID_SQ) + /* Leave CPU0 for RBDR and other interrupts */ + cpu = nicvf_netdev_qidx(nic, vec) + 1; + else + cpu = 0; + + cpumask_set_cpu(cpumask_local_spread(cpu, nic->node), + nic->affinity_mask[vec]); + irqnum = nic->msix_entries[vec].vector; + irq_set_affinity_hint(irqnum, nic->affinity_mask[vec]); + } +} + static int nicvf_register_interrupts(struct nicvf *nic) { int irq, ret = 0; @@ -944,8 +969,13 @@ static int nicvf_register_interrupts(struct nicvf *nic) ret = request_irq(nic->msix_entries[irq].vector, nicvf_qs_err_intr_handler, 0, nic->irq_name[irq], nic); - if (!ret) - nic->irq_allocated[irq] = true; + if (ret) + goto err; + + nic->irq_allocated[irq] = true; + + /* Set IRQ affinities */ + nicvf_set_irq_affinity(nic); err: if (ret) @@ -963,6 +993,9 @@ static void nicvf_unregister_interrupts(struct nicvf *nic) if (!nic->irq_allocated[irq]) continue; + irq_set_affinity_hint(nic->msix_entries[irq].vector, NULL); + free_cpumask_var(nic->affinity_mask[irq]); + if (irq < NICVF_INTR_ID_SQ) free_irq(nic->msix_entries[irq].vector, nic->napi[irq]); else