From patchwork Tue Apr 2 09:37:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavan Chebbi X-Patchwork-Id: 13613595 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-oa1-f43.google.com (mail-oa1-f43.google.com [209.85.160.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B7321E525 for ; Tue, 2 Apr 2024 09:35:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712050539; cv=none; b=CUOFQMfYg/TJguqHTT2Eb7Cb0RqCa87iudFqqEkpNcQBq9njVNT9yZIStBA21l9TBSuti0yMhgATJWE6J2cCd3+IOBvvjJqYH4sJ1U5QonPchwOUYbImGzzxJoIR+G7jS/ByMaGrpNJTMvi2FnqwwmS9MhN03ioxT9tX6DEchMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712050539; c=relaxed/simple; bh=FUHlpvQyN1efWvroWvswyMxrhwQ98V/kAEVp4PErJy0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=KkDIc/Be84Tve+qezLlt4VkHgmureLB5fZ39KUiFDD+thyv2LfwLkaA4HysvND4dr6Ad5zsYo6VpiXlbB2AC2cAjBb11D3a1ftJVd8uvwltDqE4tyCvzb9hMhKzsul0yiw2PcrUiRlXDAEsgleAsYAXyM0MTxqLoggF0Se5zN6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=aYBvx3th; arc=none smtp.client-ip=209.85.160.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="aYBvx3th" Received: by mail-oa1-f43.google.com with SMTP id 586e51a60fabf-22e691c754cso437041fac.2 for ; Tue, 02 Apr 2024 02:35:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1712050537; x=1712655337; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=P0WOMoWOF667ES2q5lxVkv1CFX/bJar9MFPKJsTgFpU=; b=aYBvx3th8dM7XdO/g4ZSXXVqL0ZWSYDpSM5O+gqruAWWtIBXxL7BEzShtB2SM8L5wI eZwR2SkMOroIFzTR5n2vDOwMhA74+njWbrHdpGEIWoEoHVIyUCH6A0xwE7diw3yDwCvD kfs5zscEhmvQVurk0fW/glpGZGY8FYQyS+NRw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712050537; x=1712655337; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P0WOMoWOF667ES2q5lxVkv1CFX/bJar9MFPKJsTgFpU=; b=nXIjmKMhn2j475WDy7UqTAqFtn+Ta5ZURl/slhX146L1Hm9ddSGnbY7JOGS+78NPhs /pjWS4uFjY9rvRHwSuzNGL5Ab73MK/8BLsCi/LPueWmhfAida5Psl4a0JUE5N/Lh2os+ C10rUTznRAouA2/v32ZzpRbL2ZTEHZt2jMWZFzn0oIMxMsr4vgXXcXjWpqBFtxR9s986 0cBFVeONKWcodo+iWs1ada1voY0NiUerF+tiZjr6QZOzwAwEaFHFeotzyFuCv/D0mk8h seolT81gAOOpcmZ0ExShGSjSi1i6h4u96PLOGLRZ5epmCmjzVBlhgQuJD/4zgUmyp674 DHew== X-Forwarded-Encrypted: i=1; AJvYcCVubLNmcAA3+j3qDiisMxd/yTFiv3utdLb8N0ZsHNuBhaQsK/E8rI0WSFbsy5SBMvvps7YUkJFFoZ9Uf5VQfPFUjFdZm6vy X-Gm-Message-State: AOJu0YzuUxN3wVSIFfSTXSImVwJ68ap+nota6dCL6g8kuh8NuLQWZ+eY 4hByciukxaHjtz4AhjBVAu+KTkSEuVLK6uyP1g2mHQ9TAwc1oHM3ZYtVBhL0GA== X-Google-Smtp-Source: AGHT+IGZOsSRpYaJVS9anIOzP9+MFO/W+gxqI5MxyXvSsTAz3CJauoN6TADR7rEf+587CvKHQ9X/Dw== X-Received: by 2002:a05:6870:898d:b0:22a:4ea0:d8ff with SMTP id f13-20020a056870898d00b0022a4ea0d8ffmr11957828oaq.51.1712050536817; Tue, 02 Apr 2024 02:35:36 -0700 (PDT) Received: from PC-MID-R740.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id q19-20020a62e113000000b006e5a3db5875sm9702087pfh.13.2024.04.02.02.35.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 02:35:36 -0700 (PDT) From: Pavan Chebbi To: michael.chan@broadcom.com Cc: davem@davemloft.net, edumazet@google.com, gospo@broadcom.com, kuba@kernel.org, netdev@vger.kernel.org, pabeni@redhat.com, Somnath Kotur , Pavan Chebbi Subject: [PATCH net-next v2 2/7] bnxt_en: Enable XPS by default on driver load Date: Tue, 2 Apr 2024 02:37:48 -0700 Message-Id: <20240402093753.331120-3-pavan.chebbi@broadcom.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20240402093753.331120-1-pavan.chebbi@broadcom.com> References: <20240402093753.331120-1-pavan.chebbi@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Somnath Kotur Enable XPS on default during NIC open. The choice of Tx queue is based on the CPU executing the thread that submits the Tx request. The pool of Tx queues will be spread evenly across both device-attached NUMA nodes(local) and remote NUMA nodes. Signed-off-by: Somnath Kotur Reviewed-by: Michael Chan Signed-off-by: Pavan Chebbi --- v2: Fixed formatting issues (double spaces, 80 chars) --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 46 ++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 6e24a341ad28..44b9332c147e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -11804,6 +11804,46 @@ static void bnxt_cfg_usr_fltrs(struct bnxt *bp) bnxt_cfg_one_usr_fltr(bp, usr_fltr); } +static int bnxt_set_xps_mapping(struct bnxt *bp) +{ + int numa_node = dev_to_node(&bp->pdev->dev); + unsigned int q_idx, map_idx, cpu, i; + const struct cpumask *cpu_mask_ptr; + int nr_cpus = num_online_cpus(); + cpumask_t *q_map; + int rc = 0; + + q_map = kcalloc(bp->tx_nr_rings_per_tc, sizeof(*q_map), GFP_KERNEL); + if (!q_map) + return -ENOMEM; + + /* Create CPU mask for all TX queues across MQPRIO traffic classes. + * Each TC has the same number of TX queues. The nth TX queue for each + * TC will have the same CPU mask. + */ + for (i = 0; i < nr_cpus; i++) { + map_idx = i % bp->tx_nr_rings_per_tc; + cpu = cpumask_local_spread(i, numa_node); + cpu_mask_ptr = get_cpu_mask(cpu); + cpumask_or(&q_map[map_idx], &q_map[map_idx], cpu_mask_ptr); + } + + /* Register CPU mask for each TX queue except the ones marked for XDP */ + for (q_idx = 0; q_idx < bp->dev->real_num_tx_queues; q_idx++) { + map_idx = q_idx % bp->tx_nr_rings_per_tc; + rc = netif_set_xps_queue(bp->dev, &q_map[map_idx], q_idx); + if (rc) { + netdev_warn(bp->dev, "Error setting XPS for q:%d\n", + q_idx); + break; + } + } + + kfree(q_map); + + return rc; +} + static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init) { int rc = 0; @@ -11866,8 +11906,12 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init) } } - if (irq_re_init) + if (irq_re_init) { udp_tunnel_nic_reset_ntf(bp->dev); + rc = bnxt_set_xps_mapping(bp); + if (rc) + netdev_warn(bp->dev, "failed to set xps mapping\n"); + } if (bp->tx_nr_rings_xdp < num_possible_cpus()) { if (!static_key_enabled(&bnxt_xdp_locking_key))