From patchwork Tue May 2 13:06:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 9708035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3D64D6021C for ; Tue, 2 May 2017 13:32:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C8E62841C for ; Tue, 2 May 2017 13:32:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D92528449; Tue, 2 May 2017 13:32:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7905A2841C for ; Tue, 2 May 2017 13:32:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=V2aSi6xjJBChLJ5uZyxph2lHynOipl3NZoohloZuJ5o=; b=GPMGIS1y3PhIBbr9d7v5SznLQh p5Zk5UBo6HrP4+/nOGwEtOd+vWe+T3Q20IR6J2q8tOnZusMdndaX4eXIt1zBc0EzXpKZqu4o8AFVi b27Kn93jFdYpSgHO6iAhjaUVVm4aAv2ECklTe7KoxwsH7g273LvY9Zh5R+bnZYWO5pX7+e9nBsMzl 8WQEwQ+ZRVN5i9xVKh3yxiHn61Fes1ZnJb0C7La4+5N0n+Nun+OdMpO0cfQlMxf77v9IbT1hxs8wA 9ZhUlZhL7zpcyaTJR9kniVrpyPbjFr5iTQkUl1BVo3MFqy+SkHl1EYVYdSlwXLOufq8fK2kciLs3V /Lv7j2Wg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1d5XvC-0001AJ-SJ; Tue, 02 May 2017 13:32:54 +0000 Received: from merlin.infradead.org ([2001:4978:20e::2]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1d5Xv4-00011z-Fu for linux-arm-kernel@bombadil.infradead.org; Tue, 02 May 2017 13:32:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=P4Dp4sNq13Nyy9BbavyypcsWJwxztYWKPCE+l/V1Hx8=; b=PQF072n2gb+ixzL2Xh1S1wk6g EAfWlKAJUJYjD80ik7guDsGFj6b48km1p9k0WXvFNBFPPmZeGV3cksEazhmAkdtHjuc+9F6Cc4Qi3 QWaxSzWIwoHDk41TTudw6HsYQ0ukTnZme2pBPGAFyxf9cXsqr13rlXPFx5Szl37+i0Fo5TaZBx2x3 9UH1U4mDwwAnILzeCq25jI1EA22PveXFJuIW/QpVdq7zG4U99nCrVJ85KFRVSAHTKtJfC0tB0SRgF mUx8pnK9/Z27cMziN97gMK8bUQQLcFJkrj9DFvLy67s1bwcIv8TtH21Jv6284Q0dHDM2laK9s95sG 38lE5SgNg==; Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]) by merlin.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1d5XXP-0006Le-Cc for linux-arm-kernel@lists.infradead.org; Tue, 02 May 2017 13:08:20 +0000 Received: by mail-pf0-x244.google.com with SMTP id g23so33094095pfj.1 for ; Tue, 02 May 2017 06:07:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=P4Dp4sNq13Nyy9BbavyypcsWJwxztYWKPCE+l/V1Hx8=; b=tka+2kZhGALZeZ1YlSfRqyfFDl1d5QcErrFD5WJ+lCjLz2+o+7N3ITOTuwF3TM0PFS ebIaUFdT7R2wbQjtFZFCTfapuYUg46zA1JieDVnvFZTDbQ5o+cvDWvw+ZOFskKenOyiG aLSHlFJ+kLBrOv/J2Aj5ZTlggOXmYPSvTTtBCtaBGewnI6lZpvFj+ckdy96FTLflyYO1 zMrX192NZHo4ChwHYjHr16Eekz3Y0KT8kFBAymhwQPX4JMzaAwOLQiFGakp3FF2AaqN3 HsLhL8xWC/ybaHP4RccwndEPvzM8POW7fuhKrISjn3MBxCc6y0SGitNpso4uR/NGfRRe AA2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=P4Dp4sNq13Nyy9BbavyypcsWJwxztYWKPCE+l/V1Hx8=; b=bWD5qlp/YWQHFJT3A29k9I6psj4tSRqeHbVDCXOr/o8s98OvlT+a1pvY3gy/ndPPS7 T/Qx7Hie4KecZmD0k0QIxPeeOiyTh6idfESsQhsRepMH+p+/AGmYZvPPiuUxamPRpsE/ EkWBC06VXAQeQ33ba0sn46Lyadmhg7e3FuHkGQljMHzzJqpyQehCLZF3+UW3dYt7VbRY oK0FlPyON0GX6Q3qg5EtxJyEfz2Fjx0U0wGNZLV1+vntF0E3++do85jiEZioESN/bfFs 1owoZai8qmAk2vqj68tVStjLaddPnRfx0G6tAo2TPB54Zj37jLHPtHsLhc/k+0Ay/3Hh BXIw== X-Gm-Message-State: AN3rC/6wyi8YuppudZEbqU5ehc5lNSKtnZ7oQL1gTA/s7EKWmfPiHL0A GzDONkyAfzBLpg== X-Received: by 10.99.111.1 with SMTP id k1mr32800089pgc.194.1493730475700; Tue, 02 May 2017 06:07:55 -0700 (PDT) Received: from machine421.in.caveonetworks.com ([14.140.2.178]) by smtp.googlemail.com with ESMTPSA id k198sm24610689pga.54.2017.05.02.06.07.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 02 May 2017 06:07:54 -0700 (PDT) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Subject: [PATCH 8/9] net: thunderx: Support for XDP header adjustment Date: Tue, 2 May 2017 18:36:57 +0530 Message-Id: <1493730418-24606-9-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1493730418-24606-1-git-send-email-sunil.kovvuri@gmail.com> References: <1493730418-24606-1-git-send-email-sunil.kovvuri@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170502_090819_499366_A7DBF85F X-CRM114-Status: GOOD ( 19.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sunil Goutham , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sunil Goutham When in XDP mode reserve XDP_PACKET_HEADROOM bytes at the start of receive buffer for XDP program to modify headers and adjust packet start. Additional code changes done to handle such packets. Signed-off-by: Sunil Goutham --- drivers/net/ethernet/cavium/thunder/nicvf_main.c | 63 ++++++++++++++++------ drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 9 +++- 2 files changed, 55 insertions(+), 17 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index bb13dee..d6477af 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -502,13 +502,15 @@ static int nicvf_init_resources(struct nicvf *nic) } static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, - struct cqe_rx_t *cqe_rx, struct snd_queue *sq) + struct cqe_rx_t *cqe_rx, struct snd_queue *sq, + struct sk_buff **skb) { struct xdp_buff xdp; struct page *page; u32 action; - u16 len; + u16 len, offset = 0; u64 dma_addr, cpu_addr; + void *orig_data; /* Retrieve packet buffer's DMA address and length */ len = *((u16 *)((void *)cqe_rx + (3 * sizeof(u64)))); @@ -517,17 +519,47 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, cpu_addr = nicvf_iova_to_phys(nic, dma_addr); if (!cpu_addr) return false; + cpu_addr = (u64)phys_to_virt(cpu_addr); + page = virt_to_page((void *)cpu_addr); - xdp.data = phys_to_virt(cpu_addr); + xdp.data_hard_start = page_address(page); + xdp.data = (void *)cpu_addr; xdp.data_end = xdp.data + len; + orig_data = xdp.data; rcu_read_lock(); action = bpf_prog_run_xdp(prog, &xdp); rcu_read_unlock(); + /* Check if XDP program has changed headers */ + if (orig_data != xdp.data) { + len = xdp.data_end - xdp.data; + offset = orig_data - xdp.data; + dma_addr -= offset; + } + switch (action) { case XDP_PASS: - /* Pass on packet to network stack */ + /* Check if it's a recycled page, if not + * unmap the DMA mapping. + * + * Recycled page holds an extra reference. + */ + if (page_ref_count(page) == 1) { + dma_addr &= PAGE_MASK; + dma_unmap_page_attrs(&nic->pdev->dev, dma_addr, + RCV_FRAG_LEN + XDP_PACKET_HEADROOM, + DMA_FROM_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC); + } + + /* Build SKB and pass on packet to network stack */ + *skb = build_skb(xdp.data, + RCV_FRAG_LEN - cqe_rx->align_pad + offset); + if (!*skb) + put_page(page); + else + skb_put(*skb, len); return false; case XDP_TX: nicvf_xdp_sq_append_pkt(nic, sq, (u64)xdp.data, dma_addr, len); @@ -537,7 +569,6 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, case XDP_ABORTED: trace_xdp_exception(nic->netdev, prog, action); case XDP_DROP: - page = virt_to_page(xdp.data); /* Check if it's a recycled page, if not * unmap the DMA mapping. * @@ -546,7 +577,8 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, if (page_ref_count(page) == 1) { dma_addr &= PAGE_MASK; dma_unmap_page_attrs(&nic->pdev->dev, dma_addr, - RCV_FRAG_LEN, DMA_FROM_DEVICE, + RCV_FRAG_LEN + XDP_PACKET_HEADROOM, + DMA_FROM_DEVICE, DMA_ATTR_SKIP_CPU_SYNC); } put_page(page); @@ -654,7 +686,7 @@ static void nicvf_rcv_pkt_handler(struct net_device *netdev, struct napi_struct *napi, struct cqe_rx_t *cqe_rx, struct snd_queue *sq) { - struct sk_buff *skb; + struct sk_buff *skb = NULL; struct nicvf *nic = netdev_priv(netdev); struct nicvf *snic = nic; int err = 0; @@ -676,15 +708,17 @@ static void nicvf_rcv_pkt_handler(struct net_device *netdev, } /* For XDP, ignore pkts spanning multiple pages */ - if (nic->xdp_prog && (cqe_rx->rb_cnt == 1)) - if (nicvf_xdp_rx(snic, nic->xdp_prog, cqe_rx, sq)) + if (nic->xdp_prog && (cqe_rx->rb_cnt == 1)) { + /* Packet consumed by XDP */ + if (nicvf_xdp_rx(snic, nic->xdp_prog, cqe_rx, sq, &skb)) return; + } else { + skb = nicvf_get_rcv_skb(snic, cqe_rx, + nic->xdp_prog ? true : false); + } - skb = nicvf_get_rcv_skb(snic, cqe_rx, nic->xdp_prog ? true : false); - if (!skb) { - netdev_dbg(nic->netdev, "Packet not received\n"); + if (!skb) return; - } if (netif_msg_pktdata(nic)) { netdev_info(nic->netdev, "%s: skb 0x%p, len=%d\n", netdev->name, @@ -1672,9 +1706,6 @@ static int nicvf_xdp_setup(struct nicvf *nic, struct bpf_prog *prog) return -EOPNOTSUPP; } - if (prog && prog->xdp_adjust_head) - return -EOPNOTSUPP; - /* ALL SQs attached to CQs i.e same as RQs, are treated as * XDP Tx queues and more Tx queues are allocated for * network stack to send pkts out. diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c index ec234b6..43428ce 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c @@ -164,6 +164,11 @@ static inline int nicvf_alloc_rcv_buffer(struct nicvf *nic, struct rbdr *rbdr, } nic->rb_page_offset = 0; + + /* Reserve space for header modifications by BPF program */ + if (rbdr->is_xdp) + buf_len += XDP_PACKET_HEADROOM; + /* Check if it's recycled */ if (pgcache) nic->rb_page = pgcache->page; @@ -183,7 +188,7 @@ static inline int nicvf_alloc_rcv_buffer(struct nicvf *nic, struct rbdr *rbdr, return -ENOMEM; } if (pgcache) - pgcache->dma_addr = *rbuf; + pgcache->dma_addr = *rbuf + XDP_PACKET_HEADROOM; nic->rb_page_offset += buf_len; } @@ -1575,6 +1580,8 @@ static void nicvf_unmap_rcv_buffer(struct nicvf *nic, u64 dma_addr, */ if (page_ref_count(page) != 1) return; + + len += XDP_PACKET_HEADROOM; /* Receive buffers in XDP mode are mapped from page start */ dma_addr &= PAGE_MASK; }