From patchwork Thu Jun 24 16:05:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342611 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5CD9C49EA5 for ; Thu, 24 Jun 2021 16:06:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B249B613ED for ; Thu, 24 Jun 2021 16:06:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230170AbhFXQIo (ORCPT ); Thu, 24 Jun 2021 12:08:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47949 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230296AbhFXQIk (ORCPT ); Thu, 24 Jun 2021 12:08:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ESlzoBrCtV1p5Bz5L61JWXgmG5edOzrWTpf13f6lwsQ=; b=Y+2T2xE46Swz6ibryGYKTYGEUvoWDzLyE3fGyM9K570YA4DAOnNuNyM5ogxTpazGxxKfBg shVVC7ImaSV7e5o2AnIfczT+cHEVHowVqmQbCh9+5houI1s2mrTq5yBUQTmt1JYvqxxWRX WrSBV2a6XCkM/lObgdgLUlz8V3CSmrQ= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-466-rCUnMlmIN7uY29pn-iii4g-1; Thu, 24 Jun 2021 12:06:19 -0400 X-MC-Unique: rCUnMlmIN7uY29pn-iii4g-1 Received: by mail-ej1-f71.google.com with SMTP id c13-20020a17090603cdb029049617c6be8eso2169025eja.19 for ; Thu, 24 Jun 2021 09:06:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ESlzoBrCtV1p5Bz5L61JWXgmG5edOzrWTpf13f6lwsQ=; b=JiQy0XztSpBeVhftkT8XhaHhHooNpvpSK6hm/BFNTlGv2se2a9z/Z9bUKonls5vCF/ 8Vx0cYu5/FAM7lGkruVWboll6GF79XYXi4kd3m/q/JPV+5pLPh/Blt5O3eoqsssptjOk oxFjCf7YMAU3gvbHCJkurSbZ3wGyJbSJ0icZ4NtELjJna5eHVqRJVHm9ElI15UHwmn2T AhqEUxUaq9OEpsGY7Ohc1kLBFIxbE6NJ2Ao6UAGBkJ91YsOV0Mw+AQC3bED5cG2zHyzY VPkP2bIzTf6KZTlP10DnzPFASOOHU1xx7X/R2eKqtxuU9rkPBu8v12Gno8rb6d26yg1H YxOg== X-Gm-Message-State: AOAM532k9I7PH3M3XET0Q1JX5FFBYeIs/LnTdzjfcWLXLx1zCJFzRr4R tRtuB7tCcPnu8neCJh/izu4tmSMPZlG+oOT2WpImh5WN8wTpz6YVvcj13xAzX4gXo0fL3hrqJc4 r8orMmWcLAucb X-Received: by 2002:a05:6402:510d:: with SMTP id m13mr8088949edd.179.1624550778324; Thu, 24 Jun 2021 09:06:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxldU8NMGgCkkQX3pZfpalJnCIJrXi/RaF7X+X+ZKboqWU5Yi+tdklycvF1YBs+a+ixBoc+Og== X-Received: by 2002:a05:6402:510d:: with SMTP id m13mr8088919edd.179.1624550778193; Thu, 24 Jun 2021 09:06:18 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id y10sm2043535edc.66.2021.06.24.09.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 2DB06180732; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v5 01/19] rcu: Create an unrcu_pointer() to remove __rcu from a pointer Date: Thu, 24 Jun 2021 18:05:51 +0200 Message-Id: <20210624160609.292325-2-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: "Paul E. McKenney" The xchg() and cmpxchg() functions are sometimes used to carry out RCU updates. Unfortunately, this can result in sparse warnings for both the old-value and new-value arguments, as well as for the return value. The arguments can be dealt with using RCU_INITIALIZER(): old_p = xchg(&p, RCU_INITIALIZER(new_p)); But a sparse warning still remains due to assigning the __rcu pointer returned from xchg to the (most likely) non-__rcu pointer old_p. This commit therefore provides an unrcu_pointer() macro that strips the __rcu. This macro can be used as follows: old_p = unrcu_pointer(xchg(&p, RCU_INITIALIZER(new_p))); Reported-by: Toke Høiland-Jørgensen Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- include/linux/rcupdate.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 9455476c5ba2..d7895b81264e 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -363,6 +363,20 @@ static inline void rcu_preempt_sleep_check(void) { } #define rcu_check_sparse(p, space) #endif /* #else #ifdef __CHECKER__ */ +/** + * unrcu_pointer - mark a pointer as not being RCU protected + * @p: pointer needing to lose its __rcu property + * + * Converts @p from an __rcu pointer to a __kernel pointer. + * This allows an __rcu pointer to be used with xchg() and friends. + */ +#define unrcu_pointer(p) \ +({ \ + typeof(*p) *_________p1 = (typeof(*p) *__force)(p); \ + rcu_check_sparse(p, __rcu); \ + ((typeof(*p) __force __kernel *)(_________p1)); \ +}) + #define __rcu_access_pointer(p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p) *__force)READ_ONCE(p); \ From patchwork Thu Jun 24 16:05:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342619 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3202FC49EA7 for ; Thu, 24 Jun 2021 16:06:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1BE09613EA for ; Thu, 24 Jun 2021 16:06:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230239AbhFXQIy (ORCPT ); Thu, 24 Jun 2021 12:08:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35779 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231569AbhFXQIo (ORCPT ); Thu, 24 Jun 2021 12:08:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550784; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gCA//v5U2zdeKF30MPJkPlDU8nWyKsxnTIF8PhfpTLo=; b=IQ2wdmJ9vgKKLCihvF70MJ/rJPJBUSaLe6XFbl3BTmJVauZLzyyxO6WRy0dUUWLu0cLw4c 8HiA6Sdvnl5T5DpMgB8Sm/WagAsNM0aNFKvQDjd5nPa4e6M9EI5y0fEeAfali1nR51XfQl pQyRHno43/caGKzTttvXeCnPx8T1BsM= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-529-n4SnMtH8OMitetuzDDtJ7g-1; Thu, 24 Jun 2021 12:06:22 -0400 X-MC-Unique: n4SnMtH8OMitetuzDDtJ7g-1 Received: by mail-ed1-f72.google.com with SMTP id v8-20020a0564023488b0290393873961f6so3593114edc.17 for ; Thu, 24 Jun 2021 09:06:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gCA//v5U2zdeKF30MPJkPlDU8nWyKsxnTIF8PhfpTLo=; b=Y3KsterjLR3mogz7uVu0MA4OAQExmW/7lb5uz8PMYB0wJh5AhUcy7X5tCkXW+CEo+a xJKuuFsqkw6AJOq2hApjTGinXzlj/AK57C5eL2s7XXIqdI09MqfLgHlp1rbGvB89tfMf A+jevots7/BjqSX714fRgXq31uQGcXU2sYn/60K+y/X+oAp0Xs/WNTfLIy8SrvyUI9l/ oeoJsMrcHWfwHJcdnJR5ZHdXiBwYh/fp5rFxpo/MmC5/6vgO7Y90B3gd4suUWYpW4XGs 3KWnkjfSNWzpE8pNKMTTHoQYrD8U7MtWYta/Hyct5IGo7UBI+B1Bg3Lx5tGHaIRT23zl 3PnA== X-Gm-Message-State: AOAM5307MuaSfFG90wyFtPukr41PUAWSCZAffja0SsXvTEX4pX5oEerr bYzKr6hk3POV8QO0/YO/5Vwj3AwsrRQmbAy2w3ODwi0nTiQFjvAnL0OCkrKu5PBdii3M9YQEYS+ THO0ep5z0RSwa X-Received: by 2002:a17:906:680f:: with SMTP id k15mr5958332ejr.75.1624550781494; Thu, 24 Jun 2021 09:06:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaQsXTowCVLdvWyPcdt8T/pD1yrSeJT5zej5tl3HbfAnh9ZWpoWQScDtHMX0OC5sfsqpPpLQ== X-Received: by 2002:a17:906:680f:: with SMTP id k15mr5958316ejr.75.1624550781251; Thu, 24 Jun 2021 09:06:21 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id fl21sm1448600ejc.79.2021.06.24.09.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 345FC180733; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v5 02/19] doc: Clarify and expand RCU updaters and corresponding readers Date: Thu, 24 Jun 2021 18:05:52 +0200 Message-Id: <20210624160609.292325-3-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: "Paul E. McKenney" This commit clarifies which primitives readers can use given that the corresponding updaters have made a specific choice. This commit also adds this information for the various RCU Tasks flavors. While in the area, it removes a paragraph that no longer applies in any straightforward manner. Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- Documentation/RCU/checklist.rst | 48 ++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 21 deletions(-) diff --git a/Documentation/RCU/checklist.rst b/Documentation/RCU/checklist.rst index 1030119294d0..07f6cb8f674d 100644 --- a/Documentation/RCU/checklist.rst +++ b/Documentation/RCU/checklist.rst @@ -211,27 +211,33 @@ over a rather long period of time, but improvements are always welcome! of the system, especially to real-time workloads running on the rest of the system. -7. As of v4.20, a given kernel implements only one RCU flavor, - which is RCU-sched for PREEMPTION=n and RCU-preempt for PREEMPTION=y. - If the updater uses call_rcu() or synchronize_rcu(), - then the corresponding readers may use rcu_read_lock() and - rcu_read_unlock(), rcu_read_lock_bh() and rcu_read_unlock_bh(), - or any pair of primitives that disables and re-enables preemption, - for example, rcu_read_lock_sched() and rcu_read_unlock_sched(). - If the updater uses synchronize_srcu() or call_srcu(), - then the corresponding readers must use srcu_read_lock() and - srcu_read_unlock(), and with the same srcu_struct. The rules for - the expedited primitives are the same as for their non-expedited - counterparts. Mixing things up will result in confusion and - broken kernels, and has even resulted in an exploitable security - issue. - - One exception to this rule: rcu_read_lock() and rcu_read_unlock() - may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() - in cases where local bottom halves are already known to be - disabled, for example, in irq or softirq context. Commenting - such cases is a must, of course! And the jury is still out on - whether the increased speed is worth it. +7. As of v4.20, a given kernel implements only one RCU flavor, which + is RCU-sched for PREEMPTION=n and RCU-preempt for PREEMPTION=y. + If the updater uses call_rcu() or synchronize_rcu(), then + the corresponding readers may use: (1) rcu_read_lock() and + rcu_read_unlock(), (2) any pair of primitives that disables + and re-enables softirq, for example, rcu_read_lock_bh() and + rcu_read_unlock_bh(), or (3) any pair of primitives that disables + and re-enables preemption, for example, rcu_read_lock_sched() and + rcu_read_unlock_sched(). If the updater uses synchronize_srcu() + or call_srcu(), then the corresponding readers must use + srcu_read_lock() and srcu_read_unlock(), and with the same + srcu_struct. The rules for the expedited RCU grace-period-wait + primitives are the same as for their non-expedited counterparts. + + If the updater uses call_rcu_tasks() or synchronize_rcu_tasks(), + then the readers must refrain from executing voluntary + context switches, that is, from blocking. If the updater uses + call_rcu_tasks_trace() or synchronize_rcu_tasks_trace(), then + the corresponding readers must use rcu_read_lock_trace() and + rcu_read_unlock_trace(). If an updater uses call_rcu_tasks_rude() + or synchronize_rcu_tasks_rude(), then the corresponding readers + must use anything that disables interrupts. + + Mixing things up will result in confusion and broken kernels, and + has even resulted in an exploitable security issue. Therefore, + when using non-obvious pairs of primitives, commenting is of + course a must. 8. Although synchronize_rcu() is slower than is call_rcu(), it usually results in simpler code. So, unless update performance is From patchwork Thu Jun 24 16:05:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342599 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4BD4C49EA5 for ; Thu, 24 Jun 2021 16:06:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B5D85613EC for ; Thu, 24 Jun 2021 16:06:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229525AbhFXQIe (ORCPT ); Thu, 24 Jun 2021 12:08:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:40173 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229464AbhFXQIe (ORCPT ); Thu, 24 Jun 2021 12:08:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LUCuBp+kMAaiYhpJP43bLBOzgEXgjURSXqNOTfyeabU=; b=JmrPqsFgURE259U5zE8lU1miRHz0EmEWmW4zXP4xtxUJIMU2dzEbjYNQIa+3cyTWIghH7g luCIQYIqFLH5VjfKJ459G8BsnmLSiRKOuwAEEneMnzC8TuHZWeIfncr9YUzbmAu2kCcJ97 M89X8KfrgxfGVOWO0QIp+FSgiQi86Y0= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-342-T_WI8_d9N4-aseZLq_fXqQ-1; Thu, 24 Jun 2021 12:06:13 -0400 X-MC-Unique: T_WI8_d9N4-aseZLq_fXqQ-1 Received: by mail-ed1-f71.google.com with SMTP id m4-20020a0564024304b0290394d27742e4so3580132edc.10 for ; Thu, 24 Jun 2021 09:06:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LUCuBp+kMAaiYhpJP43bLBOzgEXgjURSXqNOTfyeabU=; b=iUrP6ynpCMey6/+dC6r6Q9fs8PH614JSsGS9fuXCeG648IzYNauEnUsKGNhWoOnSeN q8sBhvQLV9sh417YV7pRLe3YMi3R3s6hl3f4ktGRS1qTMPE9BUSEVCLVkWZf/0LcF7v1 cX3dCVBhfEtVbzpC80FFJjp+3ABPQGRU3vVSzt99EuBfuzEfyhhX0FJ0HpToPKa+x/JR ilmy7vsFQnoW+Zs6MY+yfDjuPeMihpBgEJYcuDFJ6o7sqIKyRyzFTCUy56SdbF2szfvQ FPrB2myceGPh06aQf1AItWs1Ho716V45gYA7gVjCfHAHaYmULBltvuh/qt5h/Gb01E4N uUkg== X-Gm-Message-State: AOAM530kbbVGcsrvn6grv7j04jT2UTgBq7jh6qefE9IDSH9F/FicqHuO up/DYf5F+3cRC9Ys9iNSmBSxZZTnOl99cF74e6nPhz1YE6JxwBSrQpI4LliP74rRdbEpZzc+AO0 bVhiRztZMhV9d X-Received: by 2002:a17:907:628a:: with SMTP id nd10mr6096706ejc.326.1624550771948; Thu, 24 Jun 2021 09:06:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz55VnSDUxxZZ06Nn0vmqQ6jiFhvp9YY16kWSNeRKwWhkFZsY6OEmU/ZAjLIAB99qnGX5fy9w== X-Received: by 2002:a17:907:628a:: with SMTP id nd10mr6096659ejc.326.1624550771402; Thu, 24 Jun 2021 09:06:11 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id ci3sm1445516ejc.0.2021.06.24.09.06.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:10 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 3FAA8180734; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v5 03/19] doc: Give XDP as example of non-obvious RCU reader/updater pairing Date: Thu, 24 Jun 2021 18:05:53 +0200 Message-Id: <20210624160609.292325-4-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit gives an example of non-obvious RCU reader/updater pairing in the guise of the XDP feature in networking, which calls BPF programs from network-driver NAPI (softirq) context. Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- Documentation/RCU/checklist.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/Documentation/RCU/checklist.rst b/Documentation/RCU/checklist.rst index 07f6cb8f674d..01cc21f17f7b 100644 --- a/Documentation/RCU/checklist.rst +++ b/Documentation/RCU/checklist.rst @@ -236,8 +236,15 @@ over a rather long period of time, but improvements are always welcome! Mixing things up will result in confusion and broken kernels, and has even resulted in an exploitable security issue. Therefore, - when using non-obvious pairs of primitives, commenting is of - course a must. + when using non-obvious pairs of primitives, commenting is + of course a must. One example of non-obvious pairing is + the XDP feature in networking, which calls BPF programs from + network-driver NAPI (softirq) context. BPF relies heavily on RCU + protection for its data structures, but because the BPF program + invocation happens entirely within a single local_bh_disable() + section in a NAPI poll cycle, this usage is safe. The reason + that this usage is safe is that readers can use anything that + disables BH when updaters use call_rcu() or synchronize_rcu(). 8. Although synchronize_rcu() is slower than is call_rcu(), it usually results in simpler code. So, unless update performance is From patchwork Thu Jun 24 16:05:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342601 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14AE2C49EA6 for ; Thu, 24 Jun 2021 16:06:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EDEE5613F0 for ; Thu, 24 Jun 2021 16:06:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229721AbhFXQIg (ORCPT ); Thu, 24 Jun 2021 12:08:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50356 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229772AbhFXQIf (ORCPT ); Thu, 24 Jun 2021 12:08:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rcNRjPgryjNmiMXDsjGjnEaUR6gkIAj0Q9ptYwP/G2w=; b=NF4uQ8pHBuDFLKW2Sw8qu98XSSpvwewG0ZBXQXyaSN/+qFujtapLc/QA6mt1O1REU6nJ8p soqwgI6vSDHGMd3/p1UkwxqxU10jGnpj7km4l7IcjpWN8fIYCuSh3+JKcwCxJG/KctGu24 m/dD9GtwA4FUvY6F4Lpisw0DNEfQjPA= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-83-IvfX3pAnP2KR7PIJywiaLQ-1; Thu, 24 Jun 2021 12:06:13 -0400 X-MC-Unique: IvfX3pAnP2KR7PIJywiaLQ-1 Received: by mail-ed1-f71.google.com with SMTP id i19-20020a05640200d3b02903948b71f25cso3599017edu.4 for ; Thu, 24 Jun 2021 09:06:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rcNRjPgryjNmiMXDsjGjnEaUR6gkIAj0Q9ptYwP/G2w=; b=r1AN3BxcLfMqkTqYDfkWvwi+6S32sGlXzRzrTkrrPEEC+wUyvsKPt7YBRh3q4zaAQJ KMdy+OFIF91U4kVdckH2cv/zSC9zrbm7lccR9a7oNlEZo8CqGWaycs6YRf11gajEp4HU BlYpvNnYq33dQj11PchwmCXvbnzKL7Wyi7aKuz1njfRQT1m7dQacHHYRRRoIqrHvY8ml d2xfpMAKO4bwpQtDvvf8cpazOufQPNaf0zzedhzM8dXHww9Fy+zEQsELAnJvV+AbXgKZ JHR2Lom/6ql8xRXhbpGsI/mKcPfuyVFVcaqrwl/fMPDW5wGAx9rj8Jr7eaIfTRuiqLyi 4MSA== X-Gm-Message-State: AOAM533JmFw1LXKWCWBlRPtqoihTk241lb4tRSr2DC6gZdo7DEKy5cQm BIAmMb+lDL51sKvcKwdkFgrhhFySHhDBjLVdpp1P2F0Y2BgsFcVZllLJV5dY45H1A1FAShhncWx qJKWmi4U8FpDI X-Received: by 2002:a17:906:70cf:: with SMTP id g15mr5992921ejk.366.1624550772344; Thu, 24 Jun 2021 09:06:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwXLWPCYjtJs8PWczQKfLhdebEoVn5ofDAARrFjqLwV5oNXsvuVt1jSqNQQLcPjdFiQD8422A== X-Received: by 2002:a17:906:70cf:: with SMTP id g15mr5992875ejk.366.1624550771974; Thu, 24 Jun 2021 09:06:11 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id dn7sm2136513edb.29.2021.06.24.09.06.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:10 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 44BDF180735; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v5 04/19] bpf: allow RCU-protected lookups to happen from bh context Date: Thu, 24 Jun 2021 18:05:54 +0200 Message-Id: <20210624160609.292325-5-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net XDP programs are called from a NAPI poll context, which means the RCU reference liveness is ensured by local_bh_disable(). Add rcu_read_lock_bh_held() as a condition to the RCU checks for map lookups so lockdep understands that the dereferences are safe from inside *either* an rcu_read_lock() section *or* a local_bh_disable() section. While both bh_disabled and rcu_read_lock() provide RCU protection, they are semantically distinct, so we need both conditions to prevent lockdep complaints. This change is done in preparation for removing the redundant rcu_read_lock()s from drivers. Acked-by: Martin KaFai Lau Signed-off-by: Toke Høiland-Jørgensen --- kernel/bpf/hashtab.c | 21 ++++++++++++++------- kernel/bpf/helpers.c | 6 +++--- kernel/bpf/lpm_trie.c | 6 ++++-- 3 files changed, 21 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 6f6681b07364..72c58cc516a3 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -596,7 +596,8 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) struct htab_elem *l; u32 hash, key_size; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -989,7 +990,8 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1082,7 +1084,8 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1148,7 +1151,8 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1202,7 +1206,8 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1276,7 +1281,8 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1311,7 +1317,8 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index a2f1f15ce432..62cf00383910 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -29,7 +29,7 @@ */ BPF_CALL_2(bpf_map_lookup_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return (unsigned long) map->ops->map_lookup_elem(map, key); } @@ -45,7 +45,7 @@ const struct bpf_func_proto bpf_map_lookup_elem_proto = { BPF_CALL_4(bpf_map_update_elem, struct bpf_map *, map, void *, key, void *, value, u64, flags) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_update_elem(map, key, value, flags); } @@ -62,7 +62,7 @@ const struct bpf_func_proto bpf_map_update_elem_proto = { BPF_CALL_2(bpf_map_delete_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_delete_elem(map, key); } diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 1b7b8a6f34ee..423549d2c52e 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -232,7 +232,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) /* Start walking the trie from the root node ... */ - for (node = rcu_dereference(trie->root); node;) { + for (node = rcu_dereference_check(trie->root, rcu_read_lock_bh_held()); + node;) { unsigned int next_bit; size_t matchlen; @@ -264,7 +265,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) * traverse down. */ next_bit = extract_bit(key->data, node->prefixlen); - node = rcu_dereference(node->child[next_bit]); + node = rcu_dereference_check(node->child[next_bit], + rcu_read_lock_bh_held()); } if (!found) From patchwork Thu Jun 24 16:05:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342603 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF345C49EAB for ; Thu, 24 Jun 2021 16:06:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D524D611CE for ; Thu, 24 Jun 2021 16:06:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230110AbhFXQIh (ORCPT ); Thu, 24 Jun 2021 12:08:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43823 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229881AbhFXQIg (ORCPT ); Thu, 24 Jun 2021 12:08:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WgT4fLA1Ib9HjFjEDe93/A77/RVuezlGm/c+0GDtz9U=; b=iACFA5kCL1baprHcVe7wLvhSgt0MosxOOBwmLQKiVGw0Ozluv+pZxj6W6H7fh2ND2muUXC AHPnCId1yKvMtD8eKhAlxZieMb/FTXxehV3v0WE4MFcJukYMdQyIQs7NZRsDFs+5+cXLdm E03CXdOjcUQSdcMCSwzJNogvNUib21I= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-453-002WRYYbO5iJwWk5Gk1wsA-1; Thu, 24 Jun 2021 12:06:14 -0400 X-MC-Unique: 002WRYYbO5iJwWk5Gk1wsA-1 Received: by mail-ej1-f72.google.com with SMTP id ci22-20020a170906c356b0290492ca430d87so2175506ejb.14 for ; Thu, 24 Jun 2021 09:06:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WgT4fLA1Ib9HjFjEDe93/A77/RVuezlGm/c+0GDtz9U=; b=cpBA+bvEHmtjofVN3wns8/Bq/vpo15DKxssS+S2R73xA0AYrTLb4Ct98eejPhPY/Fp T0pL4nMd4BLndaCfAZzClNilZnpsDobR1Rnf9VrZGQuozyeXo2XzRF4+sy3wvOYY5drR 4+ZeiCWS3E4BMjos9b/X8B6xaKo9yPMrpBt/z0WbSZENVBaNw2/qqa8Ep15tdaveUhcu URSvFlTBxBOtgCWgfcATZtJibzUbqa45Vzp7t6c1HyiU58XAAG8dXaP7T9adQVNkladX o2zYV776aCGMAeSpE/H+lhSR6oBA3vFlrI1JsIwRx1k9fYTC6M6Jo1ZvAXYbpbrbt8am zBvA== X-Gm-Message-State: AOAM532ln0qEy3q8KCVOQ7B6GnlpPqgQ8H+hudBuyR4yNnuJcBrElgeU YSPZ+dHRdcdX+fSRLXZ6u+uBHgu6UaAQdIK3y/sprhN9iwhscRKbs75htiuQv4oAaYaYzRkYUX8 zItIrG87Nhocn X-Received: by 2002:a17:906:4f14:: with SMTP id t20mr5979478eju.398.1624550773274; Thu, 24 Jun 2021 09:06:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzd7KBDDZzZEK7yyDkdAfoZnQGMF2UIx+AqCScd4luY5ha5n5UL21wL3Mm0LJ+BVmKYWQ3oxA== X-Received: by 2002:a17:906:4f14:: with SMTP id t20mr5979419eju.398.1624550772754; Thu, 24 Jun 2021 09:06:12 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id p18sm1308798edu.8.2021.06.24.09.06.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:12 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 4A114180736; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v5 05/19] xdp: add proper __rcu annotations to redirect map entries Date: Thu, 24 Jun 2021 18:05:55 +0200 Message-Id: <20210624160609.292325-6-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net XDP_REDIRECT works by a three-step process: the bpf_redirect() and bpf_redirect_map() helpers will lookup the target of the redirect and store it (along with some other metadata) in a per-CPU struct bpf_redirect_info. Next, when the program returns the XDP_REDIRECT return code, the driver will call xdp_do_redirect() which will use the information thus stored to actually enqueue the frame into a bulk queue structure (that differs slightly by map type, but shares the same principle). Finally, before exiting its NAPI poll loop, the driver will call xdp_do_flush(), which will flush all the different bulk queues, thus completing the redirect. Pointers to the map entries will be kept around for this whole sequence of steps, protected by RCU. However, there is no top-level rcu_read_lock() in the core code; instead drivers add their own rcu_read_lock() around the XDP portions of the code, but somewhat inconsistently as Martin discovered[0]. However, things still work because everything happens inside a single NAPI poll sequence, which means it's between a pair of calls to local_bh_disable()/local_bh_enable(). So Paul suggested[1] that we could document this intention by using rcu_dereference_check() with rcu_read_lock_bh_held() as a second parameter, thus allowing sparse and lockdep to verify that everything is done correctly. This patch does just that: we add an __rcu annotation to the map entry pointers and remove the various comments explaining the NAPI poll assurance strewn through devmap.c in favour of a longer explanation in filter.c. The goal is to have one coherent documentation of the entire flow, and rely on the RCU annotations as a "standard" way of communicating the flow in the map code (which can additionally be understood by sparse and lockdep). The RCU annotation replacements result in a fairly straight-forward replacement where READ_ONCE() becomes rcu_dereference_check(), WRITE_ONCE() becomes rcu_assign_pointer() and xchg() and cmpxchg() gets wrapped in the proper constructs to cast the pointer back and forth between __rcu and __kernel address space (for the benefit of sparse). The one complication is that xskmap has a few constructions where double-pointers are passed back and forth; these simply all gain __rcu annotations, and only the final reference/dereference to the inner-most pointer gets changed. With this, everything can be run through sparse without eliciting complaints, and lockdep can verify correctness even without the use of rcu_read_lock() in the drivers. Subsequent patches will clean these up from the drivers. [0] https://lore.kernel.org/bpf/20210415173551.7ma4slcbqeyiba2r@kafai-mbp.dhcp.thefacebook.com/ [1] https://lore.kernel.org/bpf/20210419165837.GA975577@paulmck-ThinkPad-P17-Gen-1/ Signed-off-by: Toke Høiland-Jørgensen --- include/linux/filter.h | 8 +++---- include/net/xdp_sock.h | 2 +- kernel/bpf/cpumap.c | 13 +++++++---- kernel/bpf/devmap.c | 49 ++++++++++++++++++------------------------ net/core/filter.c | 28 ++++++++++++++++++++++++ net/xdp/xsk.c | 4 ++-- net/xdp/xsk.h | 4 ++-- net/xdp/xskmap.c | 29 ++++++++++++++----------- 8 files changed, 83 insertions(+), 54 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index 688856e0b28a..472f97074da0 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -763,11 +763,9 @@ DECLARE_BPF_DISPATCHER(xdp) static __always_inline u32 bpf_prog_run_xdp(const struct bpf_prog *prog, struct xdp_buff *xdp) { - /* Caller needs to hold rcu_read_lock() (!), otherwise program - * can be released while still running, or map elements could be - * freed early while still having concurrent users. XDP fastpath - * already takes rcu_read_lock() when fetching the program, so - * it's not necessary here anymore. + /* Driver XDP hooks are invoked within a single NAPI poll cycle and thus + * under local_bh_disable(), which provides the needed RCU protection + * for accessing map entries. */ return __BPF_PROG_RUN(prog, xdp, BPF_DISPATCHER_FUNC(xdp)); } diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index 9c0722c6d7ac..fff069d2ed1b 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -37,7 +37,7 @@ struct xdp_umem { struct xsk_map { struct bpf_map map; spinlock_t lock; /* Synchronize map updates */ - struct xdp_sock *xsk_map[]; + struct xdp_sock __rcu *xsk_map[]; }; struct xdp_sock { diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index a1a0c4e791c6..480e936c54d0 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -74,7 +74,7 @@ struct bpf_cpu_map_entry { struct bpf_cpu_map { struct bpf_map map; /* Below members specific for map type */ - struct bpf_cpu_map_entry **cpu_map; + struct bpf_cpu_map_entry __rcu **cpu_map; }; static DEFINE_PER_CPU(struct list_head, cpu_map_flush_list); @@ -469,7 +469,7 @@ static void __cpu_map_entry_replace(struct bpf_cpu_map *cmap, { struct bpf_cpu_map_entry *old_rcpu; - old_rcpu = xchg(&cmap->cpu_map[key_cpu], rcpu); + old_rcpu = unrcu_pointer(xchg(&cmap->cpu_map[key_cpu], RCU_INITIALIZER(rcpu))); if (old_rcpu) { call_rcu(&old_rcpu->rcu, __cpu_map_entry_free); INIT_WORK(&old_rcpu->kthread_stop_wq, cpu_map_kthread_stop); @@ -551,7 +551,7 @@ static void cpu_map_free(struct bpf_map *map) for (i = 0; i < cmap->map.max_entries; i++) { struct bpf_cpu_map_entry *rcpu; - rcpu = READ_ONCE(cmap->cpu_map[i]); + rcpu = rcu_dereference_raw(cmap->cpu_map[i]); if (!rcpu) continue; @@ -562,6 +562,10 @@ static void cpu_map_free(struct bpf_map *map) kfree(cmap); } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_cpu_map *cmap = container_of(map, struct bpf_cpu_map, map); @@ -570,7 +574,8 @@ static void *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - rcpu = READ_ONCE(cmap->cpu_map[key]); + rcpu = rcu_dereference_check(cmap->cpu_map[key], + rcu_read_lock_bh_held()); return rcpu; } diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 2a75e6c2d27d..2f6bd75cd682 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -73,7 +73,7 @@ struct bpf_dtab_netdev { struct bpf_dtab { struct bpf_map map; - struct bpf_dtab_netdev **netdev_map; /* DEVMAP type only */ + struct bpf_dtab_netdev __rcu **netdev_map; /* DEVMAP type only */ struct list_head list; /* these are only used for DEVMAP_HASH type maps */ @@ -226,7 +226,7 @@ static void dev_map_free(struct bpf_map *map) for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev; - dev = dtab->netdev_map[i]; + dev = rcu_dereference_raw(dtab->netdev_map[i]); if (!dev) continue; @@ -259,6 +259,10 @@ static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) return 0; } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__dev_map_hash_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_dtab *dtab = container_of(map, struct bpf_dtab, map); @@ -410,15 +414,9 @@ static void bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) trace_xdp_devmap_xmit(bq->dev_rx, dev, sent, cnt - sent, err); } -/* __dev_flush is called from xdp_do_flush() which _must_ be signaled - * from the driver before returning from its napi->poll() routine. The poll() - * routine is called either from busy_poll context or net_rx_action signaled - * from NET_RX_SOFTIRQ. Either way the poll routine must complete before the - * net device can be torn down. On devmap tear down we ensure the flush list - * is empty before completing to ensure all flush operations have completed. - * When drivers update the bpf program they may need to ensure any flush ops - * are also complete. Using synchronize_rcu or call_rcu will suffice for this - * because both wait for napi context to exit. +/* __dev_flush is called from xdp_do_flush() which _must_ be signalled from the + * driver before returning from its napi->poll() routine. See the comment above + * xdp_do_flush() in filter.c. */ void __dev_flush(void) { @@ -433,9 +431,9 @@ void __dev_flush(void) } } -/* rcu_read_lock (from syscall and BPF contexts) ensures that if a delete and/or - * update happens in parallel here a dev_put won't happen until after reading - * the ifindex. +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. */ static void *__dev_map_lookup_elem(struct bpf_map *map, u32 key) { @@ -445,12 +443,14 @@ static void *__dev_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - obj = READ_ONCE(dtab->netdev_map[key]); + obj = rcu_dereference_check(dtab->netdev_map[key], + rcu_read_lock_bh_held()); return obj; } -/* Runs under RCU-read-side, plus in softirq under NAPI protection. - * Thus, safe percpu variable access. +/* Runs in NAPI, i.e., softirq under local_bh_disable(). Thus, safe percpu + * variable access, and map elements stick around. See comment above + * xdp_do_flush() in filter.c. */ static void bq_enqueue(struct net_device *dev, struct xdp_frame *xdpf, struct net_device *dev_rx, struct bpf_prog *xdp_prog) @@ -735,14 +735,7 @@ static int dev_map_delete_elem(struct bpf_map *map, void *key) if (k >= map->max_entries) return -EINVAL; - /* Use call_rcu() here to ensure any rcu critical sections have - * completed as well as any flush operations because call_rcu - * will wait for preempt-disable region to complete, NAPI in this - * context. And additionally, the driver tear down ensures all - * soft irqs are complete before removing the net device in the - * case of dev_put equals zero. - */ - old_dev = xchg(&dtab->netdev_map[k], NULL); + old_dev = unrcu_pointer(xchg(&dtab->netdev_map[k], NULL)); if (old_dev) call_rcu(&old_dev->rcu, __dev_map_entry_free); return 0; @@ -851,7 +844,7 @@ static int __dev_map_update_elem(struct net *net, struct bpf_map *map, * Remembering the driver side flush operation will happen before the * net device is removed. */ - old_dev = xchg(&dtab->netdev_map[i], dev); + old_dev = unrcu_pointer(xchg(&dtab->netdev_map[i], RCU_INITIALIZER(dev))); if (old_dev) call_rcu(&old_dev->rcu, __dev_map_entry_free); @@ -1031,10 +1024,10 @@ static int dev_map_notification(struct notifier_block *notifier, for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev, *odev; - dev = READ_ONCE(dtab->netdev_map[i]); + dev = rcu_dereference(dtab->netdev_map[i]); if (!dev || netdev != dev->dev) continue; - odev = cmpxchg(&dtab->netdev_map[i], dev, NULL); + odev = unrcu_pointer(cmpxchg(&dtab->netdev_map[i], RCU_INITIALIZER(dev), NULL)); if (dev == odev) call_rcu(&dev->rcu, __dev_map_entry_free); diff --git a/net/core/filter.c b/net/core/filter.c index d062053994c7..d22895caa164 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -3897,6 +3897,34 @@ static const struct bpf_func_proto bpf_xdp_adjust_meta_proto = { .arg2_type = ARG_ANYTHING, }; +/* XDP_REDIRECT works by a three-step process, implemented in the functions + * below: + * + * 1. The bpf_redirect() and bpf_redirect_map() helpers will lookup the target + * of the redirect and store it (along with some other metadata) in a per-CPU + * struct bpf_redirect_info. + * + * 2. When the program returns the XDP_REDIRECT return code, the driver will + * call xdp_do_redirect() which will use the information in struct + * bpf_redirect_info to actually enqueue the frame into a map type-specific + * bulk queue structure. + * + * 3. Before exiting its NAPI poll loop, the driver will call xdp_do_flush(), + * which will flush all the different bulk queues, thus completing the + * redirect. + * + * Pointers to the map entries will be kept around for this whole sequence of + * steps, protected by RCU. However, there is no top-level rcu_read_lock() in + * the core code; instead, the RCU protection relies on everything happening + * inside a single NAPI poll sequence, which means it's between a pair of calls + * to local_bh_disable()/local_bh_enable(). + * + * The map entries are marked as __rcu and the map code makes sure to + * dereference those pointers with rcu_dereference_check() in a way that works + * for both sections that to hold an rcu_read_lock() and sections that are + * called from NAPI without a separate rcu_read_lock(). The code below does not + * use RCU annotations, but relies on those in the map code. + */ void xdp_do_flush(void) { __dev_flush(); diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index cd62d4ba87a9..996da915f520 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -749,7 +749,7 @@ static void xsk_unbind_dev(struct xdp_sock *xs) } static struct xsk_map *xsk_get_map_list_entry(struct xdp_sock *xs, - struct xdp_sock ***map_entry) + struct xdp_sock __rcu ***map_entry) { struct xsk_map *map = NULL; struct xsk_map_node *node; @@ -785,7 +785,7 @@ static void xsk_delete_from_maps(struct xdp_sock *xs) * might be updates to the map between * xsk_get_map_list_entry() and xsk_map_try_sock_delete(). */ - struct xdp_sock **map_entry = NULL; + struct xdp_sock __rcu **map_entry = NULL; struct xsk_map *map; while ((map = xsk_get_map_list_entry(xs, &map_entry))) { diff --git a/net/xdp/xsk.h b/net/xdp/xsk.h index edcf249ad1f1..a4bc4749faac 100644 --- a/net/xdp/xsk.h +++ b/net/xdp/xsk.h @@ -31,7 +31,7 @@ struct xdp_mmap_offsets_v1 { struct xsk_map_node { struct list_head node; struct xsk_map *map; - struct xdp_sock **map_entry; + struct xdp_sock __rcu **map_entry; }; static inline struct xdp_sock *xdp_sk(struct sock *sk) @@ -40,7 +40,7 @@ static inline struct xdp_sock *xdp_sk(struct sock *sk) } void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, - struct xdp_sock **map_entry); + struct xdp_sock __rcu **map_entry); void xsk_clear_pool_at_qid(struct net_device *dev, u16 queue_id); int xsk_reg_pool_at_qid(struct net_device *dev, struct xsk_buff_pool *pool, u16 queue_id); diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 9df75ea4a567..2e48d0e094d9 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -12,7 +12,7 @@ #include "xsk.h" static struct xsk_map_node *xsk_map_node_alloc(struct xsk_map *map, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { struct xsk_map_node *node; @@ -42,7 +42,7 @@ static void xsk_map_sock_add(struct xdp_sock *xs, struct xsk_map_node *node) } static void xsk_map_sock_delete(struct xdp_sock *xs, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { struct xsk_map_node *n, *tmp; @@ -124,6 +124,10 @@ static int xsk_map_gen_lookup(struct bpf_map *map, struct bpf_insn *insn_buf) return insn - insn_buf; } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__xsk_map_lookup_elem(struct bpf_map *map, u32 key) { struct xsk_map *m = container_of(map, struct xsk_map, map); @@ -131,12 +135,11 @@ static void *__xsk_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - return READ_ONCE(m->xsk_map[key]); + return rcu_dereference_check(m->xsk_map[key], rcu_read_lock_bh_held()); } static void *xsk_map_lookup_elem(struct bpf_map *map, void *key) { - WARN_ON_ONCE(!rcu_read_lock_held()); return __xsk_map_lookup_elem(map, *(u32 *)key); } @@ -149,7 +152,8 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags) { struct xsk_map *m = container_of(map, struct xsk_map, map); - struct xdp_sock *xs, *old_xs, **map_entry; + struct xdp_sock __rcu **map_entry; + struct xdp_sock *xs, *old_xs; u32 i = *(u32 *)key, fd = *(u32 *)value; struct xsk_map_node *node; struct socket *sock; @@ -179,7 +183,7 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, } spin_lock_bh(&m->lock); - old_xs = READ_ONCE(*map_entry); + old_xs = rcu_dereference_protected(*map_entry, lockdep_is_held(&m->lock)); if (old_xs == xs) { err = 0; goto out; @@ -191,7 +195,7 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, goto out; } xsk_map_sock_add(xs, node); - WRITE_ONCE(*map_entry, xs); + rcu_assign_pointer(*map_entry, xs); if (old_xs) xsk_map_sock_delete(old_xs, map_entry); spin_unlock_bh(&m->lock); @@ -208,7 +212,8 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, static int xsk_map_delete_elem(struct bpf_map *map, void *key) { struct xsk_map *m = container_of(map, struct xsk_map, map); - struct xdp_sock *old_xs, **map_entry; + struct xdp_sock __rcu **map_entry; + struct xdp_sock *old_xs; int k = *(u32 *)key; if (k >= map->max_entries) @@ -216,7 +221,7 @@ static int xsk_map_delete_elem(struct bpf_map *map, void *key) spin_lock_bh(&m->lock); map_entry = &m->xsk_map[k]; - old_xs = xchg(map_entry, NULL); + old_xs = unrcu_pointer(xchg(map_entry, NULL)); if (old_xs) xsk_map_sock_delete(old_xs, map_entry); spin_unlock_bh(&m->lock); @@ -231,11 +236,11 @@ static int xsk_map_redirect(struct bpf_map *map, u32 ifindex, u64 flags) } void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { spin_lock_bh(&map->lock); - if (READ_ONCE(*map_entry) == xs) { - WRITE_ONCE(*map_entry, NULL); + if (rcu_access_pointer(*map_entry) == xs) { + rcu_assign_pointer(*map_entry, NULL); xsk_map_sock_delete(xs, map_entry); } spin_unlock_bh(&map->lock); From patchwork Thu Jun 24 16:05:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342615 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A90A9C49EA7 for ; Thu, 24 Jun 2021 16:06:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9C9DB613FE for ; Thu, 24 Jun 2021 16:06:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231169AbhFXQIp (ORCPT ); Thu, 24 Jun 2021 12:08:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57076 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231157AbhFXQIl (ORCPT ); Thu, 24 Jun 2021 12:08:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w/RD8INVqrZhxhRYQu3p0iItEELqlKBFxEmUhyTn65o=; b=dJiOvCnF9+tbGOGQ30GfipsO1wzHsLtNI3s6L2vSNe8bF2kG6ze7dvjLRghXSMGQbjEzZI +bbbG5aj74DBUY0N26Tx8CJ5KKr8e5gsbu5fx4RttZ8a9Rov2k56Vart2Er9Tk2O5HXZUB HUzomUBrGksxVah5Le1dWoq0IPt3+yk= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-88-tsb-r6tHPdeeWnwL7E1Iiw-1; Thu, 24 Jun 2021 12:06:20 -0400 X-MC-Unique: tsb-r6tHPdeeWnwL7E1Iiw-1 Received: by mail-ej1-f72.google.com with SMTP id o12-20020a17090611ccb02904876d1b5532so2188029eja.11 for ; Thu, 24 Jun 2021 09:06:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w/RD8INVqrZhxhRYQu3p0iItEELqlKBFxEmUhyTn65o=; b=V7p/IlsrMLRSh+RIrCWVH7iFIQxfRCCxcdSHRmLjHwx9l9lRxyR/oJgmrCiTydOzQ1 C4tybdCElibwICtkN7Ky6m6YA9VvYeI/gTo+MAOL4iEmrZgdqj9mzpF37++ZiQCjR3Hu l0FyeayETiiZx+3pi7KTf5BZew+hurnSk8UJqdJBmI58j1wJy4xdTgBqw3cQfnJGCBIZ HqGXZXyTggvmmFeTtqnByFkWId4VxZigvuFgUA7DGFjiNG8aRV5zaqybqBgM+VvEmUS+ J+kFf4KXle6ZfPF1Esv3AJMO2E3GOZoofit20RD2eqQbn3JRQiCn/xnfqf/LHE1PwXvZ AMew== X-Gm-Message-State: AOAM530j4jPGmQXS8qPq5bWVwYPlWZEn1JWq89AEr5ovztcoc+9cne/O UTWf9ArUbfOithq9xExovH4tuJUmhoKdqOrECRukRifyyazN1hNVHzSRAWXEi3DqgArB+Rtpn96 jOs9gM3Wi53JP X-Received: by 2002:a05:6402:1772:: with SMTP id da18mr8150705edb.28.1624550779496; Thu, 24 Jun 2021 09:06:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyrSaG+tP51X33gJ8kk/YPPDA/aApXSH8Z4MmxGnq2m1V2HpmdBnG2zEKLird12e9BTsl8H4g== X-Received: by 2002:a05:6402:1772:: with SMTP id da18mr8150691edb.28.1624550779371; Thu, 24 Jun 2021 09:06:19 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id cb14sm2231763edb.68.2021.06.24.09.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 4EDF3180737; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= Subject: [PATCH bpf-next v5 06/19] sched: remove unneeded rcu_read_lock() around BPF program invocation Date: Thu, 24 Jun 2021 18:05:56 +0200 Message-Id: <20210624160609.292325-7-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The rcu_read_lock() call in cls_bpf and act_bpf are redundant: on the TX side, there's already a call to rcu_read_lock_bh() in __dev_queue_xmit(), and on RX there's a covering rcu_read_lock() in netif_receive_skb{,_list}_internal(). With the previous patches we also amended the lockdep checks in the map code to not require any particular RCU flavour, so we can just get rid of the rcu_read_lock()s. Signed-off-by: Toke Høiland-Jørgensen --- net/sched/act_bpf.c | 2 -- net/sched/cls_bpf.c | 3 --- 2 files changed, 5 deletions(-) diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c index e48e980c3b93..e409a0005717 100644 --- a/net/sched/act_bpf.c +++ b/net/sched/act_bpf.c @@ -43,7 +43,6 @@ static int tcf_bpf_act(struct sk_buff *skb, const struct tc_action *act, tcf_lastuse_update(&prog->tcf_tm); bstats_cpu_update(this_cpu_ptr(prog->common.cpu_bstats), skb); - rcu_read_lock(); filter = rcu_dereference(prog->filter); if (at_ingress) { __skb_push(skb, skb->mac_len); @@ -56,7 +55,6 @@ static int tcf_bpf_act(struct sk_buff *skb, const struct tc_action *act, } if (skb_sk_is_prefetched(skb) && filter_res != TC_ACT_OK) skb_orphan(skb); - rcu_read_unlock(); /* A BPF program may overwrite the default action opcode. * Similarly as in cls_bpf, if filter_res == -1 we use the diff --git a/net/sched/cls_bpf.c b/net/sched/cls_bpf.c index 6e3e63db0e01..fa739efa59f4 100644 --- a/net/sched/cls_bpf.c +++ b/net/sched/cls_bpf.c @@ -85,8 +85,6 @@ static int cls_bpf_classify(struct sk_buff *skb, const struct tcf_proto *tp, struct cls_bpf_prog *prog; int ret = -1; - /* Needed here for accessing maps. */ - rcu_read_lock(); list_for_each_entry_rcu(prog, &head->plist, link) { int filter_res; @@ -131,7 +129,6 @@ static int cls_bpf_classify(struct sk_buff *skb, const struct tcf_proto *tp, break; } - rcu_read_unlock(); return ret; } From patchwork Thu Jun 24 16:05:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342607 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B7DCC49EB7 for ; Thu, 24 Jun 2021 16:06:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED224613EC for ; Thu, 24 Jun 2021 16:06:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231527AbhFXQIn (ORCPT ); Thu, 24 Jun 2021 12:08:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27125 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230257AbhFXQIj (ORCPT ); Thu, 24 Jun 2021 12:08:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xn+g4Oy4TbzIlhU9jJ2+v158p0Us5HC5PaMOKTXC9pw=; b=fWJpM3R2VCR0xrPEDWTq1C7DNkpJDTFnk1nasqMwFrmD+JeI9uiAtFoOjwOklzdrXWi3BJ XRsZRSktWuQhd7kXTVydXuaohO7rgeBhPOvnNx0BvlXMSAyZLm0FOYMkYNRLEAVrEr8azs syEazuEmACCStusTKrmLBbjfoHjMofE= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-323-kAblAKqIM1KszIwWm9m1cQ-1; Thu, 24 Jun 2021 12:06:19 -0400 X-MC-Unique: kAblAKqIM1KszIwWm9m1cQ-1 Received: by mail-ej1-f69.google.com with SMTP id ho42-20020a1709070eaab02904a77ea3380eso2191823ejc.4 for ; Thu, 24 Jun 2021 09:06:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xn+g4Oy4TbzIlhU9jJ2+v158p0Us5HC5PaMOKTXC9pw=; b=nW5+S9j7zFuGvE/sYm4TgtxhIM7xVdpMFww3SvkZ+6eEjrj7quVTSGHkAq0S9wHqZR Jq3e53PofQSbO+XXVqojYiTuTVlV/R/LqTq9ailJ6YhCNXzADv+tFBjFmNo5z/KoWcOr Q7C/WvvO+sviEjKoQAtyfRgUoxXbT0DQ3ED5adcvEnm6Sl/bgF3GvgC3zPSTBgvS2VwJ Arh4R/mj9L2eNdZ374BvatAkWDt81ISWHX8+vYmAxShNHHnGRq5Zd6T9kXVmITHM4FIZ 1VAJclhTWqrFt8WmgUUu4BqwcxOEi6JJX5+d1cH8jM6U1mZwx6Mnefqcr6pmlRwxHGED R3Bg== X-Gm-Message-State: AOAM532Msmi5V3fJ4Z36DFb1OzUyI3NUQdHLQ26lU6t1rJLRiaZ1U9WZ iFAwHkM9hYDsl3TLidqtc3q9yxdpfDqtHn/EmRJBTMy5265LAEemg23HKlxNph8df622RD08sbh X3ZSli0DkHzYw X-Received: by 2002:a05:6402:4cb:: with SMTP id n11mr8118335edw.292.1624550776918; Thu, 24 Jun 2021 09:06:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxpnYPJAzm/4qtNBc7xfhD4QoTY11++wL13HFV4VgIRqY/z6BaZza/LEmUfZUO3YnT8+UY0ZA== X-Received: by 2002:a05:6402:4cb:: with SMTP id n11mr8118313edw.292.1624550776778; Thu, 24 Jun 2021 09:06:16 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id x17sm2267495edr.88.2021.06.24.09.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 53E8B180738; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Guy Tzalik , Saeed Bishara Subject: [PATCH bpf-next v5 07/19] ena: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:05:57 +0200 Message-Id: <20210624160609.292325-8-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The ena driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Guy Tzalik Cc: Saeed Bishara Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 3bb0e66b2c7e..44ef6b88f715 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -382,7 +382,6 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp) struct xdp_frame *xdpf; u64 *xdp_stat; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_bpf_prog); if (!xdp_prog) @@ -439,8 +438,6 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp) ena_increase_stat(xdp_stat, 1, &rx_ring->syncp); out: - rcu_read_unlock(); - return verdict; } From patchwork Thu Jun 24 16:05:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342609 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FDAAC49EA7 for ; Thu, 24 Jun 2021 16:06:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5AFF3613F2 for ; Thu, 24 Jun 2021 16:06:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231432AbhFXQIn (ORCPT ); Thu, 24 Jun 2021 12:08:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43667 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230225AbhFXQIj (ORCPT ); Thu, 24 Jun 2021 12:08:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550779; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6wgUfE3YuyYqn3UXIkFwKI8I46axO+CjBlTP2HaYNQM=; b=H+CQFTNYFlqZ91AajQI6h6a777VAXYunpekDTPsc9LMWkdjF/ryNIZJYiPskxakOvlxBer Vh+SG7hIQJjcKnEVfRWfmVhOkSG/+X7ZY7k52PPARMLsQKPmvG+kMW0jF5ALlNEnDzy0wH A5dzy7MP5ozpLxPciXBEl+kTjpZGzcg= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-312-fORYSm2mNHKoIKoKPlzUHA-1; Thu, 24 Jun 2021 12:06:17 -0400 X-MC-Unique: fORYSm2mNHKoIKoKPlzUHA-1 Received: by mail-ed1-f70.google.com with SMTP id o16-20020aa7c7d00000b02903951279f8f3so793930eds.11 for ; Thu, 24 Jun 2021 09:06:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6wgUfE3YuyYqn3UXIkFwKI8I46axO+CjBlTP2HaYNQM=; b=DbrAQuWT/BtEIrNdcV91uHWoIcXEswD5QTcsZNsisLVYk037G6CTSh5fWMAVpPy49r e+/Og2yRLCg/qg99L1aKMll4t633okkA3/yKkgEgaHcJasgWzIHavAxjy+RQAUQ5sSNu hVv/DjZO/2phdIo9DQJQC5dpfSCZxckXHeuw/dVX/h159qHPVMHe5NBNZTRKRmHQz0gf v+CEy+1xddUB3wPO+PKfZ2o4e40kqCt8OEyAibzkMOxcvIJvw3IX1p0l4/jV08YSgWuk YLEg0TNtmE0NESeaBKZ6IntZ1szEsLEsvjhnHDWQj97hh7RukYCevPztBBS1k4GDAlsB Aq1g== X-Gm-Message-State: AOAM533aYj4WKM0Xeye8nnU+/kVUrIUz3XuQNgdCHa+4oEyqrTZm6F36 C60PpzYH1UeHKEmox6UmVAirDWIOIcKGun5/vZEbYsSG8ajppgyjhu4cnSwVkXl3pkCdEkxij6p UtTr77392CfJi X-Received: by 2002:a05:6402:1001:: with SMTP id c1mr8215065edu.26.1624550776303; Thu, 24 Jun 2021 09:06:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxno4q/K3m9BqPMkDP1H6oi4QyML09kHW15PWie27ZLhrpfBPKHQyKrcZaDXyviYFPp667eYw== X-Received: by 2002:a05:6402:1001:: with SMTP id c1mr8215048edu.26.1624550776175; Thu, 24 Jun 2021 09:06:16 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id cf3sm641548edb.39.2021.06.24.09.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 5F5CC180739; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Michael Chan Subject: [PATCH bpf-next v5 08/19] bnxt: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:05:58 +0200 Message-Id: <20210624160609.292325-9-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The bnxt driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Michael Chan Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c index ec9564e584e0..bee6e091a997 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c @@ -138,9 +138,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons, xdp_prepare_buff(&xdp, *data_ptr - offset, offset, *len, false); orig_data = xdp.data; - rcu_read_lock(); act = bpf_prog_run_xdp(xdp_prog, &xdp); - rcu_read_unlock(); tx_avail = bnxt_tx_avail(bp, txr); /* If the tx ring is not full, we must not update the rx producer yet From patchwork Thu Jun 24 16:05:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342605 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA4DC49EA5 for ; Thu, 24 Jun 2021 16:06:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99721613EA for ; Thu, 24 Jun 2021 16:06:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231321AbhFXQIm (ORCPT ); Thu, 24 Jun 2021 12:08:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:54119 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229878AbhFXQIi (ORCPT ); Thu, 24 Jun 2021 12:08:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2arA4F9H8piXSu2BaYZfrwo91oZXpALZLKMt5odcOCY=; b=cxJHIkM5DPs+Z3+An7FZA7UqGNoRZKRVPiM+qFS8xEBTcWT79opsgTHBx7zMboqkWJubDr +Gbb7r7xDA03SLUjD7BNnSZ09tpZLjxBjQoAbL1IZzlXDWOxf0yahYoiCRuy/kwF3NLu4o N3ISjM4/z80wJHT7WbAPivhWMWl/F0E= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-545-aHF2feOHNciDquy5lCmujg-1; Thu, 24 Jun 2021 12:06:17 -0400 X-MC-Unique: aHF2feOHNciDquy5lCmujg-1 Received: by mail-ej1-f71.google.com with SMTP id u4-20020a1709061244b02904648b302151so2179269eja.17 for ; Thu, 24 Jun 2021 09:06:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2arA4F9H8piXSu2BaYZfrwo91oZXpALZLKMt5odcOCY=; b=C32f/eCpRLQbwyWaN4luo7m7dsRgHzvazzmc7VAOviDCN5d9ZaRCNveJk3KqDadduA 7XrDHRsBddWXdyiqeFLsc6TANMWdck484J+70PdxN9bQfoCfki4HPWhG9+rd/wthh8Nv NYbW3ZwDoUeIl6jR7sQPa3VoaeSqpwc5zOOxWhhtvh9RWg3ZfnB3oDBlQ+A9sj6p9blX lhQpzk/Vd4sbsqHtkO+mfhqTdZZ4aRqG2JncylqAzCHspd5/hEagtJ4GF0HTdEU2YZNh Xp+mDXcnmFzI4cbOIiqxFDwoshuF4vfXGHPyKELO98aCkJxMG5BwD/MXdj7wuEkv1FnA OdBg== X-Gm-Message-State: AOAM531PYvavCguoouEM/+DoS4VUiV77uxOzLwJ/jgjaN5pk6kl/mcCs pbueQdp3gotxzG8ozuUukiBTffhniQD9glmkV8WDX99hGdsv5UcpuUmyEuMWYptzcZNl5kr81ek tohP0LKTHpg+b X-Received: by 2002:aa7:c3d6:: with SMTP id l22mr8228020edr.245.1624550775703; Thu, 24 Jun 2021 09:06:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw09H7v0DCCEjs/tE/s2mmEWHfFhayzievmvBhlMCyz27KgLfOMPcrR1ar5NxO3zWi0Usga7Q== X-Received: by 2002:aa7:c3d6:: with SMTP id l22mr8227949edr.245.1624550775257; Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id o4sm2122595edc.94.2021.06.24.09.06.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:14 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 66FF318073A; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Sunil Goutham , linux-arm-kernel@lists.infradead.org Subject: [PATCH bpf-next v5 09/19] thunderx: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:05:59 +0200 Message-Id: <20210624160609.292325-10-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The thunderx driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Sunil Goutham Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/cavium/thunder/nicvf_main.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index c33b4e837515..e2b290135fd9 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -555,9 +555,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, xdp_prepare_buff(&xdp, hard_start, data - hard_start, len, false); orig_data = xdp.data; - rcu_read_lock(); action = bpf_prog_run_xdp(prog, &xdp); - rcu_read_unlock(); len = xdp.data_end - xdp.data; /* Check if XDP program has changed headers */ From patchwork Thu Jun 24 16:06:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342617 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02CE8C49EA5 for ; Thu, 24 Jun 2021 16:06:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E2173613EE for ; Thu, 24 Jun 2021 16:06:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231157AbhFXQIq (ORCPT ); Thu, 24 Jun 2021 12:08:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46151 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231250AbhFXQIm (ORCPT ); Thu, 24 Jun 2021 12:08:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8n6XcLLXm/HC8/e7TDiaIKa4D1+079NdF0CtM9PlMq8=; b=VQfWnt4zia9pxvAY3ZolyWQb1DZiqNHWkk9gscRMnuo/cU+MGikTLxu7pTuli1lLSrzh2u prv+DUPAdBuAF/2y93V4oZD9rF22Eb6UYECHQSPjik7654zBk2ICS4SlHkURBTt0wDv8m9 Lt5HtF4su8MdYpS+Bk087E03odsLcnY= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-157-vytXenA2MdKg65VTtdx5DQ-1; Thu, 24 Jun 2021 12:06:22 -0400 X-MC-Unique: vytXenA2MdKg65VTtdx5DQ-1 Received: by mail-ed1-f71.google.com with SMTP id h11-20020a50ed8b0000b02903947b9ca1f3so3614424edr.7 for ; Thu, 24 Jun 2021 09:06:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8n6XcLLXm/HC8/e7TDiaIKa4D1+079NdF0CtM9PlMq8=; b=PhAfjToPpdRuf5gg4OsJtmOS/H+F1TcaO+q2zSGL2bvw9K4q7hBDNTdRzHWrRbeDyH pW8jJxBZnIGopQE+Zx5nFvNcaGo3fJLfH+tpRp0gvi7Qh2Uaj0P63+gE3w04iXnnX/NH V6T/QcIoRoKi7WJqblDYQ5MQh3on5Rn/OZcbVXSmAPGFS4tgE9R9BQOA4SEHxMpx5O6o tv3K5TJ4nDTHVObdt9jG7rDWf2rUsMICfSeDLJdtoPMo4f9CZnVoUScHo6elcKPxXRsm JyO5EDlMk2AaJNMs0dOZXp4oDDF2KNHH+b6kwfCgbAffyhIXvlBWVMQHVmG1t7Eaj4T8 UQCw== X-Gm-Message-State: AOAM531tcIC79DT7yqudyIMZjSk69IUsGtFGZpMWoDT5Od7TWc35jiaM j8C3AckgSgxJh8PTe1CVRrVV2apCWOxgYEPiGBIGBvxJIR1PBH8DRsEl+xqsF+4V/0wOv+VCc9J oe+vL+bwJT+G4 X-Received: by 2002:a50:b8e6:: with SMTP id l93mr8489733ede.25.1624550780923; Thu, 24 Jun 2021 09:06:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx9a+Zc1daHM7E7EaR+rO9SkTP3jW41sNbM6ubu90/JqfixkamnHJK4VwcKKUiE6oKNLXWagA== X-Received: by 2002:a50:b8e6:: with SMTP id l93mr8489702ede.25.1624550780757; Thu, 24 Jun 2021 09:06:20 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id p18sm1308833edu.8.2021.06.24.09.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:15 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 6EED918073B; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Madalin Bucur , Ioana Ciornei , Ioana Radulescu , Camelia Groza Subject: [PATCH bpf-next v5 10/19] freescale: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:00 +0200 Message-Id: <20210624160609.292325-11-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The dpaa and dpaa2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Madalin Bucur Cc: Ioana Ciornei Cc: Ioana Radulescu Reviewed-by: Camelia Groza Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 8 +------- drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 3 --- 2 files changed, 1 insertion(+), 10 deletions(-) diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c index 177c020bf34a..e6826561cf11 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c @@ -2558,13 +2558,9 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, u32 xdp_act; int err; - rcu_read_lock(); - xdp_prog = READ_ONCE(priv->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) return XDP_PASS; - } xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE, &dpaa_fq->xdp_rxq); @@ -2638,8 +2634,6 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, break; } - rcu_read_unlock(); - return xdp_act; } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 8433aa730c42..973352393bd4 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -352,8 +352,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, u32 xdp_act = XDP_PASS; int err, offset; - rcu_read_lock(); - xdp_prog = READ_ONCE(ch->xdp.prog); if (!xdp_prog) goto out; @@ -414,7 +412,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, ch->xdp.res |= xdp_act; out: - rcu_read_unlock(); return xdp_act; } From patchwork Thu Jun 24 16:06:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342621 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B05F2C49EAF for ; Thu, 24 Jun 2021 16:06:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8DEB7613F0 for ; Thu, 24 Jun 2021 16:06:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231621AbhFXQIy (ORCPT ); Thu, 24 Jun 2021 12:08:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42206 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230377AbhFXQIo (ORCPT ); Thu, 24 Jun 2021 12:08:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550785; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d8AUSo1RzaNAGEUROtgo1E1rKFmBeyherDHigclTTCY=; b=IE8bP2WbFK3ImObfziU1L9Cp1GcqPEN8fQ05Hh8CxN6bCZCivfzV6P5Eg5DNW8m19/0RWm JwA5pHB+cBs679QfKboJD5KthCjfOVOWj7t8EWYnUQ4gVXFNfHosbYj8oz9k9O5nbLEgsU 2G+6AkjhXBgmEB+f6zbZX1kNrgw9JNY= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-128-H4vygurrO--kUhUJQGNWhA-1; Thu, 24 Jun 2021 12:06:23 -0400 X-MC-Unique: H4vygurrO--kUhUJQGNWhA-1 Received: by mail-ed1-f72.google.com with SMTP id j19-20020aa7c4130000b029039497d5cdbeso3581221edq.15 for ; Thu, 24 Jun 2021 09:06:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d8AUSo1RzaNAGEUROtgo1E1rKFmBeyherDHigclTTCY=; b=moN765Fbp4cMVS+qNsfEa9G+roq5WiCSrrpIzJH/oa7yvXE5jid7T9Rr33cfRXNCS8 l0jCXqswIEHYXmIVG8nooFWY5Yo8s6sN6FeghL/DIcBOSp+KMG40rhqCnXrM7b73Z8Hr kBLudVQyWEQ/Lj++O+pOAgz3ir8QeaGcM5PD+qNWTNJm5hr7BXajwuUwfgOSWpbvsJIm 9xjCDi/NfcmQT+98IKc7ELhCF6r9v8W8R+HPSvJtVxnTFg5yCVv10fUvM3dzXkRXl6mx RxtpzKY+0+NzDeigPq2Kyz6teOApUk9rDnkuGRZMWHK1H6UOTiZSun7os/bSoKrFgcPx hiHA== X-Gm-Message-State: AOAM530y5wiukyE8epYuLBV6ZAzN0W6FZ5LdH7VZDsUojnw+3klNhF7G BNQEs1H2irHA7qZzxaBmgT2/U3nMs/2RK4faYW2Vh3AdmvYg8mOfxo1qMeiFwjoZbB0/PPiiJlP C01KDKouS11v1 X-Received: by 2002:aa7:db93:: with SMTP id u19mr8156325edt.227.1624550782748; Thu, 24 Jun 2021 09:06:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxnTEqbmUGGZILpfV8wM2Ht6OYgL4HLYG/bFhxmow1xj2znDDrOXghxHZUpB9hR+99yIK7QTg== X-Received: by 2002:aa7:db93:: with SMTP id u19mr8156296edt.227.1624550782604; Thu, 24 Jun 2021 09:06:22 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id u22sm2217624edr.11.2021.06.24.09.06.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:21 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 7746518073C; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesse Brandeburg , Tony Nguyen , intel-wired-lan@lists.osuosl.org Subject: [PATCH bpf-next v5 11/19] net: intel: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:01 +0200 Message-Id: <20210624160609.292325-12-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The Intel drivers all have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Jesse Brandeburg Cc: Tony Nguyen Cc: intel-wired-lan@lists.osuosl.org Tested-by: Jesper Dangaard Brouer # i40e Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 2 -- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 3 --- drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +----- drivers/net/ethernet/intel/ice/ice_xsk.c | 3 --- drivers/net/ethernet/intel/igb/igb_main.c | 2 -- drivers/net/ethernet/intel/igc/igc_main.c | 7 ++----- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 -- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 3 --- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 2 -- 9 files changed, 3 insertions(+), 27 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index b883ab809df3..38eb8151ee9a 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -2298,7 +2298,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2334,7 +2333,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) break; } xdp_out: - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 68f177a86403..e7e778ca074c 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -153,7 +153,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); /* NB! xdp_prog will always be !NULL, due to the fact that * this path is enabled by setting an XDP program. */ @@ -164,7 +163,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); if (err) goto out_failure; - rcu_read_unlock(); return I40E_XDP_REDIR; } @@ -188,7 +186,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) result = I40E_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index 917eba7fdd0c..dd791ca34fab 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -1135,15 +1135,11 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size); #endif - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) goto construct_skb; - } xdp_res = ice_run_xdp(rx_ring, &xdp, xdp_prog); - rcu_read_unlock(); if (!xdp_res) goto construct_skb; if (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) { diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index 239b9bf10794..8a093368f631 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -466,7 +466,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) struct ice_ring *xdp_ring; u32 act; - rcu_read_lock(); /* ZC patch is enabled only when XDP program is set, * so here it can not be NULL */ @@ -478,7 +477,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); if (err) goto out_failure; - rcu_read_unlock(); return ICE_XDP_REDIR; } @@ -503,7 +501,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 5db303d64d14..7e6435dc7e80 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -8381,7 +8381,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -8416,7 +8415,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 3f6b6d4543a8..95323095094d 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -2240,18 +2240,15 @@ static struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(adapter->xdp_prog); if (!prog) { res = IGC_XDP_PASS; - goto unlock; + goto out; } res = __igc_xdp_run_prog(adapter, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 2ac5b82676f3..ffff69efd78a 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -2199,7 +2199,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2237,7 +2236,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index f72d2978263b..96dd1a4f956a 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -100,7 +100,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); act = bpf_prog_run_xdp(xdp_prog, xdp); @@ -108,7 +107,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); if (err) goto out_failure; - rcu_read_unlock(); return IXGBE_XDP_REDIR; } @@ -134,7 +132,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, result = IXGBE_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index dc56931fc1dc..c714e1ecd308 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -1054,7 +1054,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -1082,7 +1081,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } From patchwork Thu Jun 24 16:06:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342625 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1484DC49EA7 for ; Thu, 24 Jun 2021 16:06:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F27E1613DC for ; Thu, 24 Jun 2021 16:06:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231250AbhFXQI5 (ORCPT ); Thu, 24 Jun 2021 12:08:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44271 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230188AbhFXQIr (ORCPT ); Thu, 24 Jun 2021 12:08:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6RneWJJLq4syfEmR6mKrQCiPS0LPskxbjtJMl3vI+wY=; b=J3HDHzJTO0rrFmvFL/HxJuI91Kf3xNexxX7uHSLx/7SpEpkCUcwk6XcWcrBL3oVipj4Hfz 6ld2vXLga9TWyqnrsbl7S6T6Iq88OcOBGAjmxf+mURhBydV48PVmqxZv3Ncyk50AZp8K1x lSfwVCbjqMeMQynzsbflQ1alZ9lPYxM= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-360-TYYNzIusPXG08pQb4PsPig-1; Thu, 24 Jun 2021 12:06:25 -0400 X-MC-Unique: TYYNzIusPXG08pQb4PsPig-1 Received: by mail-ed1-f72.google.com with SMTP id v12-20020aa7dbcc0000b029038fc8e57037so3623556edt.0 for ; Thu, 24 Jun 2021 09:06:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6RneWJJLq4syfEmR6mKrQCiPS0LPskxbjtJMl3vI+wY=; b=MYVhAitxUke0JDPb06qCUlL4MLv0JWy9aZRoujf3TLSY5jugduwFpS8BP6JiMb5C/M mjCZxkh53oSW4HegEHXACC5kZ8Y5CSUe9mKMtnORACuU0Dz/tZQeQkZ8np3Sfy7jPN3t dWMFBPhH+7T8PIg2TiZ7Hh0nbr+B/uE40rU+cPrng2QXTBzUNJrpfzuh6lSPC52y5NL8 klesOlUjxRmdaIRwN6xNSfMwr5BlCd+/GhcbSjelKpTDdZiNgZnhsjt2PzMugCKXC9YO aizk9srsQ2mjvBo1zDpKX5ccdQ6tMNs9nRLs2fYvvEOaat2cSxbsa3HWQ3eLnUx8Y68f Cu4A== X-Gm-Message-State: AOAM530G+z3ONqiNwrvqiuSuBfwTXsUz1N6vhoyT5GLPkjAdp/nFxEH6 B5wynOsmahdemAT7gq4NKMSP99TNKYNJ5b+MOuPacDtplDlMCSw78Ef6J3Clnef5Iw+PHufG173 EfeLjzd8tl1bU X-Received: by 2002:a05:6402:1c1a:: with SMTP id ck26mr8090746edb.230.1624550783311; Thu, 24 Jun 2021 09:06:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwsnVsDAn+RfC06sxjMqbpeG3bGOhaMNmAiQdvprfLfKdKQ+jGnt0sMSwNSCElGhaGOQmVlHg== X-Received: by 2002:a05:6402:1c1a:: with SMTP id ck26mr8090699edb.230.1624550782981; Thu, 24 Jun 2021 09:06:22 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id e6sm1460005ejm.35.2021.06.24.09.06.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:21 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 82C1E18073D; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Thomas Petazzoni , Marcin Wojtas , Russell King Subject: [PATCH bpf-next v5 12/19] marvell: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:02 +0200 Message-Id: <20210624160609.292325-13-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The mvneta and mvpp2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Thomas Petazzoni Cc: Marcin Wojtas Cc: Russell King Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/marvell/mvneta.c | 2 -- drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 4 ---- 2 files changed, 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index c15ce06427d0..ada4e26a5492 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -2373,7 +2373,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, /* Get number of received packets */ rx_todo = mvneta_rxq_busy_desc_num_get(pp, rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(pp->xdp_prog); /* Fairness NAPI loop */ @@ -2451,7 +2450,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, xdp_buf.data_hard_start = NULL; sinfo.nr_frags = 0; } - rcu_read_unlock(); if (xdp_buf.data_hard_start) mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1); diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index 9bca8c8f9f8d..c31677527a02 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -3881,8 +3881,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, int rx_done = 0; u32 xdp_ret = 0; - rcu_read_lock(); - xdp_prog = READ_ONCE(port->xdp_prog); /* Get number of received packets and clamp the to-do */ @@ -4028,8 +4026,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, mvpp2_bm_pool_put(port, pool, dma_addr, phys_addr); } - rcu_read_unlock(); - if (xdp_ret & MVPP2_XDP_REDIR) xdp_do_flush_map(); From patchwork Thu Jun 24 16:06:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342623 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7C44C49EB7 for ; Thu, 24 Jun 2021 16:06:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF4B1613EB for ; Thu, 24 Jun 2021 16:06:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232005AbhFXQI5 (ORCPT ); Thu, 24 Jun 2021 12:08:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41029 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231653AbhFXQIr (ORCPT ); Thu, 24 Jun 2021 12:08:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624550787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4HZENA/DgIRqzTQFaPlI7HLLaKbjRPeaazW1V9p71pE=; b=Sop6EizUMfgZKiQtE/7+YgYvImWxiaNSzl+XFMAOrSWqQsl0ipWchFFEVDlTmqvS9iH4Vi ofIDgKyCLK+oaWymHlvasXfiTJ/fMYcrJIqX/fpsTLb8FeTqRHymyftk/2YUhKXvVznIOa 5UE7HS9Ib9CXFL9gh0eDz5FB/vrhewI= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-251-qq1PKpYgMeuBhpnNJRPgzg-1; Thu, 24 Jun 2021 12:06:25 -0400 X-MC-Unique: qq1PKpYgMeuBhpnNJRPgzg-1 Received: by mail-ed1-f72.google.com with SMTP id j19-20020aa7c4130000b029039497d5cdbeso3581278edq.15 for ; Thu, 24 Jun 2021 09:06:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4HZENA/DgIRqzTQFaPlI7HLLaKbjRPeaazW1V9p71pE=; b=iTqOUWkDrhvLVAxGnVcJFqQ1VVFUP/MqspsaUlp0pyqncjz4jnLcqRpWWET80x/w7i +ZK4CWyvCvLUIm0ycbgxArj4QhyoIYzw1ToFqBAl/Jgsz8rUDFp73EObiOGWtrX6t2Yq p1Zd8w/7rKlOFB/39goJNPkeExZyms1wDssqpkojtWt/m+mNvRjQLRcy/q/KUWvJ8w2Y syHrsYr30PlrzhX9VT+eLqQVEwUWxAT9r7UrzShJqyC/23OAKjjOiqDstiawRKSFvBFu eSofknCIFjgxMSYiVMvuaC6eX1HUetkZ7TKFiV/T2UhfKHrCFmGUkrWTZQkhXGuxsOKW fM2Q== X-Gm-Message-State: AOAM531esfAFsOfFSBnOti+GnFB1kZgB8IR6y79kvKyyS76zAvelo393 UP2B2WWVyeW3mnXCyT4wsK8WpMvWC2Amtt2QvAzG+sfPcwAuJWjNFMcZYwZ8sNzVwPXGbOQp/Tu rQZcDAywTxqGv X-Received: by 2002:a17:907:20ed:: with SMTP id rh13mr5923175ejb.137.1624550784125; Thu, 24 Jun 2021 09:06:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJymuoDGHbfvZKplY9RBsGybZkq21Hh0wdUnS+ml2EhRktVxqqDq4LhKonWx2QxmNk4uIIhRDQ== X-Received: by 2002:a17:907:20ed:: with SMTP id rh13mr5923141ejb.137.1624550783729; Thu, 24 Jun 2021 09:06:23 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id g12sm2149959edb.23.2021.06.24.09.06.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:06:22 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 8DA7418073E; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Tariq Toukan Subject: [PATCH bpf-next v5 13/19] mlx4: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:03 +0200 Message-Id: <20210624160609.292325-14-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The mlx4 driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Also switch the RCU dereferences in the driver loop itself to the _bh variants. Cc: Tariq Toukan Reviewed-by: Tariq Toukan Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index cea62b8f554c..442991d91c15 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -679,9 +679,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud ring = priv->rx_ring[cq_ring]; - /* Protect accesses to: ring->xdp_prog, priv->mac_hash list */ - rcu_read_lock(); - xdp_prog = rcu_dereference(ring->xdp_prog); + xdp_prog = rcu_dereference_bh(ring->xdp_prog); xdp_init_buff(&xdp, priv->frag_info[0].frag_stride, &ring->xdp_rxq); doorbell_pending = false; @@ -744,7 +742,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud /* Drop the packet, since HW loopback-ed it */ mac_hash = ethh->h_source[MLX4_EN_MAC_HASH_IDX]; bucket = &priv->mac_hash[mac_hash]; - hlist_for_each_entry_rcu(entry, bucket, hlist) { + hlist_for_each_entry_rcu_bh(entry, bucket, hlist) { if (ether_addr_equal_64bits(entry->mac, ethh->h_source)) goto next; @@ -899,8 +897,6 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud break; } - rcu_read_unlock(); - if (likely(polled)) { if (doorbell_pending) { priv->tx_cq[TX_XDP][cq_ring]->xdp_busy = true; From patchwork Thu Jun 24 16:06:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342635 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA1F8C49EA5 for ; Thu, 24 Jun 2021 16:14:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D4261613DC for ; Thu, 24 Jun 2021 16:14:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229884AbhFXQQR (ORCPT ); Thu, 24 Jun 2021 12:16:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37029 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230008AbhFXQQP (ORCPT ); Thu, 24 Jun 2021 12:16:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624551236; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3OGKlT+K3Cpr2CAOizG3FJ/dNahuGZz8NpheYYFR6ws=; b=c07ChdGBVDKtrrTD/ENTbPYUj4p9O/vKcc1f8b6j4+2kDGOI13hwGFT2tUgURW3djz5zj5 ypUOs+Q9X4WIc3ISumO1eOLZOmAjdRlAk/RPgH99JTHpgUMQ7HqmhdEaRqSzpqTAliWlkj FPC7PTc7ApqGglRgcrg6SvwT8rmaXf8= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-404-YzHZQrH5Ofiek63ZOk_sMQ-1; Thu, 24 Jun 2021 12:13:54 -0400 X-MC-Unique: YzHZQrH5Ofiek63ZOk_sMQ-1 Received: by mail-ed1-f71.google.com with SMTP id dy23-20020a05640231f7b0290394996f1452so3626917edb.18 for ; Thu, 24 Jun 2021 09:13:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3OGKlT+K3Cpr2CAOizG3FJ/dNahuGZz8NpheYYFR6ws=; b=VHiVfIsB6koVcmi7fsH1IT7ISys//kpAB91iSpcUYQtw8MLZa+aSWiJQnKo8+38/y6 lcIgkdaXF7YfLglUhDuO7OouNN8Vu6VG1LkdM/7OCoILS4wo2QWGooRP4uqQ1F3cFZVH 3PaMejsL66ZynmBOwjObR/wJ3GqnkIYavIPg5B8TvTYbE5Lozw6Fs66IL9jH6PxTcbuR 99DZszUBHDMGD5IzawDjzDqR1UaKk+FGB/eVbDejZhJe8AKeeUyMVBiIraH7N9ngqyR4 FpNAZCBP1JgdwWJ8AH+rXV2FVJ+bYaj9C7ciVkfRyQBZ0OFxL47s9jtIB2nkdAsC95AZ lEPg== X-Gm-Message-State: AOAM530fj0tVWTgW99cXJvpOP2/VwilA3bA9AaCx8230pl9kKFeDnXdP VCdji1tQaC/VQyDrksQO7bK5F2Bwp0G3Qg8TBa+n5avjq7y8n6FOWL6C+HQYRSz8T4JRLGpPpu1 eBZrSVnaHi4Na X-Received: by 2002:a17:906:1701:: with SMTP id c1mr6021309eje.425.1624551233597; Thu, 24 Jun 2021 09:13:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxkMmuHvtFL+2+qbxdEqWtnQFiQm8TsBCugpF27GOyLxyYhdPSN4NVwA3+KZgpg2/w7k0IVOQ== X-Received: by 2002:a17:906:1701:: with SMTP id c1mr6021291eje.425.1624551233393; Thu, 24 Jun 2021 09:13:53 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id gx4sm1472384ejc.34.2021.06.24.09.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:13:51 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 97A1018073F; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Simon Horman , oss-drivers@netronome.com Subject: [PATCH bpf-next v5 14/19] nfp: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:04 +0200 Message-Id: <20210624160609.292325-15-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The nfp driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. While this is not actually an issue for the nfp driver because it doesn't support XDP_REDIRECT (and thus doesn't call xdp_do_flush()), the rcu_read_lock() is still unneeded. And With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Simon Horman Cc: oss-drivers@netronome.com Reviewed-by: Simon Horman Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/netronome/nfp/nfp_net_common.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index eeb30680b4dc..5dfa4799c34f 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -1819,7 +1819,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) struct xdp_buff xdp; int idx; - rcu_read_lock(); xdp_prog = READ_ONCE(dp->xdp_prog); true_bufsz = xdp_prog ? PAGE_SIZE : dp->fl_bufsz; xdp_init_buff(&xdp, PAGE_SIZE - NFP_NET_RX_BUF_HEADROOM, @@ -2036,7 +2035,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) if (!nfp_net_xdp_complete(tx_ring)) pkts_polled = budget; } - rcu_read_unlock(); return pkts_polled; } From patchwork Thu Jun 24 16:06:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342633 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9F77C49EAB for ; Thu, 24 Jun 2021 16:13:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F285613EC for ; Thu, 24 Jun 2021 16:13:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230296AbhFXQQR (ORCPT ); Thu, 24 Jun 2021 12:16:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37852 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229884AbhFXQQP (ORCPT ); Thu, 24 Jun 2021 12:16:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624551235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZRUA/Y9lIwzUgTw5uRoa39eJ4yIyIch6xpJH66I9Qdc=; b=cS7bO3y0smmzslcOcbQjqNTn7G/V0yhkbIgWUNSzlql8EUoWkmRyWNm3VeS6MCE6NTZIDo a51RgdNMWYUixd21eTRAswB8GFYnURIckZMm61/FdBwqvMkJwSTOiDW1+mzBKZ6LMPyrS0 80IM0qpyNzmaLnxis7skY3hV1MbpDU8= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-404-Jg7_ZU_zNV6fH1QQLWIdGw-1; Thu, 24 Jun 2021 12:13:54 -0400 X-MC-Unique: Jg7_ZU_zNV6fH1QQLWIdGw-1 Received: by mail-ej1-f70.google.com with SMTP id g6-20020a1709064e46b02903f57f85ac45so2186459ejw.15 for ; Thu, 24 Jun 2021 09:13:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZRUA/Y9lIwzUgTw5uRoa39eJ4yIyIch6xpJH66I9Qdc=; b=qfpeoFrVgkFaxwW9UHGH/Zza9g+DccwGkWssbymLz2J9nKrDnnV5wDydOWRuxKME4S W2o6FHosR3d4ZONW+sgfyMEFoSr2/eU/4yq4XCI3G6LXKRicfKbfBz+bAXTYzabonObj /LG0nKr394bSrU+ppenYsI0JthBxP88XoPbbKaCB0BvydVLf5FnGst0d7GnAn8n0aZjS 6WLQku/lLjS0haX2uLpaItiPD0fZCoKPwjm9j1WIPLtipsP78glDqrDjjCdiaJVCtltt C3EFOZ2osX3NBge9T0kB91i64HhRDdMNqLOSULZ5EX46Dmvxl3o/Gy56FrkY/bKp/5cJ Tdkg== X-Gm-Message-State: AOAM533U9fm5ECRy1GsYihhM+NvZnrVG1nIMgRSjrFN7UIdEwwComzqD S4D7ecGkL0HjUoeKarwmRmwQBd2UyTIwrNbKSnP9kyVIyyekw2rmB+TrJMapQ0Tori4loUmlzw8 VVwcPw3uEsJRH X-Received: by 2002:a17:907:6224:: with SMTP id ms36mr6015252ejc.423.1624551233263; Thu, 24 Jun 2021 09:13:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwT+78mtmzK35e8akO5/2A3g2OAeHLhe/rtrcwTbal1bJnAixBqQ+JAWrNGT1lmcQfMaMJFnw== X-Received: by 2002:a17:907:6224:: with SMTP id ms36mr6015201ejc.423.1624551232778; Thu, 24 Jun 2021 09:13:52 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id aq21sm1572080ejc.83.2021.06.24.09.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:13:51 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id A1C97180740; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Ariel Elior , GR-everest-linux-l2@marvell.com Subject: [PATCH bpf-next v5 15/19] qede: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:05 +0200 Message-Id: <20210624160609.292325-16-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The qede driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Ariel Elior Cc: GR-everest-linux-l2@marvell.com Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/qlogic/qede/qede_fp.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c index 8e150dd4f899..065e9004598e 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_fp.c +++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c @@ -1089,13 +1089,7 @@ static bool qede_rx_xdp(struct qede_dev *edev, xdp_prepare_buff(&xdp, page_address(bd->data), *data_offset, *len, false); - /* Queues always have a full reset currently, so for the time - * being until there's atomic program replace just mark read - * side for map helpers. - */ - rcu_read_lock(); act = bpf_prog_run_xdp(prog, &xdp); - rcu_read_unlock(); /* Recalculate, as XDP might have changed the headers */ *data_offset = xdp.data - xdp.data_hard_start; From patchwork Thu Jun 24 16:06:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342637 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30093C49EAB for ; Thu, 24 Jun 2021 16:14:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 16B0A613EB for ; Thu, 24 Jun 2021 16:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231169AbhFXQQU (ORCPT ); Thu, 24 Jun 2021 12:16:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45600 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229853AbhFXQQR (ORCPT ); Thu, 24 Jun 2021 12:16:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624551237; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cOyh19f+n3Klc9n0Uf50USUZ/OAvnTAcZiRzl4mD+JA=; b=bvZ5Rdx5mkM9FtutcKafhMMIGgD5MPRuICcDIc3ChUORK4Qt73BsbuNJ6Jxj0uKbDp8O0U odXUhXshnwlcimCYTEu0lXH9Rn5iq7+Of1C//Yqc73hFFkLZMIjHh8cwQi9OVwCl6xLJh9 oHNc+6N7EAt5c4szuLT/4Z+M1UkUdqE= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-6-VcAfpJqsO26wt8MgsvDRzQ-1; Thu, 24 Jun 2021 12:13:55 -0400 X-MC-Unique: VcAfpJqsO26wt8MgsvDRzQ-1 Received: by mail-ed1-f71.google.com with SMTP id cb4-20020a0564020b64b02903947455afa5so3610529edb.9 for ; Thu, 24 Jun 2021 09:13:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cOyh19f+n3Klc9n0Uf50USUZ/OAvnTAcZiRzl4mD+JA=; b=Hu/RSsqWtabQhukg8OaZJ+gg4QmnwQw4H8/oIBSj98j0QRDa3DYo/eGYR6aJ3BXJK4 Z50GPfHLjrH0/ttvr9qyeREnAjrJce6ml189VwnY+rpwm5jF0cPUl12J5akoPz9t6DDS 59NkJH6Thkc2NWGYpmkobnnEqPoxz0KNeIdGNWGgh1sSzYtNorTkq5nyVAHKx/axLcV+ BRC5ZqqQAVKGmRJXOY8vOiTbii+PLPjyfCTYj0CCe7Js8ejii+Ei+nmmTS1+pkosw5NA gqwxLMIOE/7NVrqIDyyqqG6n3qLjKDOQkD/szSdI4YnVvnfTuOAaSgHX1RCGQaG/quHL u/RA== X-Gm-Message-State: AOAM5306+DARt5Jx+JKQnjb0+kyOXk6xXraeZqwq6UFp7BjfbOe5gMb1 4G8/m4PJR86tvqgJcTT89q5+KSE8A1nIwV2mE7bUvi1qfCy521Il9MNJht7r+e5gYapvGBeD+a7 sz3qY6Fv9ocrG X-Received: by 2002:aa7:dc0d:: with SMTP id b13mr8086190edu.288.1624551234002; Thu, 24 Jun 2021 09:13:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwO83p9KKeIz6WYXSUxDTv54YrbHmZN8vEVgN4NSgGIUd/3XQocFyAZ4VFu9V7EzZ2au7ZN8g== X-Received: by 2002:aa7:dc0d:: with SMTP id b13mr8086172edu.288.1624551233868; Thu, 24 Jun 2021 09:13:53 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id d2sm1433158ejo.13.2021.06.24.09.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:13:51 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id AC1B8180741; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Edward Cree , Martin Habets Subject: [PATCH bpf-next v5 16/19] sfc: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:06 +0200 Message-Id: <20210624160609.292325-17-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The sfc driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Edward Cree Cc: Martin Habets Acked-by: Edward Cree Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/sfc/rx.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c index 17b8119c48e5..606750938b89 100644 --- a/drivers/net/ethernet/sfc/rx.c +++ b/drivers/net/ethernet/sfc/rx.c @@ -260,18 +260,14 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel, s16 offset; int err; - rcu_read_lock(); - xdp_prog = rcu_dereference(efx->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + xdp_prog = rcu_dereference_bh(efx->xdp_prog); + if (!xdp_prog) return true; - } rx_queue = efx_channel_get_rx_queue(channel); if (unlikely(channel->rx_pkt_n_frags > 1)) { /* We can't do XDP on fragmented packets - drop. */ - rcu_read_unlock(); efx_free_rx_buffers(rx_queue, rx_buf, channel->rx_pkt_n_frags); if (net_ratelimit()) @@ -296,7 +292,6 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel, rx_buf->len, false); xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp); - rcu_read_unlock(); offset = (u8 *)xdp.data - *ehp; From patchwork Thu Jun 24 16:06:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342629 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7121CC49EA6 for ; Thu, 24 Jun 2021 16:13:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 57619613E3 for ; Thu, 24 Jun 2021 16:13:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229721AbhFXQQO (ORCPT ); Thu, 24 Jun 2021 12:16:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49341 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229445AbhFXQQO (ORCPT ); Thu, 24 Jun 2021 12:16:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624551234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yDyNUpCpBTNbFxDC2WQuaaPWytT5QLCtZC1t66JXAlk=; b=JfxWsaCs22FzY1pLsCJjrbzP7oF3+E2SSiq2gLzrag15uHro/eMFaXyVu8n8LrV0m9FP9b iFnS5BD3EwdeDeLrj24NlaZQn+SkmuPXw/6h4Esj3fMB06AzYMlC3wpPSWHE0N5BL9SP9q 0b3fg/Ra9LVb6klSDCJGUw1ENNgXbqc= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-24-WnXsgRxoPPSKLk09xKemyw-1; Thu, 24 Jun 2021 12:13:53 -0400 X-MC-Unique: WnXsgRxoPPSKLk09xKemyw-1 Received: by mail-ej1-f69.google.com with SMTP id w22-20020a17090652d6b029048a3391d9f6so2210393ejn.12 for ; Thu, 24 Jun 2021 09:13:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yDyNUpCpBTNbFxDC2WQuaaPWytT5QLCtZC1t66JXAlk=; b=RvxBMFPuzSBKIpJu2Zo+JPSWsavTaWDsZns5eixYes4xjZYVfExponuoS9lNnKb7rv NwGpFLMFbWS3yIORPjHhiDqC7t3eZxuruFzErIMmA6YOJg7l5s1p8nwBGkC4mPr5OUew BHBLcnrdB1jC3wezL3Q6h+aEeRXvJtXF+qSpZFAe9O99XhUJgqvkB1C/F1ZVL8fp0EKz 5/Iui92PDDqe9u1aEv+waI0l1YQMuTLu+scdgJSCAeIxfdSxC+ou9kvS1Vet/L14RN4k TsHg8wvPlS0P7+F6ZEOtN6qN5ilf+Xda81uKuGK4cvqZDhEKCkDaDiTkIXp48E1+cJNO uacA== X-Gm-Message-State: AOAM531fmlxW5PtnhxlIR+m6TrwE6ajh1liJdzyYKlaonLerncsYG5id 2Ic5PghYPymGGOsVacV9Va/ctMqKjZ5H3bBVqHytgBA6zzS0sUqRz4GjvXMy/tNRELaubqcnG1b MNQ+5eRNRdBzc X-Received: by 2002:a17:906:528b:: with SMTP id c11mr6149366ejm.156.1624551232256; Thu, 24 Jun 2021 09:13:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxmiy/R1V/yo+psYvdlKpfKYxOjFaoS5khi6mncmtsUTrJsrGvD/i4IhsMzsu6FIJJ29TNGlw== X-Received: by 2002:a17:906:528b:: with SMTP id c11mr6149341ejm.156.1624551231979; Thu, 24 Jun 2021 09:13:51 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id w10sm2246691edv.34.2021.06.24.09.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:13:51 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id B294B180742; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jassi Brar , Ilias Apalodimas Subject: [PATCH bpf-next v5 17/19] netsec: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:07 +0200 Message-Id: <20210624160609.292325-18-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The netsec driver has a rcu_read_lock()/rcu_read_unlock() pair around the full RX loop, covering everything up to and including xdp_do_flush(). This is actually the correct behaviour, but because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), it is also technically redundant. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep the rcu_read_lock() around anymore, so let's just remove it. Cc: Jassi Brar Cc: Ilias Apalodimas Acked-by: Ilias Apalodimas Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/socionext/netsec.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index dfc85cc68173..20d148c019d8 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -958,7 +958,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) xdp_init_buff(&xdp, PAGE_SIZE, &dring->xdp_rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(priv->xdp_prog); dma_dir = page_pool_get_dma_dir(dring->page_pool); @@ -1069,8 +1068,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) } netsec_finalize_xdp_rx(priv, xdp_act, xdp_xmit); - rcu_read_unlock(); - return done; } From patchwork Thu Jun 24 16:06:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342631 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57A65C49EA5 for ; Thu, 24 Jun 2021 16:13:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3E125613DC for ; Thu, 24 Jun 2021 16:13:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230170AbhFXQQQ (ORCPT ); Thu, 24 Jun 2021 12:16:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:38010 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229853AbhFXQQP (ORCPT ); Thu, 24 Jun 2021 12:16:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624551235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T2TFlJi9GKn8kHSzVGGvW65aiKXAw7C9LqqB3xQR0H8=; b=On/IZPNM4EFUMoScWbbOD1gzWKp8v1zBEecp3/5QMMZEX0f7VgLcbS3XbqLxVWImZyxdHl Wd3W5nTaM3BnEzOZNGQg3hA17uVoVeCR1/fyMydIyKjiPM62B4zQ6B7p7uhQWTyAWW0G+Q IPAlIlNCNjZSecIzQiC80ds9HeJ64ek= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-393-qaHoTQblO923ROyAhjrnQA-1; Thu, 24 Jun 2021 12:13:54 -0400 X-MC-Unique: qaHoTQblO923ROyAhjrnQA-1 Received: by mail-ed1-f71.google.com with SMTP id y17-20020a0564023591b02903951740fab5so219931edc.23 for ; Thu, 24 Jun 2021 09:13:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T2TFlJi9GKn8kHSzVGGvW65aiKXAw7C9LqqB3xQR0H8=; b=fOqkUIFrTygF4XgdKGpTkrzOzI+bGKs+c8lq0nz79FJsBnkrjsYcBnp2h5i0ewkAvO LkXT14kAOGpTyOt77CnkTmjHNLMc5s35nWLWG3NQfU+mBHStByPN9SuPjkVGHxi4zkEh DTkawWMgkvpPxCqncWUkS+H96Vqelg+0jMUdIMFMTARQSUZV7+V9UEjxdr2fa62KJ9Vw iqeKsk/imiUXvCDzeBA0umLoZ02BsNMFLwtByYzOKECrmEF800uldn0bPBqFjg1lTIqt MDL6lK5SVxjRJ09UlReAbgAwFr81+BH4iKwnac2bwK0tURm3G1KWiirp2Ot6dVJwOIsP 545A== X-Gm-Message-State: AOAM530vgKzE00HXU0n37nl0dIlRTncbMmELpASUC6/0Z/Vl7q5QmhOu Cx0ajZsCpg0jW3qb11oZaLJ49All7bvyS+g4NSnnYSPROAp4P5MisIAwsj1NKwyLOhLV3V6EKIR End+qtABDUkTR X-Received: by 2002:a05:6402:204:: with SMTP id t4mr8227418edv.34.1624551232804; Thu, 24 Jun 2021 09:13:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxBzTBV14wD4eImYSJr0NCBJUPZ6x2+8VMbO/vpHKDq2WgvbGSgIyf02FNDKLvGPedudfKloA== X-Received: by 2002:a05:6402:204:: with SMTP id t4mr8227364edv.34.1624551232385; Thu, 24 Jun 2021 09:13:52 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id b24sm1421071ejl.61.2021.06.24.09.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:13:51 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id B9907180743; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu Subject: [PATCH bpf-next v5 18/19] stmmac: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:08 +0200 Message-Id: <20210624160609.292325-19-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The stmmac driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Giuseppe Cavallaro Cc: Alexandre Torgue Cc: Jose Abreu Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 16820873b01d..219535ab2c0c 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -4651,7 +4651,6 @@ static int stmmac_xdp_xmit_back(struct stmmac_priv *priv, return res; } -/* This function assumes rcu_read_lock() is held by the caller. */ static int __stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog, struct xdp_buff *xdp) @@ -4693,17 +4692,14 @@ static struct sk_buff *stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); if (!prog) { res = STMMAC_XDP_PASS; - goto unlock; + goto out; } res = __stmmac_xdp_run_prog(priv, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } @@ -4973,10 +4969,8 @@ static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue) buf->xdp->data_end = buf->xdp->data + buf1_len; xsk_buff_dma_sync_for_cpu(buf->xdp, rx_q->xsk_pool); - rcu_read_lock(); prog = READ_ONCE(priv->xdp_prog); res = __stmmac_xdp_run_prog(priv, prog, buf->xdp); - rcu_read_unlock(); switch (res) { case STMMAC_XDP_PASS: From patchwork Thu Jun 24 16:06:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12342639 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C62E9C49EA7 for ; Thu, 24 Jun 2021 16:14:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2613613DC for ; Thu, 24 Jun 2021 16:14:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231527AbhFXQQV (ORCPT ); Thu, 24 Jun 2021 12:16:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27635 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230052AbhFXQQS (ORCPT ); Thu, 24 Jun 2021 12:16:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624551238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=436OaxB7FEvDC8Cj4cGRRV7bgo1JzvFAvSGPF+E5hVA=; b=Jan/KNbZsQ6l5N33RAmp+3SZgMwQBRR1zqGFBWdyQD3TNpYBQNVzcSckFJNxBoatZwxsDm KTAuO5rBRZfv9t9ze7IW6dAPSx1drcxXshCNuNcwy4vkEEvUlqaoRkRkcldNoAEvHjOvYX MKqI0YBCwcZvdwiARA60frbepynq5BQ= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-261-768-Pu25Pmi8hIm2eMO4mQ-1; Thu, 24 Jun 2021 12:13:57 -0400 X-MC-Unique: 768-Pu25Pmi8hIm2eMO4mQ-1 Received: by mail-ed1-f69.google.com with SMTP id p23-20020aa7cc970000b02903948bc39fd5so3602299edt.13 for ; Thu, 24 Jun 2021 09:13:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=436OaxB7FEvDC8Cj4cGRRV7bgo1JzvFAvSGPF+E5hVA=; b=p567rtUiO/nVZOTnljHxQ90abx/ct31mCxpUbGdkGQYF8/16x6axgDh0LkqwT0zkjS GvT3gJyCqHTMOus5JW0hPmnpz1UvrlMokEaJc2c8e1aPx1NepDmc0qJG93/giTpybwH2 PzpWRQvM6UPK0jFIKG4V0zBih46CmilRf124XSnJy3GXkZCLLJ1vrp9LyBu9vO5zWPcG KwxR/ZuO9yTIqjNr1dMu5IKMYFJRsbzmkX5b679t1ilu5PRsfvOAkvsxnDjkHKL2cArd sgrVCY0g60YkC76Wu6jgasPjXCbJQbX913Ak1Zi4wPzymMk3M6QO7FvPhYvIYxZ5uIJQ VnRg== X-Gm-Message-State: AOAM533XtDOzg/ocyxqjrKAGFPLo4OqYbZkiNqsQOjM98orUAvjcebxq dZhlNK8zGLSDZBJgqQu/3ihoTHWrgcbJOTv3SpKfcxyQl2jai4tP97pLGzhmyRZhr7K2gBcQE4S LeFD8B/NCNheK X-Received: by 2002:a17:906:6c92:: with SMTP id s18mr6001742ejr.246.1624551235823; Thu, 24 Jun 2021 09:13:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyTqg1XkldV4Ka01Fa9wZUX9Na6VfS2RopbGr4a7OYV7FQUPzbFORmoCWAMgTSsKp/g8aWzXw== X-Received: by 2002:a17:906:6c92:: with SMTP id s18mr6001702ejr.246.1624551235374; Thu, 24 Jun 2021 09:13:55 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id yh11sm1408621ejb.16.2021.06.24.09.13.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 09:13:54 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id C0535180744; Thu, 24 Jun 2021 18:06:10 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Grygorii Strashko , linux-omap@vger.kernel.org Subject: [PATCH bpf-next v5 19/19] net: ti: remove rcu_read_lock() around XDP program invocation Date: Thu, 24 Jun 2021 18:06:09 +0200 Message-Id: <20210624160609.292325-20-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210624160609.292325-1-toke@redhat.com> References: <20210624160609.292325-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The cpsw driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Grygorii Strashko Cc: linux-omap@vger.kernel.org Tested-by: Grygorii Strashko Reviewed-by: Grygorii Strashko Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/ti/cpsw_priv.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/ti/cpsw_priv.c b/drivers/net/ethernet/ti/cpsw_priv.c index 5862f0a4a975..ecc2a6b7e28f 100644 --- a/drivers/net/ethernet/ti/cpsw_priv.c +++ b/drivers/net/ethernet/ti/cpsw_priv.c @@ -1328,13 +1328,9 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp, struct bpf_prog *prog; u32 act; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); - if (!prog) { - ret = CPSW_XDP_PASS; - goto out; - } + if (!prog) + return CPSW_XDP_PASS; act = bpf_prog_run_xdp(prog, xdp); /* XDP prog might have changed packet data and boundaries */ @@ -1378,10 +1374,8 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp, ndev->stats.rx_bytes += *len; ndev->stats.rx_packets++; out: - rcu_read_unlock(); return ret; drop: - rcu_read_unlock(); page_pool_recycle_direct(cpsw->page_pool[ch], page); return ret; }