From patchwork Wed Jun 9 10:33:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309551 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD5F3C48BDF for ; Wed, 9 Jun 2021 10:33:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8E29661185 for ; Wed, 9 Jun 2021 10:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237402AbhFIKf3 (ORCPT ); Wed, 9 Jun 2021 06:35:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33236 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237125AbhFIKf3 (ORCPT ); Wed, 9 Jun 2021 06:35:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gxp4+onf8wuO87Iii4KRL2PGeBHDLINdkwBz3RXDQ8I=; b=T2CJqXdpOltf5zPOQAefDXoAqJoDp/l8SysxRlyzRBPiWMMcAi3xIuHIMA8iaqvo67Hb/h biVz9uBa6ON7EcPV2QpmrDBvgrmqM39rA/TY8d0LasbfXvuNYn2yZIqJ1XiG7tPbqc1mqK RpKfIbtzzhMmqqxgKI0Kx2Jfnm5vC9k= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-318-TjScLYWfOQ6PKftqD2nGKA-1; Wed, 09 Jun 2021 06:33:33 -0400 X-MC-Unique: TjScLYWfOQ6PKftqD2nGKA-1 Received: by mail-ej1-f72.google.com with SMTP id b10-20020a170906194ab02903ea7d084cd3so7849288eje.1 for ; Wed, 09 Jun 2021 03:33:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Gxp4+onf8wuO87Iii4KRL2PGeBHDLINdkwBz3RXDQ8I=; b=GjHRYBQmBX7KcyHvcCIynyCp+fU+zoUn7QvrtAf3nueG5A2jO/j6XA4wOFMlbVurW7 E4mt1ihIKCdWD4wx8P620WQpEPkwXmQbDWw4H6GesIEtuTqP8VJrjy58FjhDGuhWapYj jCLeMfE/AB2r1+NDKzIVkeSFxO+lLyE0lqSb+3jr36NiL0hWEC6Vi4jUmvIhMIN+foNZ YBOQfbeq2si4UQgc5xrmPqFRoqwhSDYQksHDq/x6iYfEhEZ43sKThxSrWwVOa5dCbwJd FuaeFPGpwvJN5Mr9j3QkqNxiaVABW3GmIQ4/Thd4ckyZSInzkdbNB3yYBgYMvS2kxZb/ 9mFg== X-Gm-Message-State: AOAM533d5pVighj8V3GKLxjgPVinpiK+FRPXrnsMMu4VrHnPTL+XenK+ buX3enpSjekvG22DrdlyzBVSWZBdpfFN1cOZnvTUd3bTOCb2S4xDdxPqDjFsBQ2bJZycm9BS9Ff WmI+zDfmHJv6h X-Received: by 2002:a05:6402:26c7:: with SMTP id x7mr2175507edd.383.1623234812141; Wed, 09 Jun 2021 03:33:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy2igLZbG/g6/sOl81ouk5Bu5Pe7/HfADpKcVG79f8SVe4ZITtwLk2lEhEJTshHty5qumc5Xw== X-Received: by 2002:a05:6402:26c7:: with SMTP id x7mr2175482edd.383.1623234811710; Wed, 09 Jun 2021 03:33:31 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id t2sm917109ejx.72.2021.06.09.03.33.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:31 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 5C2EE1802AC; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= Subject: [PATCH bpf-next 01/17] rcu: Create an unrcu_pointer() to remove __rcu from a pointer Date: Wed, 9 Jun 2021 12:33:10 +0200 Message-Id: <20210609103326.278782-2-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: "Paul E. McKenney" The xchg() and cmpxchg() functions are sometimes used to carry out RCU updates. Unfortunately, this can result in sparse warnings for both the old-value and new-value arguments, as well as for the return value. The arguments can be dealt with using RCU_INITIALIZER(): old_p = xchg(&p, RCU_INITIALIZER(new_p)); But a sparse warning still remains due to assigning the __rcu pointer returned from xchg to the (most likely) non-__rcu pointer old_p. This commit therefore provides an unrcu_pointer() macro that strips the __rcu. This macro can be used as follows: old_p = unrcu_pointer(xchg(&p, RCU_INITIALIZER(new_p))); Reported-by: Toke Høiland-Jørgensen Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- include/linux/rcupdate.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 9455476c5ba2..d7895b81264e 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -363,6 +363,20 @@ static inline void rcu_preempt_sleep_check(void) { } #define rcu_check_sparse(p, space) #endif /* #else #ifdef __CHECKER__ */ +/** + * unrcu_pointer - mark a pointer as not being RCU protected + * @p: pointer needing to lose its __rcu property + * + * Converts @p from an __rcu pointer to a __kernel pointer. + * This allows an __rcu pointer to be used with xchg() and friends. + */ +#define unrcu_pointer(p) \ +({ \ + typeof(*p) *_________p1 = (typeof(*p) *__force)(p); \ + rcu_check_sparse(p, __rcu); \ + ((typeof(*p) __force __kernel *)(_________p1)); \ +}) + #define __rcu_access_pointer(p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p) *__force)READ_ONCE(p); \ From patchwork Wed Jun 9 10:33:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309549 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 835DAC48BD1 for ; Wed, 9 Jun 2021 10:33:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BB7461185 for ; Wed, 9 Jun 2021 10:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238842AbhFIKfa (ORCPT ); Wed, 9 Jun 2021 06:35:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25875 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234299AbhFIKfa (ORCPT ); Wed, 9 Jun 2021 06:35:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TZ38gMUjEr+gzQd0/QpPSZkBw8zgTtRoyYxgvJDwPiA=; b=NIlDDVp5b60dA/Hy6kVPmTvgIbkYN/BuMhgdUNDbTYENJGtvz2fIMF1pH/kIuFMkVZBZr6 P/1+0ka8B0uKhvTSuLLd7MJVHy3GItNh98rbiDEzjIGeefL8DOHVC5TEzypduzw/JEnXTY JItpvplEtPedJhpqPfNvE1ttwhdIb0A= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-554-G9ZEKnjRNP-wx-tBICtpNA-1; Wed, 09 Jun 2021 06:33:33 -0400 X-MC-Unique: G9ZEKnjRNP-wx-tBICtpNA-1 Received: by mail-ej1-f71.google.com with SMTP id nd10-20020a170907628ab02903a324b229bfso7871154ejc.7 for ; Wed, 09 Jun 2021 03:33:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TZ38gMUjEr+gzQd0/QpPSZkBw8zgTtRoyYxgvJDwPiA=; b=pegUqml2c6JkLS16gdpk3D9Ani8M97ivnDKZgxftj056Gv56DhE3ubVISBGAwoYU1W P/DwroT+c7gp6yzFrl7SkiiArVOLP02/XlnZTxwVZ1pJF9QY4ZWGUV3zpCDkHfply6k7 Z7kJUrTC6cSo7UQ0hf3rpqD9NVOtB42XoJBcLRAexhNDhIF3nPryfZ51ydjhvc6Ij54V kC6q1RYf1GzhsNerG+AxRnHN0gAQiug2tpTxZv0KLkGHaaIQxHzqVX0kgF5QS9K3VeR7 a0g5gBkT3wTMdLnUgrGiHc9kCFMQzCS24KQGoAeUQrO65WTxdTXSBsvz8RzPR0yLC2fA DHvw== X-Gm-Message-State: AOAM532AViaH4GCKi2PHPVJS1C00AqO1BLf54FdKOv7gk1bHps603Vgv 6YxMatok65rFjolky6VTvTJd3hjcot1pX6Weo2rSpvH+OiwljAyCpBiz8WISxPjKHhbkw5/eRbh Ty/wgmF0DWjnG X-Received: by 2002:a05:6402:22fa:: with SMTP id dn26mr29419029edb.230.1623234812774; Wed, 09 Jun 2021 03:33:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4ZjOdgxvzRTTNY2mqpmg+pC9YqQNk4wWQ4bO1hEvEggwyYS0+lnFyVMWnGslD9Q5MForIEg== X-Received: by 2002:a05:6402:22fa:: with SMTP id dn26mr29419016edb.230.1623234812620; Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id a19sm717702ejk.46.2021.06.09.03.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 64B9F1802B1; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= Subject: [PATCH bpf-next 02/17] bpf: allow RCU-protected lookups to happen from bh context Date: Wed, 9 Jun 2021 12:33:11 +0200 Message-Id: <20210609103326.278782-3-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net XDP programs are called from a NAPI poll context, which means the RCU reference liveness is ensured by local_bh_disable(). Add rcu_read_lock_bh_held() as a condition to the RCU checks for map lookups so lockdep understands that the dereferences are safe from inside *either* an rcu_read_lock() section *or* a local_bh_disable() section. This is done in preparation for removing the redundant rcu_read_lock()s from the drivers. Signed-off-by: Toke Høiland-Jørgensen Acked-by: Martin KaFai Lau --- kernel/bpf/hashtab.c | 21 ++++++++++++++------- kernel/bpf/helpers.c | 6 +++--- kernel/bpf/lpm_trie.c | 6 ++++-- 3 files changed, 21 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 6f6681b07364..72c58cc516a3 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -596,7 +596,8 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) struct htab_elem *l; u32 hash, key_size; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -989,7 +990,8 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1082,7 +1084,8 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1148,7 +1151,8 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1202,7 +1206,8 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1276,7 +1281,8 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1311,7 +1317,8 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 544773970dbc..e880f6bb6f28 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -28,7 +28,7 @@ */ BPF_CALL_2(bpf_map_lookup_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return (unsigned long) map->ops->map_lookup_elem(map, key); } @@ -44,7 +44,7 @@ const struct bpf_func_proto bpf_map_lookup_elem_proto = { BPF_CALL_4(bpf_map_update_elem, struct bpf_map *, map, void *, key, void *, value, u64, flags) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_update_elem(map, key, value, flags); } @@ -61,7 +61,7 @@ const struct bpf_func_proto bpf_map_update_elem_proto = { BPF_CALL_2(bpf_map_delete_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_delete_elem(map, key); } diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 1b7b8a6f34ee..423549d2c52e 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -232,7 +232,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) /* Start walking the trie from the root node ... */ - for (node = rcu_dereference(trie->root); node;) { + for (node = rcu_dereference_check(trie->root, rcu_read_lock_bh_held()); + node;) { unsigned int next_bit; size_t matchlen; @@ -264,7 +265,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) * traverse down. */ next_bit = extract_bit(key->data, node->prefixlen); - node = rcu_dereference(node->child[next_bit]); + node = rcu_dereference_check(node->child[next_bit], + rcu_read_lock_bh_held()); } if (!found) From patchwork Wed Jun 9 10:33:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309559 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A67EC48BD1 for ; Wed, 9 Jun 2021 10:33:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 821DB613AD for ; Wed, 9 Jun 2021 10:33:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238844AbhFIKfe (ORCPT ); Wed, 9 Jun 2021 06:35:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43239 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238903AbhFIKfb (ORCPT ); Wed, 9 Jun 2021 06:35:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c/SrqPGFLKHkMSppfFjGdsFHhz75MKKcU6rUhA3fuUE=; b=B0mbVZl/WIpapdiEQ3DlRw6F6/ADNABUPTca/tt0eBYVaITHe+9b8re4k5jgB4J9trOYEq fTgn+Z89Xi9A4zZdPhVkT5PZaNltgHbsBKzHzlHLsgndu8SToTJg1EiDKM3cjVVe67ubeW yuD5S8SV6bmArHxx4EKSEBIx/0k/dV8= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-350-nZKnQVf3Mzuix2C7w8fJRw-1; Wed, 09 Jun 2021 06:33:33 -0400 X-MC-Unique: nZKnQVf3Mzuix2C7w8fJRw-1 Received: by mail-ej1-f69.google.com with SMTP id w13-20020a170906384db02903d9ad6b26d8so7865976ejc.0 for ; Wed, 09 Jun 2021 03:33:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c/SrqPGFLKHkMSppfFjGdsFHhz75MKKcU6rUhA3fuUE=; b=DS6ns9gvJ/sMouuV5Ggb3DyahKxJEj7X1XUpXfU55GIybC0rXSPOX//sYSW37HIV+l C0ROtB0I+ZlVEI5mser8nBj06xN13Qo+DtnKyvb3+jRmvKpUK2BxMrtDRX3WBZnLzjEe NxzOM2W5F8qPXjL6AtenSKmpH03d0TtekSKG/CymnRNiXUcWVFZJnx/NE4UNavksK8xm I8w0yTbfUuPyUTPc5wExMLj9optDsDnh0pt+IKbPVzm+jBCfnI4qvlgbEzZYQ7Bd3P6t SBGuLWK1X2T2YV2oO5+QVzUG3xw0/3CcE0GkDDL5OGm0EFgOz+zekhdzkAzMPfe/ZQw5 TSew== X-Gm-Message-State: AOAM533EkF5RNG4IVWl1O8wK7ImfsFL67N0I+c4LCRtZfn+88JTig8Qq qYxTYomveZr51FFh5Znd+Lst+GC6eZGnT7ndIqCGS/3gIzSYOe4FaQvbTMxHBofRO6fZsUU6QOy LFJLXqm15a5Cl X-Received: by 2002:a17:906:7052:: with SMTP id r18mr27442490ejj.449.1623234812601; Wed, 09 Jun 2021 03:33:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4/qlqfnwcNgWwFsRkkZlUJhNrhFNB6oEw7PGNWLub37Dm+YNrF8XK6GjyP+L1HoY7Bz9mQQ== X-Received: by 2002:a17:906:7052:: with SMTP id r18mr27442470ejj.449.1623234812401; Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id n26sm930539ejc.27.2021.06.09.03.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 6E3F3180727; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= Subject: [PATCH bpf-next 03/17] dev: add rcu_read_lock_bh_held() as a valid check when getting a RCU dev ref Date: Wed, 9 Jun 2021 12:33:12 +0200 Message-Id: <20210609103326.278782-4-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Some of the XDP helpers (in particular, xdp_do_redirect()) will get a struct net_device reference using dev_get_by_index_rcu(). These are called from a NAPI poll context, which means the RCU reference liveness is ensured by local_bh_disable(). Add rcu_read_lock_bh_held() as a condition to the RCU list traversal in dev_get_by_index_rcu() so lockdep understands that the dereferences are safe from *both* an rcu_read_lock() *and* with local_bh_disable(). Signed-off-by: Toke Høiland-Jørgensen --- net/core/dev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/core/dev.c b/net/core/dev.c index febb23708184..a499c5ffe4a5 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1002,7 +1002,7 @@ struct net_device *dev_get_by_index_rcu(struct net *net, int ifindex) struct net_device *dev; struct hlist_head *head = dev_index_hash(net, ifindex); - hlist_for_each_entry_rcu(dev, head, index_hlist) + hlist_for_each_entry_rcu(dev, head, index_hlist, rcu_read_lock_bh_held()) if (dev->ifindex == ifindex) return dev; From patchwork Wed Jun 9 10:33:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309557 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EC5AC48BCF for ; Wed, 9 Jun 2021 10:33:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7CB761185 for ; Wed, 9 Jun 2021 10:33:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238834AbhFIKfc (ORCPT ); Wed, 9 Jun 2021 06:35:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39008 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238843AbhFIKfb (ORCPT ); Wed, 9 Jun 2021 06:35:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lA0E7W0BCCx2QGJzuEGWPZGNlEjsEMEQlaf3J77qZdA=; b=GIsZJhetwqpy6Xd+GC02ykQLt6+I9bU6nHfzqIdGgWmWlT7nFQ6iYsYU4BxrV+ejycuLt/ 03HIcQYRb70JqURQSTzrTYf+g4c50BUe2S3PY0yFzfsoOdAmyg7L9rBqC3EN5GHoM6RMTB lVALND0V8ltVnjiUIYuAhGBiGb9ImCw= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-595-izrsoop_Oe2S7zAi4rmemA-1; Wed, 09 Jun 2021 06:33:35 -0400 X-MC-Unique: izrsoop_Oe2S7zAi4rmemA-1 Received: by mail-ej1-f69.google.com with SMTP id n8-20020a1709067b48b02904171dc68f87so3337875ejo.21 for ; Wed, 09 Jun 2021 03:33:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lA0E7W0BCCx2QGJzuEGWPZGNlEjsEMEQlaf3J77qZdA=; b=WH1FKGH3yPXC7iSNTlkXVVdkJGnkJxNwg4ds+wqlShwXNCIOG62LUAuNBKAJK3vCOa e/poJzNB1d+MNk2H8SHuZtvIrUQBeVrKOXZJnIvIea6g7sspYu41Ia0T1StoW8Zd/9DO wv2/Ndu0H20Qxg4Jld5O/5981hgMlBUnW8w4s4nmeETV8FAFLIhi0MaHq3vJjv/+6A4c mru3YDhT7bwlkSEXlvDlsvtyXkD9eLv6Ow3ad61lLBsP13S4QAOsEVNNjXzn2ZL+rvaO L72m5QiN1E/nzICfR/YFGXnFqFjsMAiqWpQSs69hY9GAHbYIUBBEjM4knaeMc3S87I+1 0IXQ== X-Gm-Message-State: AOAM532x0D47r8a19mO3Ei94dy99pgfQzGhOhcPFxrYF2Ohaek5/NZ+/ oYp7gagRxzf/Mljx/lk308+2DrFfSzYj+7aFUpiZ/G/heQ72OW8FMl6VyB0z+Z41fuBiOO0AOCy KGQjgfMb716Dk X-Received: by 2002:aa7:dbc9:: with SMTP id v9mr30177814edt.183.1623234812383; Wed, 09 Jun 2021 03:33:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyGRmshimkWPt0vvV7LWlfx0oXRF96qtbwlTJ9zkDBZ31R4IV/OKfYJzl8wx0QQOOV3l8TwQw== X-Received: by 2002:aa7:dbc9:: with SMTP id v9mr30177780edt.183.1623234811943; Wed, 09 Jun 2021 03:33:31 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id e25sm937431eja.15.2021.06.09.03.33.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:31 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 77442180728; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= Subject: [PATCH bpf-next 04/17] xdp: add proper __rcu annotations to redirect map entries Date: Wed, 9 Jun 2021 12:33:13 +0200 Message-Id: <20210609103326.278782-5-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net XDP_REDIRECT works by a three-step process: the bpf_redirect() and bpf_redirect_map() helpers will lookup the target of the redirect and store it (along with some other metadata) in a per-CPU struct bpf_redirect_info. Next, when the program returns the XDP_REDIRECT return code, the driver will call xdp_do_redirect() which will use the information thus stored to actually enqueue the frame into a bulk queue structure (that differs slightly by map type, but shares the same principle). Finally, before exiting its NAPI poll loop, the driver will call xdp_do_flush(), which will flush all the different bulk queues, thus completing the redirect. Pointers to the map entries will be kept around for this whole sequence of steps, protected by RCU. However, there is no top-level rcu_read_lock() in the core code; instead drivers add their own rcu_read_lock() around the XDP portions of the code, but somewhat inconsistently as Martin discovered[0]. However, things still work because everything happens inside a single NAPI poll sequence, which means it's between a pair of calls to local_bh_disable()/local_bh_enable(). So Paul suggested[1] that we could document this intention by using rcu_dereference_check() with rcu_read_lock_bh_held() as a second parameter, thus allowing sparse and lockdep to verify that everything is done correctly. This patch does just that: we add an __rcu annotation to the map entry pointers and remove the various comments explaining the NAPI poll assurance strewn through devmap.c in favour of a longer explanation in filter.c. The goal is to have one coherent documentation of the entire flow, and rely on the RCU annotations as a "standard" way of communicating the flow in the map code (which can additionally be understood by sparse and lockdep). The RCU annotation replacements result in a fairly straight-forward replacement where READ_ONCE() becomes rcu_dereference_check(), WRITE_ONCE() becomes rcu_assign_pointer() and xchg() and cmpxchg() gets wrapped in the proper constructs to cast the pointer back and forth between __rcu and __kernel address space (for the benefit of sparse). The one complication is that xskmap has a few constructions where double-pointers are passed back and forth; these simply all gain __rcu annotations, and only the final reference/dereference to the inner-most pointer gets changed. With this, everything can be run through sparse without eliciting complaints, and lockdep can verify correctness even without the use of rcu_read_lock() in the drivers. Subsequent patches will clean these up from the drivers. [0] https://lore.kernel.org/bpf/20210415173551.7ma4slcbqeyiba2r@kafai-mbp.dhcp.thefacebook.com/ [1] https://lore.kernel.org/bpf/20210419165837.GA975577@paulmck-ThinkPad-P17-Gen-1/ Signed-off-by: Toke Høiland-Jørgensen --- include/net/xdp_sock.h | 2 +- kernel/bpf/cpumap.c | 14 ++++++++---- kernel/bpf/devmap.c | 52 +++++++++++++++++++----------------------- net/core/filter.c | 28 +++++++++++++++++++++++ net/xdp/xsk.c | 4 ++-- net/xdp/xsk.h | 4 ++-- net/xdp/xskmap.c | 29 +++++++++++++---------- 7 files changed, 83 insertions(+), 50 deletions(-) diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index 9c0722c6d7ac..fff069d2ed1b 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -37,7 +37,7 @@ struct xdp_umem { struct xsk_map { struct bpf_map map; spinlock_t lock; /* Synchronize map updates */ - struct xdp_sock *xsk_map[]; + struct xdp_sock __rcu *xsk_map[]; }; struct xdp_sock { diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index a1a0c4e791c6..bfa94efc5c05 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -74,7 +74,7 @@ struct bpf_cpu_map_entry { struct bpf_cpu_map { struct bpf_map map; /* Below members specific for map type */ - struct bpf_cpu_map_entry **cpu_map; + struct bpf_cpu_map_entry __rcu **cpu_map; }; static DEFINE_PER_CPU(struct list_head, cpu_map_flush_list); @@ -469,7 +469,7 @@ static void __cpu_map_entry_replace(struct bpf_cpu_map *cmap, { struct bpf_cpu_map_entry *old_rcpu; - old_rcpu = xchg(&cmap->cpu_map[key_cpu], rcpu); + old_rcpu = unrcu_pointer(xchg(&cmap->cpu_map[key_cpu], RCU_INITIALIZER(rcpu))); if (old_rcpu) { call_rcu(&old_rcpu->rcu, __cpu_map_entry_free); INIT_WORK(&old_rcpu->kthread_stop_wq, cpu_map_kthread_stop); @@ -551,7 +551,8 @@ static void cpu_map_free(struct bpf_map *map) for (i = 0; i < cmap->map.max_entries; i++) { struct bpf_cpu_map_entry *rcpu; - rcpu = READ_ONCE(cmap->cpu_map[i]); + rcpu = rcu_dereference_check(cmap->cpu_map[i], + rcu_read_lock_bh_held()); if (!rcpu) continue; @@ -562,6 +563,10 @@ static void cpu_map_free(struct bpf_map *map) kfree(cmap); } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_cpu_map *cmap = container_of(map, struct bpf_cpu_map, map); @@ -570,7 +575,8 @@ static void *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - rcpu = READ_ONCE(cmap->cpu_map[key]); + rcpu = rcu_dereference_check(cmap->cpu_map[key], + rcu_read_lock_bh_held()); return rcpu; } diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 2a75e6c2d27d..ae6d9bfeae06 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -73,7 +73,7 @@ struct bpf_dtab_netdev { struct bpf_dtab { struct bpf_map map; - struct bpf_dtab_netdev **netdev_map; /* DEVMAP type only */ + struct bpf_dtab_netdev __rcu **netdev_map; /* DEVMAP type only */ struct list_head list; /* these are only used for DEVMAP_HASH type maps */ @@ -226,7 +226,7 @@ static void dev_map_free(struct bpf_map *map) for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev; - dev = dtab->netdev_map[i]; + dev = rcu_dereference_raw(dtab->netdev_map[i]); if (!dev) continue; @@ -259,6 +259,10 @@ static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) return 0; } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__dev_map_hash_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_dtab *dtab = container_of(map, struct bpf_dtab, map); @@ -266,7 +270,8 @@ static void *__dev_map_hash_lookup_elem(struct bpf_map *map, u32 key) struct bpf_dtab_netdev *dev; hlist_for_each_entry_rcu(dev, head, index_hlist, - lockdep_is_held(&dtab->index_lock)) + (lockdep_is_held(&dtab->index_lock) || + rcu_read_lock_bh_held())) if (dev->idx == key) return dev; @@ -410,15 +415,9 @@ static void bq_xmit_all(struct xdp_dev_bulk_queue *bq, u32 flags) trace_xdp_devmap_xmit(bq->dev_rx, dev, sent, cnt - sent, err); } -/* __dev_flush is called from xdp_do_flush() which _must_ be signaled - * from the driver before returning from its napi->poll() routine. The poll() - * routine is called either from busy_poll context or net_rx_action signaled - * from NET_RX_SOFTIRQ. Either way the poll routine must complete before the - * net device can be torn down. On devmap tear down we ensure the flush list - * is empty before completing to ensure all flush operations have completed. - * When drivers update the bpf program they may need to ensure any flush ops - * are also complete. Using synchronize_rcu or call_rcu will suffice for this - * because both wait for napi context to exit. +/* __dev_flush is called from xdp_do_flush() which _must_ be signalled from the + * driver before returning from its napi->poll() routine. See the comment above + * xdp_do_flush() in filter.c. */ void __dev_flush(void) { @@ -433,9 +432,9 @@ void __dev_flush(void) } } -/* rcu_read_lock (from syscall and BPF contexts) ensures that if a delete and/or - * update happens in parallel here a dev_put won't happen until after reading - * the ifindex. +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. */ static void *__dev_map_lookup_elem(struct bpf_map *map, u32 key) { @@ -445,12 +444,14 @@ static void *__dev_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - obj = READ_ONCE(dtab->netdev_map[key]); + obj = rcu_dereference_check(dtab->netdev_map[key], + rcu_read_lock_bh_held()); return obj; } -/* Runs under RCU-read-side, plus in softirq under NAPI protection. - * Thus, safe percpu variable access. +/* Runs in NAPI, i.e., softirq under local_bh_disable(). Thus, safe percpu + * variable access, and map elements stick around. See comment above + * xdp_do_flush() in filter.c. */ static void bq_enqueue(struct net_device *dev, struct xdp_frame *xdpf, struct net_device *dev_rx, struct bpf_prog *xdp_prog) @@ -735,14 +736,7 @@ static int dev_map_delete_elem(struct bpf_map *map, void *key) if (k >= map->max_entries) return -EINVAL; - /* Use call_rcu() here to ensure any rcu critical sections have - * completed as well as any flush operations because call_rcu - * will wait for preempt-disable region to complete, NAPI in this - * context. And additionally, the driver tear down ensures all - * soft irqs are complete before removing the net device in the - * case of dev_put equals zero. - */ - old_dev = xchg(&dtab->netdev_map[k], NULL); + old_dev = unrcu_pointer(xchg(&dtab->netdev_map[k], NULL)); if (old_dev) call_rcu(&old_dev->rcu, __dev_map_entry_free); return 0; @@ -851,7 +845,7 @@ static int __dev_map_update_elem(struct net *net, struct bpf_map *map, * Remembering the driver side flush operation will happen before the * net device is removed. */ - old_dev = xchg(&dtab->netdev_map[i], dev); + old_dev = unrcu_pointer(xchg(&dtab->netdev_map[i], RCU_INITIALIZER(dev))); if (old_dev) call_rcu(&old_dev->rcu, __dev_map_entry_free); @@ -1031,10 +1025,10 @@ static int dev_map_notification(struct notifier_block *notifier, for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev, *odev; - dev = READ_ONCE(dtab->netdev_map[i]); + dev = rcu_dereference(dtab->netdev_map[i]); if (!dev || netdev != dev->dev) continue; - odev = cmpxchg(&dtab->netdev_map[i], dev, NULL); + odev = unrcu_pointer(cmpxchg(&dtab->netdev_map[i], RCU_INITIALIZER(dev), NULL)); if (dev == odev) call_rcu(&dev->rcu, __dev_map_entry_free); diff --git a/net/core/filter.c b/net/core/filter.c index caa88955562e..0b7db5c70385 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -3922,6 +3922,34 @@ static const struct bpf_func_proto bpf_xdp_adjust_meta_proto = { .arg2_type = ARG_ANYTHING, }; +/* XDP_REDIRECT works by a three-step process, implemented in the functions + * below: + * + * 1. The bpf_redirect() and bpf_redirect_map() helpers will lookup the target + * of the redirect and store it (along with some other metadata) in a per-CPU + * struct bpf_redirect_info. + * + * 2. When the program returns the XDP_REDIRECT return code, the driver will + * call xdp_do_redirect() which will use the information in struct + * bpf_redirect_info to actually enqueue the frame into a map type-specific + * bulk queue structure. + * + * 3. Before exiting its NAPI poll loop, the driver will call xdp_do_flush(), + * which will flush all the different bulk queues, thus completing the + * redirect. + * + * Pointers to the map entries will be kept around for this whole sequence of + * steps, protected by RCU. However, there is no top-level rcu_read_lock() in + * the core code; instead, the RCU protection relies on everything happening + * inside a single NAPI poll sequence, which means it's between a pair of calls + * to local_bh_disable()/local_bh_enable(). + * + * The map entries are marked as __rcu and the map code makes sure to + * dereference those pointers with rcu_dereference_check() in a way that works + * for both sections that to hold an rcu_read_lock() and sections that are + * called from NAPI without a separate rcu_read_lock(). The code below does not + * use RCU annotations, but relies on those in the map code. + */ void xdp_do_flush(void) { __dev_flush(); diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index cd62d4ba87a9..996da915f520 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -749,7 +749,7 @@ static void xsk_unbind_dev(struct xdp_sock *xs) } static struct xsk_map *xsk_get_map_list_entry(struct xdp_sock *xs, - struct xdp_sock ***map_entry) + struct xdp_sock __rcu ***map_entry) { struct xsk_map *map = NULL; struct xsk_map_node *node; @@ -785,7 +785,7 @@ static void xsk_delete_from_maps(struct xdp_sock *xs) * might be updates to the map between * xsk_get_map_list_entry() and xsk_map_try_sock_delete(). */ - struct xdp_sock **map_entry = NULL; + struct xdp_sock __rcu **map_entry = NULL; struct xsk_map *map; while ((map = xsk_get_map_list_entry(xs, &map_entry))) { diff --git a/net/xdp/xsk.h b/net/xdp/xsk.h index edcf249ad1f1..a4bc4749faac 100644 --- a/net/xdp/xsk.h +++ b/net/xdp/xsk.h @@ -31,7 +31,7 @@ struct xdp_mmap_offsets_v1 { struct xsk_map_node { struct list_head node; struct xsk_map *map; - struct xdp_sock **map_entry; + struct xdp_sock __rcu **map_entry; }; static inline struct xdp_sock *xdp_sk(struct sock *sk) @@ -40,7 +40,7 @@ static inline struct xdp_sock *xdp_sk(struct sock *sk) } void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, - struct xdp_sock **map_entry); + struct xdp_sock __rcu **map_entry); void xsk_clear_pool_at_qid(struct net_device *dev, u16 queue_id); int xsk_reg_pool_at_qid(struct net_device *dev, struct xsk_buff_pool *pool, u16 queue_id); diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 9df75ea4a567..a754262779ba 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -12,7 +12,7 @@ #include "xsk.h" static struct xsk_map_node *xsk_map_node_alloc(struct xsk_map *map, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { struct xsk_map_node *node; @@ -42,7 +42,7 @@ static void xsk_map_sock_add(struct xdp_sock *xs, struct xsk_map_node *node) } static void xsk_map_sock_delete(struct xdp_sock *xs, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { struct xsk_map_node *n, *tmp; @@ -124,6 +124,10 @@ static int xsk_map_gen_lookup(struct bpf_map *map, struct bpf_insn *insn_buf) return insn - insn_buf; } +/* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or + * by local_bh_disable() (from XDP calls inside NAPI). The + * rcu_read_lock_bh_held() below makes lockdep accept both. + */ static void *__xsk_map_lookup_elem(struct bpf_map *map, u32 key) { struct xsk_map *m = container_of(map, struct xsk_map, map); @@ -131,12 +135,11 @@ static void *__xsk_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - return READ_ONCE(m->xsk_map[key]); + return rcu_dereference_check(m->xsk_map[key], rcu_read_lock_bh_held()); } static void *xsk_map_lookup_elem(struct bpf_map *map, void *key) { - WARN_ON_ONCE(!rcu_read_lock_held()); return __xsk_map_lookup_elem(map, *(u32 *)key); } @@ -149,7 +152,8 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags) { struct xsk_map *m = container_of(map, struct xsk_map, map); - struct xdp_sock *xs, *old_xs, **map_entry; + struct xdp_sock __rcu **map_entry; + struct xdp_sock *xs, *old_xs; u32 i = *(u32 *)key, fd = *(u32 *)value; struct xsk_map_node *node; struct socket *sock; @@ -179,7 +183,7 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, } spin_lock_bh(&m->lock); - old_xs = READ_ONCE(*map_entry); + old_xs = rcu_dereference_check(*map_entry, rcu_read_lock_bh_held()); if (old_xs == xs) { err = 0; goto out; @@ -191,7 +195,7 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, goto out; } xsk_map_sock_add(xs, node); - WRITE_ONCE(*map_entry, xs); + rcu_assign_pointer(*map_entry, xs); if (old_xs) xsk_map_sock_delete(old_xs, map_entry); spin_unlock_bh(&m->lock); @@ -208,7 +212,8 @@ static int xsk_map_update_elem(struct bpf_map *map, void *key, void *value, static int xsk_map_delete_elem(struct bpf_map *map, void *key) { struct xsk_map *m = container_of(map, struct xsk_map, map); - struct xdp_sock *old_xs, **map_entry; + struct xdp_sock __rcu **map_entry; + struct xdp_sock *old_xs; int k = *(u32 *)key; if (k >= map->max_entries) @@ -216,7 +221,7 @@ static int xsk_map_delete_elem(struct bpf_map *map, void *key) spin_lock_bh(&m->lock); map_entry = &m->xsk_map[k]; - old_xs = xchg(map_entry, NULL); + old_xs = unrcu_pointer(xchg(map_entry, NULL)); if (old_xs) xsk_map_sock_delete(old_xs, map_entry); spin_unlock_bh(&m->lock); @@ -231,11 +236,11 @@ static int xsk_map_redirect(struct bpf_map *map, u32 ifindex, u64 flags) } void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, - struct xdp_sock **map_entry) + struct xdp_sock __rcu **map_entry) { spin_lock_bh(&map->lock); - if (READ_ONCE(*map_entry) == xs) { - WRITE_ONCE(*map_entry, NULL); + if (rcu_dereference(*map_entry) == xs) { + rcu_assign_pointer(*map_entry, NULL); xsk_map_sock_delete(xs, map_entry); } spin_unlock_bh(&map->lock); From patchwork Wed Jun 9 10:33:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309555 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6E86C48BDF for ; Wed, 9 Jun 2021 10:33:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B16EF61375 for ; Wed, 9 Jun 2021 10:33:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238843AbhFIKfd (ORCPT ); Wed, 9 Jun 2021 06:35:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45809 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237099AbhFIKfb (ORCPT ); Wed, 9 Jun 2021 06:35:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I6RFVrIlNJV4mNL6ANvqt/Qw3/Fb8/fimwbEL77gRdY=; b=Se/Jnr20uz842cvy5EzrPDhTqyUpEMx/0VYixupQpQMpNGrUIJAp3nEQgsnnj0TNLA2RWU qTumO9CSva93AGmTCJMHIm3A8nS0RKMAGEdd/Jmfkt1W/4r47SuVaHnGVpT7ebelsrWyNo YtHsHq1Zn+fRR3PICuXXhvZecB6Qddk= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-277-YyiJCZ-LPjWnWRccwhywJQ-1; Wed, 09 Jun 2021 06:33:35 -0400 X-MC-Unique: YyiJCZ-LPjWnWRccwhywJQ-1 Received: by mail-ed1-f71.google.com with SMTP id t11-20020a056402524bb029038ffacf1cafso2860371edd.5 for ; Wed, 09 Jun 2021 03:33:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I6RFVrIlNJV4mNL6ANvqt/Qw3/Fb8/fimwbEL77gRdY=; b=rfcG6jkXWIgzDYE8Y2KKdCHF7YhofkMkDtQrTi24pQ62EwJ62NAFIg+ms24by1NEMU DHAmFBPW+sGMthujnVxDk9liJHeGav7snaSYKbC9SS5D0PuBmdL/FaR+kVZTxkOcTtBh tjznNMlFsk9esyaLzPzqKrvIfXA+SLEiEhgttQKbGKJ9lkCsC35vaFv2wc02x7bIfjV1 oYYiOmiT06UUU3qm6zQUNvGsmmgniUEtjSHYnYYL4c5SU+VU9rjzSFR5QsfTTG3NrDx+ ZaVZPcdU1ZJu4jm9uU5b/7BkSX5ncg3wRVOZWgRv9GU+Nsb8ZvvpUPHET7fn8y5xN3v6 JkJw== X-Gm-Message-State: AOAM533cRpeR43veJMxpKinrumrHHluFmIuTIS1KrlLt0TT6/L1IoDzm SfIZtM+1TSvlht/yHsxs1x8pifK1DlUKMkj8yjVpa4czMEClkF60owQSzlrmghdmT64R69AD7EU VtiBtJwTwL/bN X-Received: by 2002:a17:906:1792:: with SMTP id t18mr16230746eje.38.1623234814355; Wed, 09 Jun 2021 03:33:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxSS/+KNKnXT0QQpuEWI8iYGTn6gTZHYpHRtIhPO3wIfSXPTo2W2Il/Qh2CqA4beAkAqr+45A== X-Received: by 2002:a17:906:1792:: with SMTP id t18mr16230736eje.38.1623234814221; Wed, 09 Jun 2021 03:33:34 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id cx7sm967596edb.65.2021.06.09.03.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 7FE50180729; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Guy Tzalik , Saeed Bishara Subject: [PATCH bpf-next 05/17] ena: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:14 +0200 Message-Id: <20210609103326.278782-6-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The ena driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Guy Tzalik Cc: Saeed Bishara Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 881f88754bf6..a4378b14af4c 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -385,7 +385,6 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp) u64 *xdp_stat; int qid; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_bpf_prog); if (!xdp_prog) @@ -443,8 +442,6 @@ static int ena_xdp_execute(struct ena_ring *rx_ring, struct xdp_buff *xdp) ena_increase_stat(xdp_stat, 1, &rx_ring->syncp); out: - rcu_read_unlock(); - return verdict; } From patchwork Wed Jun 9 10:33:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309553 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7219C48BE5 for ; Wed, 9 Jun 2021 10:33:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AABAC61375 for ; Wed, 9 Jun 2021 10:33:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238901AbhFIKfb (ORCPT ); Wed, 9 Jun 2021 06:35:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29035 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238834AbhFIKfa (ORCPT ); Wed, 9 Jun 2021 06:35:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RLm94AjK7kFd76KBYEVaqEhzQehsWkzr0FHrLWea2VI=; b=i6ZYOnWdXrZdxa3duUjJDjOHj6Buns74bNbMYSWdqXdsShlGXWwqQT8k4+2V4c5VRPxRi0 ByJYYXcePjFft+VaJZ8QP+R774RC1ep5StSDsZuvNla90wHtTrAStuMzBqo5G0mZfYR/K7 J2elTHDnyrpFBkYyK5dnwvOSJmqi9BA= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-589-5YA3Ckl2NXm2eD5_4W9D9Q-1; Wed, 09 Jun 2021 06:33:34 -0400 X-MC-Unique: 5YA3Ckl2NXm2eD5_4W9D9Q-1 Received: by mail-ej1-f72.google.com with SMTP id p18-20020a1709067852b02903dab2a3e1easo7854691ejm.17 for ; Wed, 09 Jun 2021 03:33:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RLm94AjK7kFd76KBYEVaqEhzQehsWkzr0FHrLWea2VI=; b=mO/sDC+k6eGDaOhd4Y4RbuFuMklS35TJpkt8ncnAfEsLoCwmZ7j1gSpBcv4cw/VrBq AM4bLSVLd86l34fWqqK2s/B+FNTFWtSaH9ddeeJuSwnXQlF2vW+i2M2t/GHWTcjSrAjz Xn+l5nzRU0ozQ9hdrI72SMqNjmLI2crAs1kvI351heomEDXFXW1o50O5j4DMsWtn8gqY 8CRnHnf4vt9ndZlxET6zNxQkFccJwk+s9xxz3TsM9IPyhPMWQEJikFBeAY5c64lg5Kk3 czyds4RntdtTy3T8GoOlvvASI1zIlMSRPNVfCgI4du7/ilz5Bu5e8/6rEi+mo8rXFrpA BsYw== X-Gm-Message-State: AOAM531eU7nCTRsOpd8IObVLfEiSs3iY4weuMzCN4nNrlgtzYXeYqG6f 4w0ZzVuwqtibZvFvfbggIwglK9u0KPiREKECQTB9Nw8zPp5oFqAmhF5ERGIFEHf/u1nuYAmfSdE ktsKHHwj7e/Br X-Received: by 2002:aa7:d5d6:: with SMTP id d22mr29821026eds.302.1623234813634; Wed, 09 Jun 2021 03:33:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyWc5YApHB9hF9X+ZdssWz2+d9p4i4aY5gUchXQSx4FgGgw0Nu4lE3bZz5FFK+6vN62FRE6Lg== X-Received: by 2002:aa7:d5d6:: with SMTP id d22mr29821013eds.302.1623234813449; Wed, 09 Jun 2021 03:33:33 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id x4sm953169edq.23.2021.06.09.03.33.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 8A97418072A; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Michael Chan Subject: [PATCH bpf-next 06/17] bnxt: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:15 +0200 Message-Id: <20210609103326.278782-7-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The bnxt driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Michael Chan Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c index ec9564e584e0..bee6e091a997 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c @@ -138,9 +138,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons, xdp_prepare_buff(&xdp, *data_ptr - offset, offset, *len, false); orig_data = xdp.data; - rcu_read_lock(); act = bpf_prog_run_xdp(xdp_prog, &xdp); - rcu_read_unlock(); tx_avail = bnxt_tx_avail(bp, txr); /* If the tx ring is not full, we must not update the rx producer yet From patchwork Wed Jun 9 10:33:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309563 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10494C48BCF for ; Wed, 9 Jun 2021 10:33:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA8CA61375 for ; Wed, 9 Jun 2021 10:33:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238872AbhFIKfu (ORCPT ); Wed, 9 Jun 2021 06:35:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41380 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238912AbhFIKfe (ORCPT ); Wed, 9 Jun 2021 06:35:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VOVdIjFGkWIiDQLhZ4O8JX1OgtVYddfMXm2FFCs+0ZU=; b=Ca+CgXLqL1676s5pbIP9mnV67820stEjcwqqQVbWs0VHPDGkoghUQSLqjy9bwd5hL5R0pp +Za0wsNL7tWjq3Qko2ZekeoDDAHRF5srMZeFezlApP0zLiHYf9GMfUu9MOo8yFk3rXzscU 6t68vOwbRbbycIUOeGS7gHS+PXQldZ8= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-397-CDuaDmSnNYeJDLIrm6AZzw-1; Wed, 09 Jun 2021 06:33:38 -0400 X-MC-Unique: CDuaDmSnNYeJDLIrm6AZzw-1 Received: by mail-ej1-f72.google.com with SMTP id 16-20020a1709063010b029037417ca2d43so7842585ejz.5 for ; Wed, 09 Jun 2021 03:33:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VOVdIjFGkWIiDQLhZ4O8JX1OgtVYddfMXm2FFCs+0ZU=; b=N9IRV1sfzVSar8VNBDHmoFleygGEUbLcRbx2zlakFQQIvxduqJAJeO5lwcyvRShMOZ +qjacmzcPx8/CyQcT37txnD34/dXF37pSHGazMGf9mhLMbYvsh2R1H8QVXpXPQcvfTn5 EjRmxMeee7CPeLa6L2b9N1z4cNkjgcuscTRbdqrVw++nVWgYhcnYdNt9rBMEtH5KhItO cQ3YD3dPPQ9NGynYTuqln7cDQtsXTa0jUWj1MW4N3l+aHvCMSAozGoWgo0FrrZhSvIgR A97IPdPrIAz8P6m7QxZ368UAnEM1oNITToCuocuBJSHUTNtUNDSB6g2cxhMXZrEmDAE4 1ZrQ== X-Gm-Message-State: AOAM532/UDIhdM3Qk/1WUA79MA5Nskv3a5xFc4SCdNMOq1whDkfCTtYq oI74tk3S9o6ODSUpVWfANSKWZJz48KdLzAdsYWAvX7qt9srKfwvTHfOKladNYp7+UctdyOupM+F CaFKtXBu4ZvTn X-Received: by 2002:aa7:c547:: with SMTP id s7mr29599535edr.239.1623234816147; Wed, 09 Jun 2021 03:33:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJsMQfvKo7/nhkmRlLMchZWxJh31qEpOl87eLrNsvu1Nhy2NQKMbTQafLxov+eQh4WGHJZIQ== X-Received: by 2002:aa7:c547:: with SMTP id s7mr29599525edr.239.1623234816007; Wed, 09 Jun 2021 03:33:36 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id c6sm916307eje.9.2021.06.09.03.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:32 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 95EA618072B; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Sunil Goutham , linux-arm-kernel@lists.infradead.org Subject: [PATCH bpf-next 07/17] thunderx: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:16 +0200 Message-Id: <20210609103326.278782-8-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The thunderx driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Sunil Goutham Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/cavium/thunder/nicvf_main.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index c33b4e837515..e2b290135fd9 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -555,9 +555,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog, xdp_prepare_buff(&xdp, hard_start, data - hard_start, len, false); orig_data = xdp.data; - rcu_read_lock(); action = bpf_prog_run_xdp(prog, &xdp); - rcu_read_unlock(); len = xdp.data_end - xdp.data; /* Check if XDP program has changed headers */ From patchwork Wed Jun 9 10:33:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309565 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99D35C48BCD for ; Wed, 9 Jun 2021 10:33:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 848E261375 for ; Wed, 9 Jun 2021 10:33:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238909AbhFIKfu (ORCPT ); Wed, 9 Jun 2021 06:35:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:56087 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238913AbhFIKfe (ORCPT ); Wed, 9 Jun 2021 06:35:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OXscz2Zggz3CM5+O8rho39EJROdLmluUAHhgL+NUYOY=; b=Mh+Dp2m31HM9pRi63ZtCVgG226K0MGDJzY37TOoy7RT6Lh3rXUU6x1vBSRG6jGKgryFhwP tieWSTHkhX6yvFp9sdFbZmE0+NiuqyF6ighk/3u7EJfDxY+MTHnfQ3LLp9u/W5M+zR60gG 4DXXm7njX05eydkE5o9dE1ffRSMoX7Y= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-35-YtWRjIshPsyvk8ks8rBrAA-1; Wed, 09 Jun 2021 06:33:38 -0400 X-MC-Unique: YtWRjIshPsyvk8ks8rBrAA-1 Received: by mail-ej1-f72.google.com with SMTP id z15-20020a1709063a0fb029040d43ca6e95so4811994eje.12 for ; Wed, 09 Jun 2021 03:33:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OXscz2Zggz3CM5+O8rho39EJROdLmluUAHhgL+NUYOY=; b=nDjGoGJbbmc6e7Iup0GmSIG9xMZUwuh8tepxPBnz7NRmis0UYAg5S+7Dkv0LQB8zeR sFlMHWz0E4vNX/XsB829iRK1EELkoVnVN4Dg5VPTTeWpojru3WmiLBDB9+75mqeFe+4B 6vK82r6Up6kEzdg7w++vo6/fQQXh7qXcYHM7ERmvL85xt8O3IwvKKTv9xLzjn9iH5kie K3xYOvDCmcK7qPUr4EQbaGXbU53mF/l9/qE/AIIe+oKoADtKo1oNwW3h+yBAynDDgVzp JKykBClSaxcyTMZGEGBgVz5duAaVtBrJnkf+4tXbkUavLdd857OSPeoOZpLsXncTRzaD TvKQ== X-Gm-Message-State: AOAM530GRv1uWgdS3B5MJqvfS2pG/1Bhh807uDW6/EBNLXHuA8RSx06e h05DQP2+IV76pIxq6jdoB7xM2ly4kkVt5FuJGCAZfbPTHUHF5SV2gR6DjMoo+SfmYHKNHRt5hd2 M9y7LE/GseaEJ X-Received: by 2002:a17:906:4d56:: with SMTP id b22mr26898792ejv.78.1623234816996; Wed, 09 Jun 2021 03:33:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzwCDKewP7XjcGyNfolErrcPHt/2wWqDkF2NBhrGIQsrtCaHXarMEITcengMHqcuVdMIj+HOg== X-Received: by 2002:a17:906:4d56:: with SMTP id b22mr26898769ejv.78.1623234816671; Wed, 09 Jun 2021 03:33:36 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id l8sm974784eds.79.2021.06.09.03.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:35 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id A1AA218072C; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Madalin Bucur , Ioana Ciornei , Ioana Radulescu Subject: [PATCH bpf-next 08/17] freescale: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:17 +0200 Message-Id: <20210609103326.278782-9-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The dpaa and dpaa2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Madalin Bucur Cc: Ioana Ciornei Cc: Ioana Radulescu Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 8 +------- drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 3 --- 2 files changed, 1 insertion(+), 10 deletions(-) diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c index 177c020bf34a..e6826561cf11 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c @@ -2558,13 +2558,9 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, u32 xdp_act; int err; - rcu_read_lock(); - xdp_prog = READ_ONCE(priv->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) return XDP_PASS; - } xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE, &dpaa_fq->xdp_rxq); @@ -2638,8 +2634,6 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, break; } - rcu_read_unlock(); - return xdp_act; } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 8433aa730c42..973352393bd4 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -352,8 +352,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, u32 xdp_act = XDP_PASS; int err, offset; - rcu_read_lock(); - xdp_prog = READ_ONCE(ch->xdp.prog); if (!xdp_prog) goto out; @@ -414,7 +412,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, ch->xdp.res |= xdp_act; out: - rcu_read_unlock(); return xdp_act; } From patchwork Wed Jun 9 10:33:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309561 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B51B5C48BCF for ; Wed, 9 Jun 2021 10:33:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9DEAD61375 for ; Wed, 9 Jun 2021 10:33:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238851AbhFIKff (ORCPT ); Wed, 9 Jun 2021 06:35:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60662 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238846AbhFIKfc (ORCPT ); Wed, 9 Jun 2021 06:35:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234818; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wLi6aGzthW1V7ZnrfDoybEkcdwTKHnTji0dPASh8hrc=; b=eCXQGVMUesU2+3TDZrxbhv/NVMe4NUdAsqA8qrn1s4XCxsgmXIgpCryyJImWKKjd1YFLis AO7st/g5SPJ8ETiA0wPCxyqcNzJAEribWmrNd230iDxfrqmrFI3YSlQJzz/kIkBCmj7poS QnnvWAK8NpH2hspf7fw1nPQG6BUsSVc= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-574-LWHHm37uM_a8GMp4NXdWFQ-1; Wed, 09 Jun 2021 06:33:37 -0400 X-MC-Unique: LWHHm37uM_a8GMp4NXdWFQ-1 Received: by mail-ej1-f71.google.com with SMTP id gv42-20020a1709072beab02903eab8e33118so7828406ejc.19 for ; Wed, 09 Jun 2021 03:33:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wLi6aGzthW1V7ZnrfDoybEkcdwTKHnTji0dPASh8hrc=; b=BjgU34Eb6KGKFeEPJ3HtEnh/a4LedQWmBr5r9nDDI6SH2UuWOcGSh5p9oLzHcmoVXI 9LqPT0e3G/wS/u7aujIQHXE7VDKfVpxR9VM3MGzx4s/jk9A6XM0F4hzvgUtDh3Pbi/Lt 8cHspb+ArEUqmgoaOm2XhkHvgO+WxAyNdtARscgJqnOJ+s1IUYyphF0qTS5pKCLC6Nfa 2i41yarBYqCsaJvcAFNz3XFwQjUVIgi8q5uQqfhgsbd5z0PXg3kWyJzYTRWzeS/M30tA hDiPFypBiaem1jnesgKVw0tZiBNr3XS40UgkWQOZ5ePbV7dEykIdyd0FJqteueR8w7My n+fQ== X-Gm-Message-State: AOAM5334585liaDJyYC6fqGrh/ZLmcykKfWXuGGs1FYBeZmmWomMgZA3 HYmyZnbN6tLn5dIRRlgpCx7MF6FgJa/k8/8c/dxxjze6XlM9Y4pWCzuU4KFU+pnUPC6RFKXg4Z+ Quu0k9hZn/2PK X-Received: by 2002:a05:6402:1d38:: with SMTP id dh24mr30493423edb.18.1623234816098; Wed, 09 Jun 2021 03:33:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxYCDMsQSZ9Dlij0MXgtUcDtV0HyvIjlYbVvq1Bssh7pKSh/y6kbcDB+aG4fWOwL/laWQyutw== X-Received: by 2002:a05:6402:1d38:: with SMTP id dh24mr30493408edb.18.1623234815929; Wed, 09 Jun 2021 03:33:35 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id m15sm926518eji.39.2021.06.09.03.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:35 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id AB08318072D; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Jesse Brandeburg , Tony Nguyen , intel-wired-lan@lists.osuosl.org Subject: [PATCH bpf-next 09/17] net: intel: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:18 +0200 Message-Id: <20210609103326.278782-10-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The Intel drivers all have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Jesse Brandeburg Cc: Tony Nguyen Cc: intel-wired-lan@lists.osuosl.org Tested-by: Jesper Dangaard Brouer # i40e Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 2 -- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 6 +----- drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +----- drivers/net/ethernet/intel/ice/ice_xsk.c | 6 +----- drivers/net/ethernet/intel/igb/igb_main.c | 2 -- drivers/net/ethernet/intel/igc/igc_main.c | 7 ++----- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 2 -- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 6 +----- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 2 -- 9 files changed, 6 insertions(+), 33 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index de70c16ef619..ae3a64b6f5f8 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -2298,7 +2298,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2329,7 +2328,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) break; } xdp_out: - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 46d884417c63..8dca53b7daff 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -153,7 +153,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); /* NB! xdp_prog will always be !NULL, due to the fact that * this path is enabled by setting an XDP program. */ @@ -162,9 +161,7 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED; } switch (act) { @@ -184,7 +181,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) result = I40E_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index e2b4b29ea207..1a311e91fb6d 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -1129,15 +1129,11 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size); #endif - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) goto construct_skb; - } xdp_res = ice_run_xdp(rx_ring, &xdp, xdp_prog); - rcu_read_unlock(); if (!xdp_res) goto construct_skb; if (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) { diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index faa7b8d96adb..d6da377f5ac3 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -463,7 +463,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) struct ice_ring *xdp_ring; u32 act; - rcu_read_lock(); /* ZC patch is enabled only when XDP program is set, * so here it can not be NULL */ @@ -473,9 +472,7 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED; } switch (act) { @@ -496,7 +493,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 038a9fd1af44..8a11b7e55326 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -8387,7 +8387,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -8420,7 +8419,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index ea998d2defa4..2b666a6ec989 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -2175,18 +2175,15 @@ static struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(adapter->xdp_prog); if (!prog) { res = IGC_XDP_PASS; - goto unlock; + goto out; } res = __igc_xdp_run_prog(adapter, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index c5ec17d19c59..27d7467534e0 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -2199,7 +2199,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2237,7 +2236,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index 91ad5b902673..ffbf8a694362 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -100,15 +100,12 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); act = bpf_prog_run_xdp(xdp_prog, xdp); if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; } switch (act) { @@ -132,7 +129,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, result = IXGBE_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index ba2ed8a43d2d..fabada4ce315 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -1054,7 +1054,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -1079,7 +1078,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } From patchwork Wed Jun 9 10:33:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309601 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2FCAC48BDF for ; Wed, 9 Jun 2021 10:38:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE054613AC for ; Wed, 9 Jun 2021 10:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238991AbhFIKkr (ORCPT ); Wed, 9 Jun 2021 06:40:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56002 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238985AbhFIKkn (ORCPT ); Wed, 9 Jun 2021 06:40:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623235129; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XVpjMkk8Gj+sNRZ7mlrzY9T8oghFils6hen5lUAvCOo=; b=VgiG2Y5bo0z0g/O3irNoy7OGc3mzhtH4aE1Jq5kDw/j3MaepBWYeWQqtRrAcErqM+mPbqi ls+eiqIioPPWR3CU+Jm0opYkVQtvnsf7osXQhoDtx+6K7FDRMxSJWhGOBOAdBJJjwIDr/B 49o8EubSKIeXR0kqSvwKE0rDErmaF6g= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-52-wwozpe7xMzmYO1AegZ_Dow-1; Wed, 09 Jun 2021 06:38:45 -0400 X-MC-Unique: wwozpe7xMzmYO1AegZ_Dow-1 Received: by mail-ej1-f70.google.com with SMTP id p18-20020a1709067852b02903dab2a3e1easo7860092ejm.17 for ; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XVpjMkk8Gj+sNRZ7mlrzY9T8oghFils6hen5lUAvCOo=; b=Ua7euNbYubG9xvGdXsVCBGIlqJtg8MEaOk3zOjMWycSSF2dTtQPZpoXoRiqSSqXYk2 iGGXTkFTo3pUeFAgDnfewaAgp5hckB9ydGdbkE0BZSHTPhQexBsFseMkh8vv36zB5qld Kw/DI8YOkh8Pws5YzYQsfm+s96OrHeaAJdQMYeL6nVDibx89L6vfbNjNfF9hEGh0lDV3 wzuSUfecXb6WoWfDHvqWuXsmcr7glGF45rXFdE4Ho+ZvSWcOIbFJqdZnWnmHoW0eWneA z4F3jH1/1Xy9+lF0ZdJNIWWCOYbHHMOcPdxhn4AoAIES424Bw5h5GqtS9atdQQQ8D0+i Bvjw== X-Gm-Message-State: AOAM533o9wS0CgIAnwdcvm09DuVGgfqPMg8caZevPqYNUuffh/KL6Pad WSjkQ2TOGww71pXF1sNnGaTJNSvU9YqmwXGaspchfVMDr2swJcUKFxqJhdQtroGfXoOSeIiM5nd o0Yu1wSPdnuCY X-Received: by 2002:a05:6402:175b:: with SMTP id v27mr29875917edx.61.1623235124603; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyeg8YZkdSRKAD9U94TGCLmHyCR0FZopyRpYHvGYd6V6x/Dm2yQwQ70S9kEXmHvlcWDOFO/Dw== X-Received: by 2002:a05:6402:175b:: with SMTP id v27mr29875895edx.61.1623235124234; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id p10sm927332ejc.14.2021.06.09.03.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:38:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id B757318072E; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Thomas Petazzoni , Marcin Wojtas , Russell King Subject: [PATCH bpf-next 10/17] marvell: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:19 +0200 Message-Id: <20210609103326.278782-11-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The mvneta and mvpp2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Thomas Petazzoni Cc: Marcin Wojtas Cc: Russell King Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/marvell/mvneta.c | 2 -- drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 4 ---- 2 files changed, 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 7d5cd9bc6c99..b9e5875b20bc 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -2370,7 +2370,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, /* Get number of received packets */ rx_todo = mvneta_rxq_busy_desc_num_get(pp, rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(pp->xdp_prog); /* Fairness NAPI loop */ @@ -2448,7 +2447,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, xdp_buf.data_hard_start = NULL; sinfo.nr_frags = 0; } - rcu_read_unlock(); if (xdp_buf.data_hard_start) mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1); diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index b2259bf1d299..521ed3c1cfe9 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -3852,8 +3852,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, int rx_done = 0; u32 xdp_ret = 0; - rcu_read_lock(); - xdp_prog = READ_ONCE(port->xdp_prog); /* Get number of received packets and clamp the to-do */ @@ -3988,8 +3986,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, mvpp2_bm_pool_put(port, pool, dma_addr, phys_addr); } - rcu_read_unlock(); - if (xdp_ret & MVPP2_XDP_REDIR) xdp_do_flush_map(); From patchwork Wed Jun 9 10:33:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309595 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10AC1C48BD1 for ; Wed, 9 Jun 2021 10:38:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED7B3613AC for ; Wed, 9 Jun 2021 10:38:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238973AbhFIKkl (ORCPT ); Wed, 9 Jun 2021 06:40:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47333 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238420AbhFIKkl (ORCPT ); Wed, 9 Jun 2021 06:40:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623235126; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=13osPXmV51CM+evqtQb1PXhLMNKO0hGWYKSryosHfNU=; b=ZckBBsN8EEH1lVMCcmMP9ssBsvDkUOU5znsVLjx3IdcPuNIDTfZBVOnE9AoH7U0s/5MFUU tORN8gsb527FTJTzgxHIaTNwowkeXzzUO/6Qms8dO1wz3ksmSMIjHIyfrGeNUIVDP8Shpl AD9BSbQIRLrd1sVZgctQJOO8WaoLsI8= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-128-1_AKk2zTPjangPKW3UPxQQ-1; Wed, 09 Jun 2021 06:38:45 -0400 X-MC-Unique: 1_AKk2zTPjangPKW3UPxQQ-1 Received: by mail-ej1-f72.google.com with SMTP id nd10-20020a170907628ab02903a324b229bfso7876696ejc.7 for ; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=13osPXmV51CM+evqtQb1PXhLMNKO0hGWYKSryosHfNU=; b=ZGsY1Fq36qlm0DLWMt1cVScttsEGnwEyFmw2DOo4jlHLCoKkLKnvFTeML08PAYN9W/ lVeaqw+EklnjKZhpDYOPqKoSMUevuJvUKlGXjZ6iHiqoZx95GvFeU2uot1VUN1P4INsJ 1KR7Ud+u+vf35459BA1Xg4ZFOp3kxAh53O7XjixMFFW+rUQgD7WlILmc25+hM5dZXPWc 3uvCq6yvUpfY0y1r15w+oTeHgAXUWnWuPb81ykrRh/ApfNafgIDj9ZetHLm8hiPC2KYD kOgCUjPP5dgc6R+o9nYQq+gWk08/ksWXbI+LhTkasAOzHwRX3tE/VHdYRXiyDOc400bm KWvg== X-Gm-Message-State: AOAM531aqB1JxvhkxVwCJgx9iA09xAyv81yXdzlkUTd+QQiIIZCXVD3p okaL9Us2DI+rtZths0LH7TmFE3r6wzK7N5I7oh+mhYKx5H12v1Q+ch46m3Es2pL7ffwjo57y5UH hRzKxgUjtCDcl X-Received: by 2002:a17:907:270c:: with SMTP id w12mr28462791ejk.175.1623235124379; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx9e5nyoFDDTAOzGAf14fh0+8uA9uZPhafO8NuCa6RdoAIPeTeu87RD7Wd8bQzzXvdc4Qhuag== X-Received: by 2002:a17:907:270c:: with SMTP id w12mr28462775ejk.175.1623235124029; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id dh18sm966980edb.92.2021.06.09.03.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:38:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id BFE8A18072F; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Tariq Toukan Subject: [PATCH bpf-next 11/17] mlx4: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:20 +0200 Message-Id: <20210609103326.278782-12-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The mlx4 driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Also switch the RCU dereferences in the driver loop itself to the _bh variants. Cc: Tariq Toukan Signed-off-by: Toke Høiland-Jørgensen Reviewed-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index e35e4d7ef4d1..3f08c14d0441 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -679,9 +679,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud ring = priv->rx_ring[cq_ring]; - /* Protect accesses to: ring->xdp_prog, priv->mac_hash list */ - rcu_read_lock(); - xdp_prog = rcu_dereference(ring->xdp_prog); + xdp_prog = rcu_dereference_bh(ring->xdp_prog); xdp_init_buff(&xdp, priv->frag_info[0].frag_stride, &ring->xdp_rxq); doorbell_pending = false; @@ -744,7 +742,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud /* Drop the packet, since HW loopback-ed it */ mac_hash = ethh->h_source[MLX4_EN_MAC_HASH_IDX]; bucket = &priv->mac_hash[mac_hash]; - hlist_for_each_entry_rcu(entry, bucket, hlist) { + hlist_for_each_entry_rcu_bh(entry, bucket, hlist) { if (ether_addr_equal_64bits(entry->mac, ethh->h_source)) goto next; @@ -899,8 +897,6 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud break; } - rcu_read_unlock(); - if (likely(polled)) { if (doorbell_pending) { priv->tx_cq[TX_XDP][cq_ring]->xdp_busy = true; From patchwork Wed Jun 9 10:33:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309597 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14DBAC48BCD for ; Wed, 9 Jun 2021 10:38:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB62861375 for ; Wed, 9 Jun 2021 10:38:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238989AbhFIKkp (ORCPT ); Wed, 9 Jun 2021 06:40:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50478 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238977AbhFIKkn (ORCPT ); Wed, 9 Jun 2021 06:40:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623235128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aw/xy7JPGgdtm8YR/KgKtd2Nuh+5odcpRTL2K7ZE9co=; b=J7SHTm+z6j88yiCuuXGdf1rYIh6qPyFTegJmGG0CGLiCZKNqUc+WSVF+9eIzQ+vuCY3RvJ AvrDEogN4PPrXp+LxOxhy5f/NHv0Nj20GMLmiaKgGbzaRQdSVFq9dZo9CIGwxLJixdkcIQ dDJUgLM4LQ8bLKR6TPxuTs5H9+ugcAo= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-220-RXf_g-JDNI-XuOcB0HO-BQ-1; Wed, 09 Jun 2021 06:38:47 -0400 X-MC-Unique: RXf_g-JDNI-XuOcB0HO-BQ-1 Received: by mail-ej1-f72.google.com with SMTP id b10-20020a170906194ab02903ea7d084cd3so7854831eje.1 for ; Wed, 09 Jun 2021 03:38:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aw/xy7JPGgdtm8YR/KgKtd2Nuh+5odcpRTL2K7ZE9co=; b=hnYl/EVccFj8TwH75CkJMsIzljuJ6kUE+ukt0lUfPt9c4pWOjmXgEf1mU7PiqtHNQ0 BsLtqn75qkS5wwKwCwu90iVFex0rW7pLmAPHmkZo7S7OE9vYzz+yS8dJ307utaZ3Oupq KvMJNOjwfPe1vUekLXnu+Wam6Ome0YecfXBGls9yILatJNrlQILxoST8SeltuxHiIzMc JNiCzHM401TeF46CjPL+qqdM7W33D6Gvjw4nmvdBXE6bAv+VPkamcqwd9h0epihxROkr CqRPHNhcTfLYw++emRA2ZhKEUhQzRujDV8yb+lHJsUnL61sbpXHUzo+TQGC3nMDzWHsz Dz5g== X-Gm-Message-State: AOAM532/Y4W2lJK3rafVscJ+Dqyjo7+z/rB3W+u+8vtsRlRQ17R4plq4 QRUVXLwH8swCr4OwRyIJO51UG6RsmyGthLztrDzuzA8B2DyiMfUM3AOCMv1RPjcdIIwGGRlUEP/ m5eKA8skKXngc X-Received: by 2002:a05:6402:42d2:: with SMTP id i18mr29889173edc.168.1623235125901; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx2ZxCPYfzIuixLEMF+JxyB09kt/temZ4xhorHzSmowYUw3Lui4JpY3hhytn+YssOjiCEeufA== X-Received: by 2002:a05:6402:42d2:: with SMTP id i18mr29889153edc.168.1623235125566; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id n13sm989526edx.30.2021.06.09.03.38.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:38:44 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id C980A180730; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Simon Horman , oss-drivers@netronome.com Subject: [PATCH bpf-next 12/17] nfp: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:21 +0200 Message-Id: <20210609103326.278782-13-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The nfp driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. While this is not actually an issue for the nfp driver because it doesn't support XDP_REDIRECT (and thus doesn't call xdp_do_flush()), the rcu_read_lock() is still unneeded. And With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Simon Horman Cc: oss-drivers@netronome.com Signed-off-by: Toke Høiland-Jørgensen Reviewed-by: Simon Horman --- drivers/net/ethernet/netronome/nfp/nfp_net_common.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index eeb30680b4dc..5dfa4799c34f 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -1819,7 +1819,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) struct xdp_buff xdp; int idx; - rcu_read_lock(); xdp_prog = READ_ONCE(dp->xdp_prog); true_bufsz = xdp_prog ? PAGE_SIZE : dp->fl_bufsz; xdp_init_buff(&xdp, PAGE_SIZE - NFP_NET_RX_BUF_HEADROOM, @@ -2036,7 +2035,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) if (!nfp_net_xdp_complete(tx_ring)) pkts_polled = budget; } - rcu_read_unlock(); return pkts_polled; } From patchwork Wed Jun 9 10:33:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309599 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 903CDC48BD1 for ; Wed, 9 Jun 2021 10:38:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 77EC0613B6 for ; Wed, 9 Jun 2021 10:38:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238064AbhFIKkq (ORCPT ); Wed, 9 Jun 2021 06:40:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:59767 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238982AbhFIKkn (ORCPT ); Wed, 9 Jun 2021 06:40:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623235129; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fbz4hNWvQ8PWHu2W2MZOy7/gDDHCdFCTVBAhydE+f+U=; b=Uk6MP8ZY5BDZ1yGBZiJeNVXT0o4xwHYpcT/j2Kdow9T8w1/kBPw3tyk2NvXFI++8JhZLmH 2BCoV/AcXm/CC6kv2pFXXt5lYAkmHGzAVeDR0byF9kCI1TxJKWfr3lVGBt17Vk2lce+aU5 v9h8MwHWb00TnwNPDA9VNRCJZ3pfe6A= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-179-nNnSk7I5PH2MXGya4D08Jw-1; Wed, 09 Jun 2021 06:38:45 -0400 X-MC-Unique: nNnSk7I5PH2MXGya4D08Jw-1 Received: by mail-ed1-f70.google.com with SMTP id ch5-20020a0564021bc5b029039389929f28so4436938edb.16 for ; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fbz4hNWvQ8PWHu2W2MZOy7/gDDHCdFCTVBAhydE+f+U=; b=egj1pAADqSh00WdR+XBc2Gi6FF1N/M8SVNZKEIsUeBj222s9a0/n7BTa2+kV2I6tNB 18XhfiPyOl/QoIWlcieMZtSNtcqEa1Pcau6aQUDg2Ui6y9pdK8m+xko04Xrbqcp5h+ml 97b0sPwg8Y4opxeI8YfdCeaOE4qOtbIUQlisSbVR+Z4spK525AenXzFCATEn5moDkOB8 qYSEzirI23KigoOu6ZxLv00mbIxqVEuv3FPrLujbRSqQOz/M9xY8fcVd208DBEa1yb8R umb0Deu7ES6TPg0hEAMXrIJmc8zvUNjD4CWcGSIBtQRQ1K/NWalAhjnZ8H+HXWd5/E+Q 7wuw== X-Gm-Message-State: AOAM531s3GhOUf8WvqGAUtxhFc9Bok/giDsWz7NQebxSihy2uQGP8jGN x0HZ1pumBwgXqgEUbur8tncWYenmAlx0SgKMCY8gJeZ34I9OQt3og29qWyg055XNfCrAc/poOjq GcREJyfdsd/Iq X-Received: by 2002:a17:906:869a:: with SMTP id g26mr27319302ejx.94.1623235124591; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwm+3mBcadPMLcwFavHgtOiY5Hjn0DBjRErakIm0gzowEegpnSOcGNewwLjLGNQf5IFEnMNOw== X-Received: by 2002:a17:906:869a:: with SMTP id g26mr27319296ejx.94.1623235124461; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id dh18sm966979edb.92.2021.06.09.03.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:38:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id D4A32180731; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Ariel Elior , GR-everest-linux-l2@marvell.com Subject: [PATCH bpf-next 13/17] qede: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:22 +0200 Message-Id: <20210609103326.278782-14-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The qede driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Ariel Elior Cc: GR-everest-linux-l2@marvell.com Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/qlogic/qede/qede_fp.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c index 8e150dd4f899..065e9004598e 100644 --- a/drivers/net/ethernet/qlogic/qede/qede_fp.c +++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c @@ -1089,13 +1089,7 @@ static bool qede_rx_xdp(struct qede_dev *edev, xdp_prepare_buff(&xdp, page_address(bd->data), *data_offset, *len, false); - /* Queues always have a full reset currently, so for the time - * being until there's atomic program replace just mark read - * side for map helpers. - */ - rcu_read_lock(); act = bpf_prog_run_xdp(prog, &xdp); - rcu_read_unlock(); /* Recalculate, as XDP might have changed the headers */ *data_offset = xdp.data - xdp.data_hard_start; From patchwork Wed Jun 9 10:33:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309591 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F9B3C48BCD for ; Wed, 9 Jun 2021 10:38:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CA33613B1 for ; Wed, 9 Jun 2021 10:38:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238944AbhFIKkl (ORCPT ); Wed, 9 Jun 2021 06:40:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32882 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238070AbhFIKkl (ORCPT ); Wed, 9 Jun 2021 06:40:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623235126; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jk02ESooGWTSdBozEufWm00wwV1iSla6r5tCLEhagY4=; b=XzH+xWCCJYVDHpbJ/uAwC4CxfvVpGLxC4SiDmGk5xjEAcCY7L9htvmbBj7H6BFW+Fx6TXC j0HXw9UqG4t82cXW7qaCqXgP/OMWV+/WSqdkGkafTcrTKczBbUJPdqIAWFDjJ1c8TSf/fN 3VEyk6ki6Z5m71BRDCBvRkA0Qkh4fjc= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-169-eu9WBf9lMz67C8k4IzRQZQ-1; Wed, 09 Jun 2021 06:38:45 -0400 X-MC-Unique: eu9WBf9lMz67C8k4IzRQZQ-1 Received: by mail-ed1-f69.google.com with SMTP id y7-20020aa7ce870000b029038fd7cdcf3bso12176695edv.15 for ; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jk02ESooGWTSdBozEufWm00wwV1iSla6r5tCLEhagY4=; b=tnObGcMfAn8K8RuNLCHr4fj010UUgtq+U8gobhfE6GMXaoE4KCN4SASWqKEtj490UQ zvukU0T66Hl9HYyh+a9x0q/wTqRX/l4xSULeXDxZaHKFFga6X3uM2wYeJq6QA6I8WoPb yaCj9hgA5vHfsSZF6dYBW9GO0v0LD1n3twpMh63dH+p4k9iBEJ+5E2plNcqFPX+e5cbk 8/vxNYuiuMwUpDstQQkXeNLkw8CwfIPZMHg7jcjKZwgc4iUt3XAAs3LPD6lM+KyCzlGq bav1eexzSDMl7N0PKQW3PnhPsb1TiSYRI0x8CtunrPOxmiFK+IUQDLRaMW8vnRrtsnwz ZTxA== X-Gm-Message-State: AOAM531STR3p3Dxa/t46oJ0qUge/hS3QDf243pDNhl2Z+gXD5KqNqrMv L+pIilFm4Flw5lXj9wlqSV5n3O8xr5/rqZNrEPHOEKhhZWZAtbVnxBGHezxowf2Zb1JuRTqL/tB iqZLWRdX6Z0sM X-Received: by 2002:a17:906:8a55:: with SMTP id gx21mr28306410ejc.179.1623235124152; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz8vD1ubOmif6w/i6ZswfYgm1TfNtkHvLNWlj29pRDEzzgimzt9ncLfA52oAlvj9EOHKV8cXw== X-Received: by 2002:a17:906:8a55:: with SMTP id gx21mr28306386ejc.179.1623235123797; Wed, 09 Jun 2021 03:38:43 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id br21sm925884ejb.124.2021.06.09.03.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:38:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id DE3D7180732; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Edward Cree , Martin Habets Subject: [PATCH bpf-next 14/17] sfc: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:23 +0200 Message-Id: <20210609103326.278782-15-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The sfc driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Edward Cree Cc: Martin Habets Signed-off-by: Toke Høiland-Jørgensen Acked-by: Edward Cree --- drivers/net/ethernet/sfc/rx.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c index 17b8119c48e5..606750938b89 100644 --- a/drivers/net/ethernet/sfc/rx.c +++ b/drivers/net/ethernet/sfc/rx.c @@ -260,18 +260,14 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel, s16 offset; int err; - rcu_read_lock(); - xdp_prog = rcu_dereference(efx->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + xdp_prog = rcu_dereference_bh(efx->xdp_prog); + if (!xdp_prog) return true; - } rx_queue = efx_channel_get_rx_queue(channel); if (unlikely(channel->rx_pkt_n_frags > 1)) { /* We can't do XDP on fragmented packets - drop. */ - rcu_read_unlock(); efx_free_rx_buffers(rx_queue, rx_buf, channel->rx_pkt_n_frags); if (net_ratelimit()) @@ -296,7 +292,6 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel, rx_buf->len, false); xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp); - rcu_read_unlock(); offset = (u8 *)xdp.data - *ehp; From patchwork Wed Jun 9 10:33:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309569 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BB5DC48BD1 for ; Wed, 9 Jun 2021 10:33:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7AC1D613AC for ; Wed, 9 Jun 2021 10:33:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238880AbhFIKfv (ORCPT ); Wed, 9 Jun 2021 06:35:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24335 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238915AbhFIKfh (ORCPT ); Wed, 9 Jun 2021 06:35:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SDs6RJCDclPtRl7f7lh2LEBzpa1knLCOi0eHBp6e438=; b=W4Xg9p0LnWx4kkzQrG3pn9r7eTH9s7nkvEMIT80WK6sZwbdXAuU8oQ0uctcInGH1+KoA3x qaF/sx9xclS4HSEP7Vs/vAitqH2GMq0Y+ZMYsM46EvmBG8lmioNAzJxWfGyFNxxqhFL4cp w2/On0wdoPSvSyyO80FWNGtOBn348Rw= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-96-ZuJS-lTHMyKPDUxyyWPOzw-1; Wed, 09 Jun 2021 06:33:40 -0400 X-MC-Unique: ZuJS-lTHMyKPDUxyyWPOzw-1 Received: by mail-ed1-f70.google.com with SMTP id g13-20020a056402090db02903935a4cb74fso6449768edz.1 for ; Wed, 09 Jun 2021 03:33:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SDs6RJCDclPtRl7f7lh2LEBzpa1knLCOi0eHBp6e438=; b=OL2k5HVb9x5DQMIbOF2RQ6Pwq7tFlrsgGeu8ny5EMz5TmpTuBg3PSaLrwD+LkJS+Yd c7JKWkY8L0v83yJQ6HY/NlwIZuEvjtS7RP+Mt3HnLB1nyb/Ch2zyVE/jsoHbV4vZdzS0 sZjnJPSA17Sve+WwW/9Mjs0iIO1ciMoVhC1a0z81pk4VJWflMR22N1ES1tm0n4PlfdB1 ttW9KfVbkoyjOaIvsjmnPQca+5MOdKQ96N7V7djUtzKMlsNlS/CxUSB20QrGP0C0TF1h RtfVYf14oBLv8EKQG4xYa+uZNWC2XFFjDnS7zmbGNycsUJeodoleCZnLdyK/bHSfebvA wUiA== X-Gm-Message-State: AOAM533BR5V2RhBoFauZnEdPoafznFdwJOG1Tv3xYn/cv1CeFYga8rKK 4mP5yXZxbOmmqXwwB/56ms4qhQf8z2ijm9Et0yojmdHYNpep5Ndgmv5FXeAectQEhULmPNyPd+h UsfLuDLsXrkIZ X-Received: by 2002:a17:906:5a9a:: with SMTP id l26mr27998544ejq.490.1623234818851; Wed, 09 Jun 2021 03:33:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy6cqSoYK0RE/EGw6NF1yHXe5cN5J2vUejl6abV3SnyWgXCCqNzpuKen3JNOpReBROIG0SuEw== X-Received: by 2002:a17:906:5a9a:: with SMTP id l26mr27998519ejq.490.1623234818517; Wed, 09 Jun 2021 03:33:38 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id o4sm946229edc.94.2021.06.09.03.33.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:37 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id E95E7180733; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Jassi Brar , Ilias Apalodimas Subject: [PATCH bpf-next 15/17] netsec: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:24 +0200 Message-Id: <20210609103326.278782-16-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The netsec driver has a rcu_read_lock()/rcu_read_unlock() pair around the full RX loop, covering everything up to and including xdp_do_flush(). This is actually the correct behaviour, but because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), it is also technically redundant. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep the rcu_read_lock() around anymore, so let's just remove it. Cc: Jassi Brar Cc: Ilias Apalodimas Signed-off-by: Toke Høiland-Jørgensen Acked-by: Ilias Apalodimas --- drivers/net/ethernet/socionext/netsec.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index dfc85cc68173..20d148c019d8 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -958,7 +958,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) xdp_init_buff(&xdp, PAGE_SIZE, &dring->xdp_rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(priv->xdp_prog); dma_dir = page_pool_get_dma_dir(dring->page_pool); @@ -1069,8 +1068,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) } netsec_finalize_xdp_rx(priv, xdp_act, xdp_xmit); - rcu_read_unlock(); - return done; } From patchwork Wed Jun 9 10:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309567 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28293C48BE0 for ; Wed, 9 Jun 2021 10:33:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1350A61185 for ; Wed, 9 Jun 2021 10:33:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237132AbhFIKfv (ORCPT ); Wed, 9 Jun 2021 06:35:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33146 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238903AbhFIKff (ORCPT ); Wed, 9 Jun 2021 06:35:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623234820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b6/ywr8A84xw5GpjlyvV4hRNlzB/uVzXQRS5Ys9UtLc=; b=LYvg19KeOaNn1RUYY9IhTFgzuk1eW/+SXH/70Fd7ZcvvSs+foxl/Ijq33XrUDdMAFwADG2 iZR4MDQpNVVkzwIJP6B5k8CI3FPmt4eeRRIgGxF8jMPZZUpphXMMhDCZ7Eu8NR0TphqM4+ ZX1VnqQEmSS+RSfKxOMv2xMvfUmjEUU= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-234-Ms80x6hjPcS96zBoqFB1gA-1; Wed, 09 Jun 2021 06:33:39 -0400 X-MC-Unique: Ms80x6hjPcS96zBoqFB1gA-1 Received: by mail-ed1-f72.google.com with SMTP id dd28-20020a056402313cb029038fc9850034so12229698edb.7 for ; Wed, 09 Jun 2021 03:33:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b6/ywr8A84xw5GpjlyvV4hRNlzB/uVzXQRS5Ys9UtLc=; b=Yv7M9ECVbT3tvku0mmBz2yQCpoc3syrzxuXg+NsVsz1V9ROEJly3FFDiniZ70OqdCP F6xkpCvLO+BdWC9+1r48FoMxrtzs6Goy9WJt3lwv4emDHgtxN7rkq4EgAPfk43ddHj62 6vyd6Wicmm801UClN5pKp11zHU4Hlm+eQQq9gq3vRZyMyD4s8JeQLXYbe5mzjPp9K8fc ybj8kTdNPYJHOP8uRdPn5N0BxhPsfvaW8A4jFiXUIxhl8A25oTGR5NLtLWdNN6dc/Ygs PpkvDEX4LpIl1T8+uRFv6Lx/uMQp9NnsV+vsyhB2aNLEwd0Fcy7USeEEETTUxnue5T+4 LVVQ== X-Gm-Message-State: AOAM530HCZo1PMzNdmlpn5iM6IFBdYPu8sc3OobE3zbBTJfJYlr/aNTE ydtBg8nwZJHV5Ut3nDvJUOrW4A/DtAiRXoRLkPGe1Qzh3inoCHAgz5rSF42VKWGidEoj+nrVUNb vmBRGeMSZvY27 X-Received: by 2002:a05:6402:511:: with SMTP id m17mr29902728edv.1.1623234818393; Wed, 09 Jun 2021 03:33:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxy8dEQdT/+2mrdhhPDQI/OtNZSHMX52Tnhpuds9eoPNSLQsCBSfjSo0BHivEiX/iSeqUt4zA== X-Received: by 2002:a05:6402:511:: with SMTP id m17mr29902711edv.1.1623234818248; Wed, 09 Jun 2021 03:33:38 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id l8sm933170ejp.40.2021.06.09.03.33.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:33:36 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id F2934180734; Wed, 9 Jun 2021 12:33:30 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu Subject: [PATCH bpf-next 16/17] stmmac: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:25 +0200 Message-Id: <20210609103326.278782-17-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The stmmac driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Giuseppe Cavallaro Cc: Alexandre Torgue Cc: Jose Abreu Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index bf9fe25fed69..08c4b999e1ba 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -4654,7 +4654,6 @@ static int stmmac_xdp_xmit_back(struct stmmac_priv *priv, return res; } -/* This function assumes rcu_read_lock() is held by the caller. */ static int __stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog, struct xdp_buff *xdp) @@ -4696,17 +4695,14 @@ static struct sk_buff *stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); if (!prog) { res = STMMAC_XDP_PASS; - goto unlock; + goto out; } res = __stmmac_xdp_run_prog(priv, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } @@ -4976,10 +4972,8 @@ static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue) buf->xdp->data_end = buf->xdp->data + buf1_len; xsk_buff_dma_sync_for_cpu(buf->xdp, rx_q->xsk_pool); - rcu_read_lock(); prog = READ_ONCE(priv->xdp_prog); res = __stmmac_xdp_run_prog(priv, prog, buf->xdp); - rcu_read_unlock(); switch (res) { case STMMAC_XDP_PASS: From patchwork Wed Jun 9 10:33:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 12309593 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2220C48BDF for ; Wed, 9 Jun 2021 10:38:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BD564613AE for ; Wed, 9 Jun 2021 10:38:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238384AbhFIKkm (ORCPT ); Wed, 9 Jun 2021 06:40:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44469 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238070AbhFIKkl (ORCPT ); Wed, 9 Jun 2021 06:40:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623235127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8R6RyRHO67ifLZIR9gGLNOyftDsWB9ahrub2+xbd3e4=; b=Go0kKejyD40UyeGrcDtAraY9B+gGp2o+GoL9+10Acgg7M6yvG1Yh39a3OQ1KIQV8/iDLm+ /4JQzW7TCYGo7YXxa7ZA0XwxuBkqPs8+hSV2WLjROBLJolVClzNBTsCWUhPw7a8u9L70Mb /2xGeV7NxWHWy+WpRkB94LsPaykKWDU= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-70-7ZHSOhVBNFOAd31gkMhbfA-1; Wed, 09 Jun 2021 06:38:45 -0400 X-MC-Unique: 7ZHSOhVBNFOAd31gkMhbfA-1 Received: by mail-ed1-f69.google.com with SMTP id c12-20020aa7d60c0000b029038fccdf4390so12192447edr.9 for ; Wed, 09 Jun 2021 03:38:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8R6RyRHO67ifLZIR9gGLNOyftDsWB9ahrub2+xbd3e4=; b=ZiZ4OFaJGtZG5TYEWKH9jmkF6LIk28JsYbweRTvNbIHBoHr8HI+m2IvjarT1aVxD6t /9Zo9wQ/wWlO+mBrDKXLSZU2wqYOH77LeaEZpCdUvYJ7OiioRzX4alW/MoQ6/v85Emp7 f4399WrgGATkhj7WmYphKU+DYoFrLKRjTDgTsFZtdWYFjFvxpibtI5jreudvfgUGE/yr NxWSeUiZXBewtgyStfDQ5RVWFJPS9TOecFnrF1/1catvw1YnB0mayqGOR92FWFyMlWK0 xs/o37hK7aJPfDJAVBvhd9fmPM/kh3DmY5wrER5REbaFl98/o7viNr1jBzroNsbr/5rs 9ZXA== X-Gm-Message-State: AOAM531asffx1SEJuq9LwWQApm+wgGFiZl6on8PvWLvuw/7s20RYbdTV p6oBnsYLfTKj9enM5NuAIOr4gmI/NjRb+BCIwXSqYct+wjrmds0Ox+adxRf3Rvc0JnSwFd3lC96 7/Iz0mnL56/Iv X-Received: by 2002:a17:906:9715:: with SMTP id k21mr28048381ejx.553.1623235124785; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyvLr0lbuBFvW5cCFJTjirrWeEr6r/YlVd/ZkLudMiEalQXD0poZgcUxpKwqjuuUBC3AxiU/Q== X-Received: by 2002:a17:906:9715:: with SMTP id k21mr28048369ejx.553.1623235124649; Wed, 09 Jun 2021 03:38:44 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id u19sm920226ejt.74.2021.06.09.03.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 03:38:43 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 0ADBF180735; Wed, 9 Jun 2021 12:33:31 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Grygorii Strashko , linux-omap@vger.kernel.org Subject: [PATCH bpf-next 17/17] net: ti: remove rcu_read_lock() around XDP program invocation Date: Wed, 9 Jun 2021 12:33:26 +0200 Message-Id: <20210609103326.278782-18-toke@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210609103326.278782-1-toke@redhat.com> References: <20210609103326.278782-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The cpsw driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Grygorii Strashko Cc: linux-omap@vger.kernel.org Signed-off-by: Toke Høiland-Jørgensen Tested-by: Grygorii Strashko Reviewed-by: Grygorii Strashko --- drivers/net/ethernet/ti/cpsw_priv.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/ti/cpsw_priv.c b/drivers/net/ethernet/ti/cpsw_priv.c index 5862f0a4a975..ecc2a6b7e28f 100644 --- a/drivers/net/ethernet/ti/cpsw_priv.c +++ b/drivers/net/ethernet/ti/cpsw_priv.c @@ -1328,13 +1328,9 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp, struct bpf_prog *prog; u32 act; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); - if (!prog) { - ret = CPSW_XDP_PASS; - goto out; - } + if (!prog) + return CPSW_XDP_PASS; act = bpf_prog_run_xdp(prog, xdp); /* XDP prog might have changed packet data and boundaries */ @@ -1378,10 +1374,8 @@ int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp, ndev->stats.rx_bytes += *len; ndev->stats.rx_packets++; out: - rcu_read_unlock(); return ret; drop: - rcu_read_unlock(); page_pool_recycle_direct(cpsw->page_pool[ch], page); return ret; }