From patchwork Fri Feb 4 22:42:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12735779 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78DC2C433EF for ; Fri, 4 Feb 2022 22:42:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236278AbiBDWmp (ORCPT ); Fri, 4 Feb 2022 17:42:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233101AbiBDWmo (ORCPT ); Fri, 4 Feb 2022 17:42:44 -0500 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F7B0D8778EC for ; Fri, 4 Feb 2022 14:42:43 -0800 (PST) Received: by mail-pj1-x1033.google.com with SMTP id p22-20020a17090adf9600b001b8783b2647so1185227pjv.5 for ; Fri, 04 Feb 2022 14:42:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U8mvndYc5GiIgFu7fMQsqr6+5Dn5yDgnXOI/6CxhF5c=; b=WyoWMTplhXG9MB+kMp1xMuP1v6I0fJ2bTJlPS8ak8vuPVgNgyQNAdRWss0Es0vUAFo x5qDgQinFvVhIlBnw2CpgHuwi+TSslV2Fx7pR4tQelvpVwq2ix40bTmmNnC6lrNCb8wp aYiY5wPRyFXjlf2a/hcoGEllGbi4YjJWKSj8XCTag61VjhIVpDt+2bKyslDMvEhl9tX+ KhoopHQj7FdfpHJ2A6LTa1f2N2fnYP2BkaUFRG9t/Fai5Ga/KFwKvp7/iwD6blz1Ob4N tghMeByWzSlU4fky2N82c5i/avkPSmbwFmKvQaF0l6O/QUJZsFQ5EbuLV42/tX3+O8EW OvWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U8mvndYc5GiIgFu7fMQsqr6+5Dn5yDgnXOI/6CxhF5c=; b=AhKGu/d7wAzQqfk5sslN+nCX8R9zaaOLJnaQNcfEdVd1cK1Q556e7qdAwWyLX7TOM0 dhv8Nm3GUO/pIbwnqHg4ThOFr0SJWTHifjbKYCYCtFQbwgrz4jfMMkodIDE8HKZ4fWae L3FcJ3L0Du1QGQS0rTt6WXXWG2d7aJlPmC458VtbTatmfEpWq1Gchu9lvWQBhQlVDxYR H5uEKHbhTyKxeGMjeqVNhFi4aIfe+M97fjry5lEyDgWKkG3ihtGFPgxLk+aOnLm8tk04 TxEpQkEc2+rO1fwr06/7AzzaODdnBdZ8zwCVZfOt8Ym00fhL56ZTWbH4t7KwkxXxobCy WI1g== X-Gm-Message-State: AOAM532Bde5K0eQW0rtcaQjkBKUdZ7LG0puFUNl/mXYC1kEDoVHdKZ6N cZo1zNRuVv3M1WZvplrF500= X-Google-Smtp-Source: ABdhPJwPGPLy4Qo+w0FPcO6QPfuchVgjaL8UZfV+tSRcKKEDEeFcMQ64Heh/ZOHhL035GANW340qqA== X-Received: by 2002:a17:90a:94cc:: with SMTP id j12mr5558467pjw.39.1644014563120; Fri, 04 Feb 2022 14:42:43 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:e0d3:6ec9:bd06:3e67]) by smtp.gmail.com with ESMTPSA id s2sm2410060pgl.21.2022.02.04.14.42.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Feb 2022 14:42:42 -0800 (PST) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet Subject: [PATCH v2 net-next 1/3] ref_tracker: implement use-after-free detection Date: Fri, 4 Feb 2022 14:42:35 -0800 Message-Id: <20220204224237.2932026-2-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220204224237.2932026-1-eric.dumazet@gmail.com> References: <20220204224237.2932026-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet Whenever ref_tracker_dir_init() is called, mark the struct ref_tracker_dir as dead. Test the dead status from ref_tracker_alloc() and ref_tracker_free() This should detect buggy dev_put()/dev_hold() happening too late in netdevice dismantle process. Signed-off-by: Eric Dumazet --- include/linux/ref_tracker.h | 2 ++ lib/ref_tracker.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/ref_tracker.h b/include/linux/ref_tracker.h index 60f3453be23e6881725d383c55f93143fda1e7a2..a443abda937d86ff534225bf16b958a9da295a7d 100644 --- a/include/linux/ref_tracker.h +++ b/include/linux/ref_tracker.h @@ -13,6 +13,7 @@ struct ref_tracker_dir { spinlock_t lock; unsigned int quarantine_avail; refcount_t untracked; + bool dead; struct list_head list; /* List of active trackers */ struct list_head quarantine; /* List of dead trackers */ #endif @@ -26,6 +27,7 @@ static inline void ref_tracker_dir_init(struct ref_tracker_dir *dir, INIT_LIST_HEAD(&dir->quarantine); spin_lock_init(&dir->lock); dir->quarantine_avail = quarantine_count; + dir->dead = false; refcount_set(&dir->untracked, 1); stack_depot_init(); } diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c index a6789c0c626b0f68ad67c264cd19177a63fb82d2..32ff6bd497f8e464eeb51a3628cb24bded0547da 100644 --- a/lib/ref_tracker.c +++ b/lib/ref_tracker.c @@ -20,6 +20,7 @@ void ref_tracker_dir_exit(struct ref_tracker_dir *dir) unsigned long flags; bool leak = false; + dir->dead = true; spin_lock_irqsave(&dir->lock, flags); list_for_each_entry_safe(tracker, n, &dir->quarantine, head) { list_del(&tracker->head); @@ -72,6 +73,8 @@ int ref_tracker_alloc(struct ref_tracker_dir *dir, gfp_t gfp_mask = gfp; unsigned long flags; + WARN_ON_ONCE(dir->dead); + if (gfp & __GFP_DIRECT_RECLAIM) gfp_mask |= __GFP_NOFAIL; *trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask); @@ -100,6 +103,8 @@ int ref_tracker_free(struct ref_tracker_dir *dir, unsigned int nr_entries; unsigned long flags; + WARN_ON_ONCE(dir->dead); + if (!tracker) { refcount_dec(&dir->untracked); return -EEXIST; From patchwork Fri Feb 4 22:42:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12735780 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19174C433EF for ; Fri, 4 Feb 2022 22:42:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352147AbiBDWms (ORCPT ); Fri, 4 Feb 2022 17:42:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233101AbiBDWmq (ORCPT ); Fri, 4 Feb 2022 17:42:46 -0500 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5FC39C061354 for ; Fri, 4 Feb 2022 14:42:45 -0800 (PST) Received: by mail-pg1-x52d.google.com with SMTP id e16so6188031pgn.4 for ; Fri, 04 Feb 2022 14:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BiDjGDalIJZgno57zKrydIx7zYwfZotf0yffC1XSl/o=; b=M/wUMmIgrAi4s4hLNLc7NqQ/OJPaUjEjf3KPyr54DOZl9UbT2awZCSVT4ugmVZ4lOT P49T2GuJL910RU5mPqf7xwNSwtySD0wynu9LFB7eWCO+HSSi3FrsQLXylNJcRKXO/e2K xeWApVjjh/3AcDkOIAq2STVWvl2kSxq+pJ0gMRaa4bNM/+HqlS4FF+o/p22ZQcb5llz7 foFNuhS/vUmqe9o5/wtqFIeHseK//XsGdISYV9AqQjllfm5SH28DiCWNZNLV4SSawIbJ TEXHtRJi53vbEPBYDgh5RY1oe66ozM0p0fRgXcYf8QGWfbzR/DpELMB/iyhrjfBDAbNb KWwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BiDjGDalIJZgno57zKrydIx7zYwfZotf0yffC1XSl/o=; b=yHpuvrC6LvX17y1a1wF4eX9uQMas4/KrHwSe8Rb2M1lPGpUeDwuxiZu9xpq7+SyJUT ocPyP56DvgkVFHV3I+0zppFG6QkManQau7Jy+plGiCCpTE202ns3KKudb5P+A3rHKtOq X5fUWHPLhKELv6b35R+9Z7tBzNgkus0CEC9UOlkO6tAXGDlhjupnbxt/2u+N54YtMsKb 0aye63m46Idn6q88hU+tBnMhzS/GFojTJopKPbPZnyFNv2bZZJtgMz4NtId7mhY6DzLo ZgdFf/WUTlOZIqJcurwxcHQlyNGat5OTtLzl3Xpb1Pepxkv+wZcbmDzCo5tVp6jVlJ9s m7nw== X-Gm-Message-State: AOAM532Im4N1jFANiN5wRH2LZqWV3PsW/OwgKeBK/4pgIix+UnxRjiuR JtAT7ZDOAgV0dabvzFo8Ku4Saj+vnSA= X-Google-Smtp-Source: ABdhPJyDidm250mnD6JZeXHLsAmJzlWU6mY4jBS1aKwuMNDtbOQ3eNuwth/G8DEDBagbeaa+PUa4xQ== X-Received: by 2002:a62:7a42:: with SMTP id v63mr5207043pfc.61.1644014564969; Fri, 04 Feb 2022 14:42:44 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:e0d3:6ec9:bd06:3e67]) by smtp.gmail.com with ESMTPSA id s2sm2410060pgl.21.2022.02.04.14.42.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Feb 2022 14:42:44 -0800 (PST) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet Subject: [PATCH v2 net-next 2/3] ref_tracker: add a count of untracked references Date: Fri, 4 Feb 2022 14:42:36 -0800 Message-Id: <20220204224237.2932026-3-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220204224237.2932026-1-eric.dumazet@gmail.com> References: <20220204224237.2932026-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet We are still chasing a netdev refcount imbalance, and we suspect we have one rogue dev_put() that is consuming a reference taken from a dev_hold_track() To detect this case, allow ref_tracker_alloc() and ref_tracker_free() to be called with a NULL @trackerp parameter, and use a dedicated refcount_t just for them. Signed-off-by: Eric Dumazet --- include/linux/ref_tracker.h | 2 ++ lib/ref_tracker.c | 12 +++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/include/linux/ref_tracker.h b/include/linux/ref_tracker.h index a443abda937d86ff534225bf16b958a9da295a7d..9ca353ab712b5e897d9b3e5cfcd7117b610dd01a 100644 --- a/include/linux/ref_tracker.h +++ b/include/linux/ref_tracker.h @@ -13,6 +13,7 @@ struct ref_tracker_dir { spinlock_t lock; unsigned int quarantine_avail; refcount_t untracked; + refcount_t no_tracker; bool dead; struct list_head list; /* List of active trackers */ struct list_head quarantine; /* List of dead trackers */ @@ -29,6 +30,7 @@ static inline void ref_tracker_dir_init(struct ref_tracker_dir *dir, dir->quarantine_avail = quarantine_count; dir->dead = false; refcount_set(&dir->untracked, 1); + refcount_set(&dir->no_tracker, 1); stack_depot_init(); } diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c index 32ff6bd497f8e464eeb51a3628cb24bded0547da..9c0c2e09df666d19aba441f568762afbd1cad4d0 100644 --- a/lib/ref_tracker.c +++ b/lib/ref_tracker.c @@ -38,6 +38,7 @@ void ref_tracker_dir_exit(struct ref_tracker_dir *dir) spin_unlock_irqrestore(&dir->lock, flags); WARN_ON_ONCE(leak); WARN_ON_ONCE(refcount_read(&dir->untracked) != 1); + WARN_ON_ONCE(refcount_read(&dir->no_tracker) != 1); } EXPORT_SYMBOL(ref_tracker_dir_exit); @@ -75,6 +76,10 @@ int ref_tracker_alloc(struct ref_tracker_dir *dir, WARN_ON_ONCE(dir->dead); + if (!trackerp) { + refcount_inc(&dir->no_tracker); + return 0; + } if (gfp & __GFP_DIRECT_RECLAIM) gfp_mask |= __GFP_NOFAIL; *trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask); @@ -98,13 +103,18 @@ int ref_tracker_free(struct ref_tracker_dir *dir, struct ref_tracker **trackerp) { unsigned long entries[REF_TRACKER_STACK_ENTRIES]; - struct ref_tracker *tracker = *trackerp; depot_stack_handle_t stack_handle; + struct ref_tracker *tracker; unsigned int nr_entries; unsigned long flags; WARN_ON_ONCE(dir->dead); + if (!trackerp) { + refcount_dec(&dir->no_tracker); + return 0; + } + tracker = *trackerp; if (!tracker) { refcount_dec(&dir->untracked); return -EEXIST; From patchwork Fri Feb 4 22:42:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12735781 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 200B1C433F5 for ; Fri, 4 Feb 2022 22:42:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352136AbiBDWms (ORCPT ); Fri, 4 Feb 2022 17:42:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352185AbiBDWmr (ORCPT ); Fri, 4 Feb 2022 17:42:47 -0500 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28107D8778ED for ; Fri, 4 Feb 2022 14:42:47 -0800 (PST) Received: by mail-pl1-x62a.google.com with SMTP id h14so6368566plf.1 for ; Fri, 04 Feb 2022 14:42:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ob7P/giOPp4kUaR7JDqub8nbfwsF5B2fJS25JZWrb7g=; b=KYU9HuyBFnGHhstZRpmtBRPMvpN+SqkD/KTbTCDV45YTIs1hdMQc1YBjPPHSdH7khz XN/pDpfe9UAQbQRhCYKhxto3QkZaP//4l2UcrJF9qQJFKBXaA1Y98mLJK/d8ScdXHLSP mNEIsMku3lxT6+mkZXP74q5vTLoMEbBZi9hojwVyI0ohR4cbuKD0YyJ8QC/nRgvyKkNI Tj5u0yPAl22H1w8Bysyd+6nVQSe0o+Deg8sDuPFeyjIt3e2SeReMLOH2yDJk/8S++2kk blJBNH2ZbGrZxWYJWMXOxjE1SoZWmVtCZfNd0xNtCMpyY2vLbF/mrltk/8y8jycnv6Gj ewSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ob7P/giOPp4kUaR7JDqub8nbfwsF5B2fJS25JZWrb7g=; b=54WNn9s1gbF8DIOsiAOgc2A98P3c5w/6yySYP28z+j0iKQJF5lufstTjdQN2gdrGED HnKTKTXmUhkqriFAGSLL/pmsnH2DZKAgcKD3hiovGvRw3qKCdXXub9IlgOQCIUZPzrgF UBKNrxUWkzaKBsih5Wyy4i3tgvBLxPW2yRn5Zzn8dmrbdPd3/c8MwwWMMa1OKiOonfN+ JJX43Hm7JHU21NoZNBHmecrqBLt0I+O8fR4S8j16SNAsr1Fzs1WvPF/toUXo4ASaLAks uzmKPny1iT0gEXAfFG3Sx1WS9iY79dCQMzhpzj52HKMsTXq0RYemepYdYjOR+IrGVsRd DlTw== X-Gm-Message-State: AOAM531Mi2+fqn/70OPQdW7OrEKT3P98egJcE1Tq8PjbyFS1RRhfVrUz oGvkTEPaBXTi3jVb/uxl/H77Sjqm7Gg= X-Google-Smtp-Source: ABdhPJzlvqFfalRICCoAUglq+esnPBevgQWxdWeSOChCyA9kGf6tDgea8cAF9qIKnTur6PSDQfO2Pw== X-Received: by 2002:a17:90b:4c0c:: with SMTP id na12mr5603016pjb.140.1644014566602; Fri, 04 Feb 2022 14:42:46 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:e0d3:6ec9:bd06:3e67]) by smtp.gmail.com with ESMTPSA id s2sm2410060pgl.21.2022.02.04.14.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Feb 2022 14:42:46 -0800 (PST) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet Subject: [PATCH v2 net-next 3/3] net: refine dev_put()/dev_hold() debugging Date: Fri, 4 Feb 2022 14:42:37 -0800 Message-Id: <20220204224237.2932026-4-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220204224237.2932026-1-eric.dumazet@gmail.com> References: <20220204224237.2932026-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet We are still chasing some syzbot reports where we think a rogue dev_put() is called with no corresponding prior dev_hold(). Unfortunately it eats a reference on dev->dev_refcnt taken by innocent dev_hold_track(), meaning that the refcount saturation splat comes too late to be useful. Make sure that 'not tracked' dev_put() and dev_hold() better use CONFIG_NET_DEV_REFCNT_TRACKER=y debug infrastructure: Prior patch in the series allowed ref_tracker_alloc() and ref_tracker_free() to be called with a NULL @trackerp parameter, and to use a separate refcount only to detect too many put() even in the following case: dev_hold_track(dev, tracker_1, GFP_ATOMIC); dev_hold(dev); dev_put(dev); dev_put(dev); // Should complain loudly here. dev_put_track(dev, tracker_1); // instead of here Add clarification about netdev_tracker_alloc() role. v2: I replaced the dev_put() in linkwatch_do_dev() with __dev_put() because callers called netdev_tracker_free(). Signed-off-by: Eric Dumazet --- include/linux/netdevice.h | 69 ++++++++++++++++++++++++++------------- net/core/dev.c | 2 +- net/core/link_watch.c | 6 ++-- 3 files changed, 50 insertions(+), 27 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index e490b84732d1654bf067b30f2bb0b0825f88dea9..3fb6fb67ed77e70314a699c9bdf8f4b26acfcc19 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3817,14 +3817,7 @@ extern unsigned int netdev_budget_usecs; /* Called by rtnetlink.c:rtnl_unlock() */ void netdev_run_todo(void); -/** - * dev_put - release reference to device - * @dev: network device - * - * Release reference to device to allow it to be freed. - * Try using dev_put_track() instead. - */ -static inline void dev_put(struct net_device *dev) +static inline void __dev_put(struct net_device *dev) { if (dev) { #ifdef CONFIG_PCPU_DEV_REFCNT @@ -3835,14 +3828,7 @@ static inline void dev_put(struct net_device *dev) } } -/** - * dev_hold - get reference to device - * @dev: network device - * - * Hold reference to device to keep it from being freed. - * Try using dev_hold_track() instead. - */ -static inline void dev_hold(struct net_device *dev) +static inline void __dev_hold(struct net_device *dev) { if (dev) { #ifdef CONFIG_PCPU_DEV_REFCNT @@ -3853,11 +3839,24 @@ static inline void dev_hold(struct net_device *dev) } } +static inline void __netdev_tracker_alloc(struct net_device *dev, + netdevice_tracker *tracker, + gfp_t gfp) +{ +#ifdef CONFIG_NET_DEV_REFCNT_TRACKER + ref_tracker_alloc(&dev->refcnt_tracker, tracker, gfp); +#endif +} + +/* netdev_tracker_alloc() can upgrade a prior untracked reference + * taken by dev_get_by_name()/dev_get_by_index() to a tracked one. + */ static inline void netdev_tracker_alloc(struct net_device *dev, netdevice_tracker *tracker, gfp_t gfp) { #ifdef CONFIG_NET_DEV_REFCNT_TRACKER - ref_tracker_alloc(&dev->refcnt_tracker, tracker, gfp); + refcount_dec(&dev->refcnt_tracker.no_tracker); + __netdev_tracker_alloc(dev, tracker, gfp); #endif } @@ -3873,8 +3872,8 @@ static inline void dev_hold_track(struct net_device *dev, netdevice_tracker *tracker, gfp_t gfp) { if (dev) { - dev_hold(dev); - netdev_tracker_alloc(dev, tracker, gfp); + __dev_hold(dev); + __netdev_tracker_alloc(dev, tracker, gfp); } } @@ -3883,10 +3882,34 @@ static inline void dev_put_track(struct net_device *dev, { if (dev) { netdev_tracker_free(dev, tracker); - dev_put(dev); + __dev_put(dev); } } +/** + * dev_hold - get reference to device + * @dev: network device + * + * Hold reference to device to keep it from being freed. + * Try using dev_hold_track() instead. + */ +static inline void dev_hold(struct net_device *dev) +{ + dev_hold_track(dev, NULL, GFP_ATOMIC); +} + +/** + * dev_put - release reference to device + * @dev: network device + * + * Release reference to device to allow it to be freed. + * Try using dev_put_track() instead. + */ +static inline void dev_put(struct net_device *dev) +{ + dev_put_track(dev, NULL); +} + static inline void dev_replace_track(struct net_device *odev, struct net_device *ndev, netdevice_tracker *tracker, @@ -3895,11 +3918,11 @@ static inline void dev_replace_track(struct net_device *odev, if (odev) netdev_tracker_free(odev, tracker); - dev_hold(ndev); - dev_put(odev); + __dev_hold(ndev); + __dev_put(odev); if (ndev) - netdev_tracker_alloc(ndev, tracker, gfp); + __netdev_tracker_alloc(ndev, tracker, gfp); } /* Carrier loss detection, dial on demand. The functions netif_carrier_on diff --git a/net/core/dev.c b/net/core/dev.c index f79744d99413434ad28b26dee9aeeb2893a0e3ae..1eaa0b88e3ba5d800484656f2c3420af57050294 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10172,7 +10172,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, dev->pcpu_refcnt = alloc_percpu(int); if (!dev->pcpu_refcnt) goto free_dev; - dev_hold(dev); + __dev_hold(dev); #else refcount_set(&dev->dev_refcnt, 1); #endif diff --git a/net/core/link_watch.c b/net/core/link_watch.c index b0f5344d1185be66d05cd1dc50cffc5ccfe883ef..95098d1a49bdf4cbc3ddeb4d345e4276f974a208 100644 --- a/net/core/link_watch.c +++ b/net/core/link_watch.c @@ -166,10 +166,10 @@ static void linkwatch_do_dev(struct net_device *dev) netdev_state_change(dev); } - /* Note: our callers are responsible for - * calling netdev_tracker_free(). + /* Note: our callers are responsible for calling netdev_tracker_free(). + * This is the reason we use __dev_put() instead of dev_put(). */ - dev_put(dev); + __dev_put(dev); } static void __linkwatch_run_queue(int urgent_only)