From patchwork Wed Jan 24 11:59:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13529072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 034FEC46CD2 for ; Wed, 24 Jan 2024 12:00:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4BF4C6B0088; Wed, 24 Jan 2024 06:59:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 46E248D0002; Wed, 24 Jan 2024 06:59:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24BA16B0089; Wed, 24 Jan 2024 06:59:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0904F8D0002 for ; Wed, 24 Jan 2024 06:59:57 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CA051A0C38 for ; Wed, 24 Jan 2024 11:59:56 +0000 (UTC) X-FDA: 81714060792.26.6DA2D7E Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf27.hostedemail.com (Postfix) with ESMTP id ACAA940006 for ; Wed, 24 Jan 2024 11:59:54 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706097595; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=+qyNQdtPtMq//XnnSxH00XLtTD6J3xWTwZ+m+PBUxJo=; b=CmSKIHG8UglAwXkvTS8gLdQsy5QOprfSkMGKTsSLJTMcKqxd1EfAUjXGZ8Wow949Mlx6lK vGkfC3556cX8lKnHOELcAj60xj2YJjYHKoROW1Nm3Em+zlCjpeWv8CGsKbHzDxmwXbh+Mx IJrd1cKcZz/oiix10/YsuTCWnlUSBA4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706097595; a=rsa-sha256; cv=none; b=q99hmeoetM1AN0I+NYmwJu2TQgII2zaaMzQ4PLMN83UTMYLviaIJq/jBQppLP7Kjk75aV/ ARAeC2lFDW9ufGv495YWYKLZX4yczgndK0y29xPOCHcnga6KiVBybPUenKY8ffz8vLlP8W TboR6nQf+A9oMwVvvZHU94yabQS+BD8= X-AuditID: a67dfc5b-d85ff70000001748-93-65b0fbb5e510 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v11 05/26] dept: Tie to Lockdep and IRQ tracing Date: Wed, 24 Jan 2024 20:59:16 +0900 Message-Id: <20240124115938.80132-6-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240124115938.80132-1-byungchul@sk.com> References: <20240124115938.80132-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa2yLYRTHPc9760rtVRLvkEyaDJnb5pZDxDXiSZAt8Y0Ejb2xstXS6iiR zMxchy2mZgvdUNVdtRWGLTXZzXWsrGYWG5nNOqXWWnUurcuXk1/O/5zfp7+Ekt9hJkhU6l2i Rq1MUbBSWjowqnjmjWCVGDdoiYbcE3HgGzxCQ1FlGQstFaUIyuwHMPTVr4Y2vxtB8PFTCgz5 LQiKu95QYG/oRFBjzmSh9f1ocPo8LDTnH2fh4KVKFp71D2PoOJuHodS6Dh6eLsHgCHygwdDH QqHhIA6NXgwBk4UDU0YMdJvPczDcFQ/NnS8ZqGmfDgUXOli4W9NMQ8Otbgytt4tY6Cz7xcDD hiYaWnJzGCj/VMJCv99Egcnn4eC5w4ihKiskyv76k4HGHAeG7MvXMThf3UFQe+QtBmvZSxbu +9wYbNZ8Cr5frUfQfXKAg0MnAhwUHjiJ4PihszQ8/dHIQFbHfAgOFbHLFpH7bg9Fsmy7SY3f SJMHJQKpPv+GI1m17RwxWnXEZo4ll+72YVLs9THEajnKEqs3jyPHBpyYfHryhCNN54I0ee80 4MSJG6SLk8QUVbqomb1kizT52o9iOu3Mvj2vrF+5DPRFeQxFSAR+nnDxsxf954JqBxNmlp8q uFwBKszj+MmCLacntJdKKP7wSMH8+TEbDsbySwXPhZ4/TPMxgutdGw6zjJ8vGOxD/6TRQmmV 448ogl8glBe002GWh27eWk5xYanAH44QDL2FzN+HKOGe2UWfRjIjGmFBcpU6PVWpSpk3K1mv Vu2ZtXVnqhWFKmXaP7zxFvK2rK9DvAQpRsmWWSpFOaNM1+pT65AgoRTjZK6oClEuS1Lq94qa nZs1uhRRW4cmSmjFeNkc/+4kOb9NuUvcIYppouZ/iiUREzLQ7Mxo3XBC/Ot3xhn2zDYdbfeM dQQ56cd1U+oiH62Ymzh4s3r8muyAbr19Um3sjf7I5ZtsXVL3NPOKK0QR1Wj2r9X33LToH5yK 0xlaPyZvH1Mb41yV+KhoDOW+nvji20r78oXtvrTXTamm0SWW3stq70x7whZVfW5l+XPHkHbq 0jwFrU1WxsdSGq3yN5g/m6pOAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/P83yf6zh7dm48YcONMS3OyD6T4Q/0zG9/tfUHbjzq5jq5 0y80pRxSqSbnCP3ayXUpd9n8KLvVlLRINSpplfzolyjXOldyx/zz2Wvv93uvvz4SSn6LmS/R 6E6Kep1aqyRSWronODnwoadcVDWXMpCVpgLXz4s05JbZCDTdL0Fgq0jC0P88BN6NDyHwNL6m wJTThCC/5wMFFbVdCKqKzxFo6ZsNra4RAvU5lwkkF5YReDM4iaHzWjaGEvtuaMgswOB0f6HB 1E/gpikZe89XDG6LlQVL4jLoLb7BwmTPGqjvestAza16Bqo6AsB8u5NAZVU9DbWPejG0PMkl 0GWbZqCh9gUNTVnpDJR+KyAwOG6hwOIaYaHZmYehPMVrM479ZqAu3YnBWPQAQ2v7UwTPLnZj sNveEqhxDWFw2HMo+HX3OYLejGEWzqe5WbiZlIHg8vlrNLyeqmMgpTMIPBO5ZEuwUDM0Qgkp jlihajyPFl4W8MLjGx9YIeVZByvk2aMFR/FKobCyHwv5oy5GsFsvEcE+ms0KqcOtWPj26hUr vLjuoYW+VhPetzBMuvGIqNXEiPrVmw5JI+5N5dNRV0/HtdvH2ET0Q52K/CQ8t443P3YyPibc cr6tzU35WMEt5h3pn725VEJxF2byxd8bia+Yw23mR25//ss0t4xv+/gO+1jGBfGmign0T7qI Lyl3/hX5cev5UnMH7WO5d9NtvcJmImkemmFFCo0uJlKt0QatMhyLiNdp4lYdPh5pR96nsSRM Zj1CP1tCqhEnQcpZsi3WMlHOqGMM8ZHViJdQSoWszf++KJcdUcefEvXHD+qjtaKhGi2Q0Mp5 sh2h4iE5F64+KR4TxShR/7/FEr/5iah5+HDtUYVj4geleElmFQ3qomwhYxnTxrjp9caE0u7N Jxqilrgngs3kd52zLmhgrS1dmR3WwO/qPXuQ3mnox3sDM/OHVQFxqh4uW9s45X9gwQH2TJHx gUdVuYJzXh3Y4L8twfB+HQ4vDM39lLr0hGKrOW10767w2LkD2/cn3ZmrpA0R6jUrKb1B/Qce LEDmMAMAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: ACAA940006 X-Rspam-User: X-Stat-Signature: uj13sbx6i98ccsuyz4s1kz9piibc91cm X-Rspamd-Server: rspam01 X-HE-Tag: 1706097594-692241 X-HE-Meta: U2FsdGVkX1+1qpErmYbFu4JAp4YnHL02kBKgiZyFNFCSPOjrJywrNKkJXphU0z7B0YO728U77CY7iWS2EuLBGT3+CMNUaWpbSp5sywMgwkQJVWwewWRqG25d6pMyL0gJ591IZhAfb8Ncg0Sq9/jwZPsKDLEqvbLYsYnrcDWxMvWk0+S2pG9o3NcaGOMWGFiAT/i3Rfk4LjiVrN0CezZxZQmIVgO/dIjxYZ4GLVnh4ixQKGPmVInyL22ylZs4O8dja97PyCOHCsT32lhSrWqSNV7V8/W9cGeH4gpnTXttbVTtV26IkVy9hmE7vRlJ4hvTOdndbORu6Z6X1V7zT9EwxW41HWew7N2ixyZY2vPNK6mgsH2JYoy2rrEypOaFQDJYIiySGecmJCqpgtNHqgE0G/rSqlwCbKgpG/1QKpwo1wKWdUEO4XakcI4mi9WC9JJyN2o61yaKWp/XPiiMiuzromGJYqVPONOQIrojXuZmPbmuohpH1B6VN0b/snL7j9NMnIlTrDPhw0DjYEZ2ZBp/6V7H96/+SqQ6CqnCqdZc3WwXknunsyOgaVTbPJN6iNY3uURmEXgEVCyGmaAIMCyC53IlosaEtzQ2I0Q8S/l9YjXBOWT2gsydWBWcYu18CD7WYFH7lrrFSutTq1KPWaqdOLSNCtgSc2wtxAQVFCfXFyZ3FsCzaW+5y04D77PfwUXXjoiejsgf6cxNEmSWNZxgvNTssaXDWyYAvny2r9pKfan/JjYpHtGqfGCVjDusZxykfGvokNTS67aMkdKuP1FXupT2dUK+dMe+X2eA2qlnVTOeaLGsIj9WKQdVDp5WTkXrVvwXu4aaj0tvPysAkDCKjGg6/IoFB4ECoRv2Hnm3sw6MR2kNoJUBL3eCNWAQN4NfUeK00A3CG+IpIbEibyjFa9TJIpsL1aO12QoI4Ek2MQmXiPQEBOmQBoSmEYeFLfRhM7KakpFSoH9ftvYYm5K sY/G8UCd QruyKTLsg+qMMeBGROluWCaHqPu3rnXVGUr4W8pRC81ObOspP5vx+Ne5MSg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: How to place Dept this way looks so ugly. But it's inevitable for now. The way should be enhanced gradually. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 7 +- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++------ include/linux/lockdep_types.h | 3 + include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 8 +-- kernel/locking/lockdep.c | 22 ++++++ 14 files changed, 127 insertions(+), 29 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 2b665c32f5fe..672dac1c3059 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -14,6 +14,7 @@ #include #include +#include #include #include @@ -61,8 +62,10 @@ extern void trace_hardirqs_off(void); # define lockdep_softirqs_enabled(p) ((p)->softirqs_enabled) # define lockdep_hardirq_enter() \ do { \ - if (__this_cpu_inc_return(hardirq_context) == 1)\ + if (__this_cpu_inc_return(hardirq_context) == 1) { \ current->hardirq_threaded = 0; \ + dept_hardirq_enter(); \ + } \ } while (0) # define lockdep_hardirq_threaded() \ do { \ @@ -137,6 +140,8 @@ do { \ # define lockdep_softirq_enter() \ do { \ current->softirq_context++; \ + if (current->softirq_context == 1) \ + dept_softirq_enter(); \ } while (0) # define lockdep_softirq_exit() \ do { \ diff --git a/include/linux/local_lock_internal.h b/include/linux/local_lock_internal.h index 975e33b793a7..39f67788fd95 100644 --- a/include/linux/local_lock_internal.h +++ b/include/linux/local_lock_internal.h @@ -21,6 +21,7 @@ typedef struct { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner = NULL, diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index dc2844b071c2..8825f535d36d 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -12,6 +12,7 @@ #include #include +#include #include struct task_struct; @@ -39,6 +40,8 @@ static inline void lockdep_copy_map(struct lockdep_map *to, */ for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) to->class_cache[i] = NULL; + + dept_map_copy(&to->dmap, &from->dmap); } /* @@ -466,7 +469,8 @@ enum xhlock_context_t { * Note that _name must not be NULL. */ #define STATIC_LOCKDEP_MAP_INIT(_name, _key) \ - { .name = (_name), .key = (void *)(_key), } + { .name = (_name), .key = (void *)(_key), \ + .dmap = DEPT_MAP_INITIALIZER(_name, _key) } static inline void lockdep_invariant_state(bool force) {} static inline void lockdep_free_task(struct task_struct *task) {} @@ -548,33 +552,89 @@ extern bool read_lock_is_recursive(void); #define lock_acquire_shared(l, s, t, n, i) lock_acquire(l, s, t, 1, 1, n, i) #define lock_acquire_shared_recursive(l, s, t, n, i) lock_acquire(l, s, t, 2, 1, n, i) -#define spin_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define spin_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define spin_release(l, i) lock_release(l, i) - -#define rwlock_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) +#define spin_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define spin_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define spin_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwlock_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) #define rwlock_acquire_read(l, s, t, i) \ do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, !read_lock_is_recursive());\ if (read_lock_is_recursive()) \ lock_acquire_shared_recursive(l, s, t, NULL, i); \ else \ lock_acquire_shared(l, s, t, NULL, i); \ } while (0) - -#define rwlock_release(l, i) lock_release(l, i) - -#define seqcount_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define seqcount_acquire_read(l, s, t, i) lock_acquire_shared_recursive(l, s, t, NULL, i) -#define seqcount_release(l, i) lock_release(l, i) - -#define mutex_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define mutex_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define mutex_release(l, i) lock_release(l, i) - -#define rwsem_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define rwsem_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define rwsem_acquire_read(l, s, t, i) lock_acquire_shared(l, s, t, NULL, i) -#define rwsem_release(l, i) lock_release(l, i) +#define rwlock_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define seqcount_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_acquire_read(l, s, t, i) \ +do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, false); \ + lock_acquire_shared_recursive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define mutex_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define mutex_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define mutex_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwsem_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define rwsem_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define rwsem_acquire_read(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_shared(l, s, t, NULL, i); \ +} while (0) +#define rwsem_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) #define lock_map_acquire(l) lock_acquire_exclusive(l, 0, 0, NULL, _THIS_IP_) #define lock_map_acquire_try(l) lock_acquire_exclusive(l, 0, 1, NULL, _THIS_IP_) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 2ebc323d345a..aecd65836b2c 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -11,6 +11,7 @@ #define __LINUX_LOCKDEP_TYPES_H #include +#include #define MAX_LOCKDEP_SUBCLASSES 8UL @@ -77,6 +78,7 @@ struct lock_class_key { struct hlist_node hash_entry; struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES]; }; + struct dept_key dkey; }; extern struct lock_class_key __lockdep_no_validate__; @@ -194,6 +196,7 @@ struct lockdep_map { int cpu; unsigned long ip; #endif + struct dept_map dmap; }; struct pin_cookie { unsigned int val; }; diff --git a/include/linux/mutex.h b/include/linux/mutex.h index a33aa9eb9fc3..04c41faace85 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -26,6 +26,7 @@ , .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define __DEP_MAP_MUTEX_INITIALIZER(lockname) diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index 36b942b67b7d..e871aca04645 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -21,7 +21,7 @@ struct percpu_rw_semaphore { }; #ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) }, #else #define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) #endif diff --git a/include/linux/rtmutex.h b/include/linux/rtmutex.h index 7d049883a08a..35889ac5eeae 100644 --- a/include/linux/rtmutex.h +++ b/include/linux/rtmutex.h @@ -81,6 +81,7 @@ do { \ .dep_map = { \ .name = #mutexname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(mutexname, NULL),\ } #else #define __DEP_MAP_RT_MUTEX_INITIALIZER(mutexname) diff --git a/include/linux/rwlock_types.h b/include/linux/rwlock_types.h index 1948442e7750..6e58dfc84997 100644 --- a/include/linux/rwlock_types.h +++ b/include/linux/rwlock_types.h @@ -10,6 +10,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL), \ } #else # define RW_DEP_MAP_INIT(lockname) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 1dd530ce8b45..1fa391e7770a 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -22,6 +22,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, #else # define __RWSEM_DEP_MAP_INIT(lockname) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index e92f9d5577ba..dee83ab183e4 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -81,7 +81,7 @@ static inline void __seqcount_init(seqcount_t *s, const char *name, #ifdef CONFIG_DEBUG_LOCK_ALLOC # define SEQCOUNT_DEP_MAP_INIT(lockname) \ - .dep_map = { .name = #lockname } + .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) } /** * seqcount_init() - runtime initializer for seqcount_t diff --git a/include/linux/spinlock_types_raw.h b/include/linux/spinlock_types_raw.h index 91cb36b65a17..3dcc551ded25 100644 --- a/include/linux/spinlock_types_raw.h +++ b/include/linux/spinlock_types_raw.h @@ -31,11 +31,13 @@ typedef struct raw_spinlock { .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SPIN, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define SPIN_DEP_MAP_INIT(lockname) \ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define LOCAL_SPIN_DEP_MAP_INIT(lockname) \ @@ -43,6 +45,7 @@ typedef struct raw_spinlock { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define RAW_SPIN_DEP_MAP_INIT(lockname) diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 127ef3b2e607..f6b8266a4bfd 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const char *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) -#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name }, +#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name, .dmap = DEPT_MAP_INITIALIZER(srcu_name, NULL) }, #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ int init_srcu_struct(struct srcu_struct *ssp); diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a3e774479f94..7e12e46dc4b7 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -244,10 +244,10 @@ static bool dept_working(void) * Even k == NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__; static bool valid_key(struct dept_key *k) { - return &__dept_no_validate__ != k; + return &__lockdep_no_validate__.dkey != k; } /* @@ -1936,7 +1936,7 @@ void dept_softirqs_off(void) dept_task()->softirqs_enabled = false; } -void dept_hardirqs_off(void) +void noinstr dept_hardirqs_off(void) { /* * Assumes that it's called with IRQ disabled so that accessing @@ -1958,7 +1958,7 @@ void dept_softirq_enter(void) /* * Ensure it's the outmost hardirq context. */ -void dept_hardirq_enter(void) +void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 151bd3de5936..e27cf9d17163 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -1215,6 +1215,8 @@ void lockdep_register_key(struct lock_class_key *key) struct lock_class_key *k; unsigned long flags; + dept_key_init(&key->dkey); + if (WARN_ON_ONCE(static_obj(key))) return; hash_head = keyhashentry(key); @@ -4310,6 +4312,8 @@ static void __trace_hardirqs_on_caller(void) */ void lockdep_hardirqs_on_prepare(void) { + dept_hardirqs_on(); + if (unlikely(!debug_locks)) return; @@ -4430,6 +4434,8 @@ EXPORT_SYMBOL_GPL(lockdep_hardirqs_on); */ void noinstr lockdep_hardirqs_off(unsigned long ip) { + dept_hardirqs_off(); + if (unlikely(!debug_locks)) return; @@ -4474,6 +4480,8 @@ void lockdep_softirqs_on(unsigned long ip) { struct irqtrace_events *trace = ¤t->irqtrace; + dept_softirqs_on_ip(ip); + if (unlikely(!lockdep_enabled())) return; @@ -4512,6 +4520,8 @@ void lockdep_softirqs_on(unsigned long ip) */ void lockdep_softirqs_off(unsigned long ip) { + dept_softirqs_off(); + if (unlikely(!lockdep_enabled())) return; @@ -4859,6 +4869,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, { int i; + ldt_init(&lock->dmap, &key->dkey, subclass, name); + for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) lock->class_cache[i] = NULL; @@ -5630,6 +5642,12 @@ void lock_set_class(struct lockdep_map *lock, const char *name, { unsigned long flags; + /* + * dept_map_(re)init() might be called twice redundantly. But + * there's no choice as long as Dept relies on Lockdep. + */ + ldt_set_class(&lock->dmap, name, &key->dkey, subclass, ip); + if (unlikely(!lockdep_enabled())) return; @@ -5647,6 +5665,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigned long ip) { unsigned long flags; + ldt_downgrade(&lock->dmap, ip); + if (unlikely(!lockdep_enabled())) return; @@ -6447,6 +6467,8 @@ void lockdep_unregister_key(struct lock_class_key *key) unsigned long flags; bool found = false; + dept_key_destroy(&key->dkey); + might_sleep(); if (WARN_ON_ONCE(static_obj(key)))