From patchwork Mon Mar 11 16:46:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 13589088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3764FC5475B for ; Mon, 11 Mar 2024 16:47:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 545C16B00D9; Mon, 11 Mar 2024 12:46:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F6B56B00DA; Mon, 11 Mar 2024 12:46:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 286A46B00DB; Mon, 11 Mar 2024 12:46:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 04FFF6B00D9 for ; Mon, 11 Mar 2024 12:46:59 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C115D40245 for ; Mon, 11 Mar 2024 16:46:58 +0000 (UTC) X-FDA: 81885337716.25.EF6C0C9 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf12.hostedemail.com (Postfix) with ESMTP id 2FD344001C for ; Mon, 11 Mar 2024 16:46:56 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=yQVOfaba; dmarc=pass (policy=none) header.from=soleen.com; spf=pass (imf12.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710175617; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cK/ZqVdlnureqUddSXvrvjnokezl2c4iuxNaHlbvXOk=; b=5QXXU0gjEJUX+MDRIWZN83WKdaJG3kG+xjGWeHRvdvD+R0NqUiJwD/VOgE7Eqxqy1jyxM+ kUEYLsFyYraxOAwzyNQlLxC0J1ULsckhrdrbG1inQnRQxYMNpGh+3WUU6AiFalVYIj78Wv Gm5TAo82nkSPcW1xOzKxR8PWVpVnFzs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=yQVOfaba; dmarc=pass (policy=none) header.from=soleen.com; spf=pass (imf12.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.222.180 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710175617; a=rsa-sha256; cv=none; b=BOIuAnG2ef16KBY+XuW+2A3iCfXIGMmuIwoFbrGqYrvBX1WfuyfUoOtd25rftSGAcbtPmG CUVsylUKCXSMsE8kWDIr5wcls+c46NW8KUWyLppRPsjbLj9/76Wha8mjo3ZGwaqArl78wa uEbqPLAJQ/J9/5XlTKfP1iWhdOw+lwE= Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-7882d75e766so213167585a.1 for ; Mon, 11 Mar 2024 09:46:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen-com.20230601.gappssmtp.com; s=20230601; t=1710175616; x=1710780416; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=cK/ZqVdlnureqUddSXvrvjnokezl2c4iuxNaHlbvXOk=; b=yQVOfaba7cGQ3AxH8o8ixCFlMmwIT1w/FFh1rIUyQIojAg3C56C1znvMySziX4X0y6 I9RpcfQ+gQvg3pqb9H0Rpd7Ro43K3yp1Tv2FgQWHqf1e9M75xDzEKeqJUU1/e95RL+Fz 4ffJht5ju2RlrPCMg/SX22foBdHwgo8slk+raKtNG5JGqM27sEaYaYNKIyH5jTxbtV01 3mLSTwPlIMpW6m/Jiu1pSG14p1p8e8gfdgita6TJRoNXBGiCysJ/MQaNeq8S1nEiY+mJ IFwoGWPzyuegyNwPYjBHNmvIBuQh93+u4okjHlDh8Wk5W8Ap+MEZSZ3A210i2B5FLTWy Ewcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710175616; x=1710780416; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cK/ZqVdlnureqUddSXvrvjnokezl2c4iuxNaHlbvXOk=; b=aF7beIUDWHAzs/6xMsxwoICrq39jSZTJRyVHgxt6j81MG+8w6iluiLYu8b11Kdm/RU PHFjUeHCRRN2g+hg/SwqVYd0BqyNwWkC+8UOjnEfnOuSKtvBoSyhFg14BEUwWxUjC6LA ywGzt8bwsEoJ/aAs6PImhWa5zEslTIliKWMI4DMvnRY7sANQW6HpNXFL5z0ZY9RmA2r/ rBQLN7MUqEoeiOben0abWzf9vD8ETgvljpsN1cHhgMKhZR0BRDnqLgtBffr6b9VeDDHS bQ6bZbxCJjrr/Ks0J0/ol8jAK9j09Ev3d4q/SHW4SHy8XknC421gubhVlqsRAoRsTfLi kgBA== X-Forwarded-Encrypted: i=1; AJvYcCVd3Rb4aC2SQs9O1qAab2QnrXQOLZN9DDq5TGz5oEwDSB0uT+8GykEtNhImdGa1Go9+GLdlooe+w9v4RHbHW7IAcmA= X-Gm-Message-State: AOJu0Yx42nBFRAIGSL9nn05JyoU9Vlehu8nTPH8xDWhvFzse+hBHz88z 0IAHR20pQ+3gIlJDpJ4MbguM2ZLuqE7ysQziWZD8TdUBp79MA+28qz+V5HJIrDo= X-Google-Smtp-Source: AGHT+IEp6VoHaP5KrWImU0EWHAjL+QS9WD2gq+SL5008BAWJX/vKvEeMCLb5ZMa54jxJQaXt/mOFWQ== X-Received: by 2002:ae9:e701:0:b0:788:c09:ce3 with SMTP id m1-20020ae9e701000000b007880c090ce3mr7062849qka.46.1710175616304; Mon, 11 Mar 2024 09:46:56 -0700 (PDT) Received: from soleen.c.googlers.com.com (150.254.86.34.bc.googleusercontent.com. [34.86.254.150]) by smtp.gmail.com with ESMTPSA id d27-20020a05620a137b00b00788228fbe05sm2851589qkl.17.2024.03.11.09.46.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Mar 2024 09:46:55 -0700 (PDT) From: Pasha Tatashin To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org, bp@alien8.de, brauner@kernel.org, bristot@redhat.com, bsegall@google.com, dave.hansen@linux.intel.com, dianders@chromium.org, dietmar.eggemann@arm.com, eric.devolder@oracle.com, hca@linux.ibm.com, hch@infradead.org, hpa@zytor.com, jacob.jun.pan@linux.intel.com, jgg@ziepe.ca, jpoimboe@kernel.org, jroedel@suse.de, juri.lelli@redhat.com, kent.overstreet@linux.dev, kinseyho@google.com, kirill.shutemov@linux.intel.com, lstoakes@gmail.com, luto@kernel.org, mgorman@suse.de, mic@digikod.net, michael.christie@oracle.com, mingo@redhat.com, mjguzik@gmail.com, mst@redhat.com, npiggin@gmail.com, peterz@infradead.org, pmladek@suse.com, rick.p.edgecombe@intel.com, rostedt@goodmis.org, surenb@google.com, tglx@linutronix.de, urezki@gmail.com, vincent.guittot@linaro.org, vschneid@redhat.com, pasha.tatashin@soleen.com Subject: [RFC 13/14] task_stack.h: Add stack_not_used() support for dynamic stack Date: Mon, 11 Mar 2024 16:46:37 +0000 Message-ID: <20240311164638.2015063-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.44.0.278.ge034bb2e1d-goog In-Reply-To: <20240311164638.2015063-1-pasha.tatashin@soleen.com> References: <20240311164638.2015063-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2FD344001C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: d7a9nwsaqxpqasz5z33uwb1uiaucuhhp X-HE-Tag: 1710175616-441745 X-HE-Meta: U2FsdGVkX1+bb45xJZ/pEf3q0kpu0togTEDC/AhG95D2zADM6v9qCyZUKsZDRzMfsBQsQvZOPH8EaFUlsBv/Q1xMBiMTx4iLkknVSeTV+UkOj+lBq9a+rfqRC9T8W/tROemH2fIJjwSqw9/aLtOSqePjhllB2ok2AeZq3ICByAujs49CdHYTKtCJdCxHhoeTLaP2a3DwOfhfq9ajLY7uxHcYiij9EmSnuNNOl4AhO6DLvdUOZ3MtnayTQZzNN14gCdrdol1EI2lZ/2ktGKqnzO2Os7AZYLC4kuT5cLE+/xHnKxeEgWsNNzmN1E4BWRngZ/F2he91kxlqGOljVXsxowCr0jegxhmp43yyFAJzbK23gNzu7LQjT1R1NtXlUO2eU9N0pg0USYDs3y9TfZVgCdbNWBl5gFokDaYXFarufrqjCVr1JEwx5u31S0WkkRhIxOn1AEd0XGqtJQJ8wiTlG5R+0LdDm5IIQAHdTGY8pm1Gne1HSItQd09zX7nPUH0LYFCnXuzJVfc/0cTq+CmTiUxVSSvM2Hxa7BEK2wPPD/YoRRnsyURhZ0EzcZ1KgS8NAPr++/aG+60yf6bJn9Ab2/XJ5N/Jby4mdEN6dPOZLl7DiYFeBxdXrFiQbQS6L47XtH4Fy7eah2dEWHVEqSjzinuUSQsMKUFLjwcFi5bI5c37JW4X7JCbKL+OSRMpghKseRAzB7WotOfev8x6oEpuKQfASFlbS/5djDL8dezl83RiGQ5NwT1D83xavdQCByJooNJwD4VNLcPjwPQHdN212BlyIz/zoooh9kDvPQhjRfUIqSX+oeX05kojIk1iaJCy8eWWLgroac13bAVk/6KCvRZIuQbmA7AI+vHWFa8negvGt4WayZzyC2wZQw5Ii1qiQwcNzpSYI+blqw1oye/9Sr3szEd4NKs/7sJzVnM11+onibdnJMrGRcBvkz3wAoZkkSg/eaUpPkcijXG3gW1 7b4l8tQb MG0a3m0ZLGQt5GpMrscckSM4YeOzJeGnEaIi6UVYCH5A8wVII5r6ooYfOE+fMREar7cHmBDpZlsZJZWwsMwJUtqW54uyLQ1uXj+Xb3P8u/hFhbgOZ+3fOG6r1s1IYQ4EVSORyyNUHy/s3FhroEA+8L9N/3AehtwrUAcFkJRkJyGCDljdSMPas7+HRF942UvjdOeA7LQ1EEA/vGbz+TKxKsdPFyTk2CitHnqOg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: CONFIG_DEBUG_STACK_USAGE is enabled by default on most architectures. Its purpose is to determine and print the maximum stack depth on thread exit. The way it works, is it starts from the buttom of the stack and searches the first non-zero word in the stack. With dynamic stack it does not work very well, as it means it faults every pages in every stack. Instead, add a specific version of stack_not_used() for dynamic stacks where instead of starting from the buttom of the stack, we start from the last page mapped in the stack. In addition to not doing uncessary page faulting, this search is optimized by skipping search through zero pages. Also, because dynamic stack does not end with MAGIC_NUMBER, there is no need to skeep the buttom most word in the stack. Signed-off-by: Pasha Tatashin --- arch/Kconfig | 1 - include/linux/sched/task_stack.h | 38 +++++++++++++++++++++++--------- 2 files changed, 27 insertions(+), 12 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index da3df347b069..759b2bb7edb6 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1266,7 +1266,6 @@ config DYNAMIC_STACK depends on HAVE_ARCH_DYNAMIC_STACK depends on VMAP_STACK depends on !KASAN - depends on !DEBUG_STACK_USAGE depends on !STACK_GROWSUP help Dynamic kernel stacks allow to save memory on machines with a lot of diff --git a/include/linux/sched/task_stack.h b/include/linux/sched/task_stack.h index 396d5418ae32..c5fb679b31ee 100644 --- a/include/linux/sched/task_stack.h +++ b/include/linux/sched/task_stack.h @@ -9,6 +9,7 @@ #include #include #include +#include #ifdef CONFIG_THREAD_INFO_IN_TASK @@ -109,6 +110,21 @@ static inline void dynamic_stack(struct task_struct *tsk) static inline void set_task_stack_end_magic(struct task_struct *tsk) {} +#ifdef CONFIG_DEBUG_STACK_USAGE +static inline unsigned long stack_not_used(struct task_struct *p) +{ + struct vm_struct *vm_area = p->stack_vm_area; + unsigned long alloc_size = vm_area->nr_pages << PAGE_SHIFT; + unsigned long stack = (unsigned long)p->stack; + unsigned long *n = (unsigned long *)(stack + THREAD_SIZE - alloc_size); + + while (!*n) + n++; + + return (unsigned long)n - stack; +} +#endif /* CONFIG_DEBUG_STACK_USAGE */ + #else /* !CONFIG_DYNAMIC_STACK */ #define task_stack_end_corrupted(task) \ @@ -123,17 +139,6 @@ static inline bool dynamic_stack_fault(struct task_struct *tsk, return false; } -#endif /* CONFIG_DYNAMIC_STACK */ - -static inline int object_is_on_stack(const void *obj) -{ - void *stack = task_stack_page(current); - - return (obj >= stack) && (obj < (stack + THREAD_SIZE)); -} - -extern void thread_stack_cache_init(void); - #ifdef CONFIG_DEBUG_STACK_USAGE #ifdef CONFIG_STACK_GROWSUP static inline unsigned long stack_not_used(struct task_struct *p) @@ -160,6 +165,17 @@ static inline unsigned long stack_not_used(struct task_struct *p) #endif /* CONFIG_STACK_GROWSUP */ #endif /* CONFIG_DEBUG_STACK_USAGE */ +#endif /* CONFIG_DYNAMIC_STACK */ + +static inline int object_is_on_stack(const void *obj) +{ + void *stack = task_stack_page(current); + + return (obj >= stack) && (obj < (stack + THREAD_SIZE)); +} + +extern void thread_stack_cache_init(void); + static inline int kstack_end(void *addr) { /* Reliable end of stack detection: