From patchwork Wed Apr 5 02:26:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13201111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F4B6C761A6 for ; Wed, 5 Apr 2023 02:27:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB6F86B007D; Tue, 4 Apr 2023 22:27:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA9E36B007B; Tue, 4 Apr 2023 22:27:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FDA76B0080; Tue, 4 Apr 2023 22:27:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8446B6B007B for ; Tue, 4 Apr 2023 22:27:19 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5AD5F8015E for ; Wed, 5 Apr 2023 02:27:19 +0000 (UTC) X-FDA: 80645750598.09.EEDC559 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf23.hostedemail.com (Postfix) with ESMTP id C163014000B for ; Wed, 5 Apr 2023 02:27:17 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=4BmqmFWL; spf=none (imf23.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680661637; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=liwfgRAmKok6/KSZncO+fQ8xOc+1qIirOvQozJbmQE8=; b=bHkkILQ/LPVcJRpeSpn1EDcltZPzIZZOHQJFjsDMdoCmmVT0eUsnOG6S+cxB5NVdU8N1BR xSqQdqcP4p4TOethgzrpdQq0G0ztq7xQFXgcBbmxsjqTIH2c/09Uz4znk2T9bOTKn71AfE ga5BErgx6TsF1+oronXprRFfjrgJKmQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=4BmqmFWL; spf=none (imf23.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680661637; a=rsa-sha256; cv=none; b=cYkvLSLmyh4E6eWD+HgTb51Mie8sGsXsbPu4ONMuJKLQvKXOiXafiuiN0KvlCtmhUXQ3Jt 4zTCxS6ZlSdMufHcLHy3ZYaO/k3/ErJniMFh3nQmZbLU8nbF3tla8LvzletIDFundxKDom q1lvDwXhdivpxzBjBlO1iHLDno6oZmM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=liwfgRAmKok6/KSZncO+fQ8xOc+1qIirOvQozJbmQE8=; b=4BmqmFWLry1SHUJ0Tq5b7pp92/ feseonXDmF3CWCJJacVakhyupfia8MqOB/G11JJDU0m6r+ttpBtfyureiWgDEOZ5IlHPzOSqtn5Ic M3tUevN407qt5DsaER/bqP4Ys/NfkBMmn5FxJLa2COKVDrM9LxO38rNJEPArux++t3y9zb3vzKPYl kykHR0/z0lMkZDVbZrNjDgI6ReCPxSXLocqTyrhX43Uqdcy93IWtlwejhfyWRgoIv1ikwcCNaWtUI FBzRL9GvDH4XJ0L+dahhBVs778M0834muN7f/xPaOYV+1h8KJJe1MFKdz3C5xqfRvV6C1pVJA9o21 CAJZ9pVA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.96 #2 (Red Hat Linux)) id 1pjsrb-003A03-1M; Wed, 05 Apr 2023 02:27:07 +0000 From: Luis Chamberlain To: david@redhat.com, patches@lists.linux.dev, linux-modules@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, pmladek@suse.com, petr.pavlu@suse.com, prarit@redhat.com, torvalds@linux-foundation.org, gregkh@linuxfoundation.org, rafael@kernel.org Cc: christophe.leroy@csgroup.eu, tglx@linutronix.de, peterz@infradead.org, song@kernel.org, rppt@kernel.org, dave@stgolabs.net, willy@infradead.org, vbabka@suse.cz, mhocko@suse.com, dave.hansen@linux.intel.com, colin.i.king@gmail.com, jim.cromie@gmail.com, catalin.marinas@arm.com, jbaron@akamai.com, rick.p.edgecombe@intel.com, mcgrof@kernel.org Subject: [PATCH v2 0/6] module: avoid userspace pressure on unwanted allocations Date: Tue, 4 Apr 2023 19:26:56 -0700 Message-Id: <20230405022702.753323-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.38.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: C163014000B X-Stat-Signature: dehtxmpgt98bruqutq8qhtwogyqdob4z X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1680661637-803348 X-HE-Meta: U2FsdGVkX1+mMA/oLc3J10Un8ln1UIbXse+4OKUH4BCqrfbh9RoYfjZURP8i3JSpzIeEjtaafAr8vBbFFxqs+b3jOVaHWmoX6+qrwQZjSc2ZNlsF7dR4TXfyamkWlKIeMliSEYdTYHbCPKa3lP0zeLOC/9KbW3G4cCGsHacWE4dv56yl59634/s3iAFFJ1vdOWFFe1B5hWFTEXHMSDgQX65IeBgnaOjWtR6gjfwKazUXiAPk/tru6R1wRK9+etY2PP2xsbMkVSPauNgmM0wR/UrQ8kXQ8cun52vpJQR0UwMB26vozumcUqIxnCCF9iC4+oq+BjPb48twpVPgfVJIjZnr6gCO8D/5mpYqFs+j90od+21Vs+J3jXQJcprohm6Wg5kfP6+HhQ5EPp6aUx4ttGTCaae/YD3rISRfa7fz3Fe+eXWTnFPUvKezwDQe8x+loiNbsxAiV5b8HR1jCPGyGL3QbUlg4rrJ++sfdsh9Islfaysf4hzxcsEEqOVu/ES794G74tms2VgFeR8UkXbR15Fed2uCArYcMTaGHkkuFrHbwWy8J4N1355PNoklSm9RNHk6uUPg+s7kD3E8xA5/1tryf4amQBEN1x/U0pvfG1j2/TzhbiIWdNZIXbDDmspIgC8HuBTjJ2KQ9b7g/KhnNjvZnT0FOn/1NHU0a9SjlcXmZEl3ao85sKERIDaeIU6SsVoW9MKOyXZBQGgsWcPRAAIDeRob9b9S0qZiihm6+1M5xuAk5VGEYlwgcOxRC+r8iXXD6jnU2Oxihxtx3cY+apBidRCJm0ma0Xj3AbuN2o2y1eS1tKq0nDRgXcWC6pyQiy5Hc07Ewefmmfo9bgb3Bd3ScXYvnC0HzWtLTuz1ulXkBlw02IfJNbI0JRvsuVV5chn6QNAp4oAeXLv3vH9m6XFet1tezmXZtwmJLgI+x7/Hji4ENaC1FQv0JmzsIG2+wXR+3NrZuixNb1rB6pj 4+1XVckM COeRvTMUXkTpVgXlNA7gt4No2IRZhB3bxhqlk6lQx04yui8O8UtktpxA+lQZJscrWUEtg01yJWl/C003F6gdAymvd8c/77W/gDF4p9V+k8DD4zWLrvrDftzFjVA8OCvcHdHTV1ryHY1CKKPyIng7dlml0BNsCdw1v98/yGetLKNX2JnAWxVneJ+w1RiVaYj4bPocRAuYaPW8Byf5uHcGU89tw6TFaBNs4i4bm8qOEKKZURq1s/Nx+OQIPi17+03OWrniopAr1jDnuy3tXwkd7pGW26S8fj4qkCiefa7+xwUjHC43TlLqhwwFJohg9PJOgJWuL9eMsVFRQhjVxxNLBLuPOdPA13nImmxasP3QGrGS21KSo5FvgGnc6f2rQb/O0aOEpaYoX49ELNSXlx4JjP2BM6uWW1WcsT4sRBJOCLbHagi7ThYyHBKTEJKFLfhmsJ8WYaIBXwIDDp2pk0Cbg8ydYLRTWGXt0tpZ8HmhCuM97mdE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This v2 series follows up on the first iteration of these patches [0]. They have the following changes made: o Rolled in fix for an kmemleak issue reported by Jim Cromie o Dropped from this series all the semaphore & and simplifications on kmod.c as that should just be sent as a separate bike-shedding opporunity patch series and it does not in any way address the the unwanted allocations. o The rest of the feedback was just from Greg KH and I've addressed all his feedback. I decided to do away with the debug.c as a separate file and leave the #ifdef CONFIG_MODULE_DEBUG eyesore at the end of main.c. I guess it's not so bad there. o *Tons* of fixes and enhancements to my counters, including tons of documentation to help ensure we don't loose track of some of the tribal knowledge and so to help ensure we have references to what our accounting looks like. Those large wasted virtual memory allocations on a simple qemu idle boring boot are simply rediculous, I am quite baffled we had not spotted this before, and so it all reveals we have quite a bit of optimizations left to do to make loading modules an even more smoother experience at bootup. If you'd like a tree this is on my 20230404-module-alloc-opts branch based on modules-next [1]. [0] https://lkml.kernel.org/r/20230329053149.3976378-1-mcgrof@kernel.org [1] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/log/?h=20230404-module-alloc-opts Luis Chamberlain (6): module: fix kmemleak annotations for non init ELF sections module: move finished_loading() module: extract patient module check into helper module: avoid allocation if module is already present and ready debugfs: add debugfs_create_atomic64_t for atomic64_t module: add debug stats to help identify memory pressure Documentation/core-api/kernel-api.rst | 22 +- fs/debugfs/file.c | 36 +++ include/linux/debugfs.h | 2 + kernel/module/Kconfig | 37 +++ kernel/module/Makefile | 1 + kernel/module/decompress.c | 4 + kernel/module/internal.h | 74 +++++ kernel/module/main.c | 198 ++++++++---- kernel/module/stats.c | 428 ++++++++++++++++++++++++++ kernel/module/tracking.c | 7 +- 10 files changed, 742 insertions(+), 67 deletions(-) create mode 100644 kernel/module/stats.c