From patchwork Sat Jan 21 07:10:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47A57C38141 for ; Sat, 21 Jan 2023 07:11:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B96BD6B0073; Sat, 21 Jan 2023 02:11:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B46716B0074; Sat, 21 Jan 2023 02:11:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94B376B0075; Sat, 21 Jan 2023 02:11:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 852946B0073 for ; Sat, 21 Jan 2023 02:11:03 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5C8F640F88 for ; Sat, 21 Jan 2023 07:11:03 +0000 (UTC) X-FDA: 80377934406.21.1126704 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf23.hostedemail.com (Postfix) with ESMTP id BF60F140008 for ; Sat, 21 Jan 2023 07:11:01 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=kCJyVDge; spf=none (imf23.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285061; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rOVb8YD5fFvB47xecqte18KDPoif4B7pWlksRr6JsfI=; b=8Fi3wfxCOswaPfI3TsIiZo06XwZfKaZK9ilxnlDOe/M3Ij9ZEttyBbjDCSSF46m0Q0TbmG Pn22+0bPwqeA1e4yd9BVEgWpvHOycDvaYQtqi5oWhnIGHkpvT0NLFvRXjMmujk308xPeY0 oXtKMbKhyt7LKEsQlHMegmd8kRhZG5k= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=kCJyVDge; spf=none (imf23.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285061; a=rsa-sha256; cv=none; b=n3eOqS5/w3cRqEOf1w57Bl4LsSO0d4QkFOWU1VOG3PKaEXTHYWMVtFIsW5qf5krvORekDc 3IA/ZA9enaBqi6Y41C2nx+CG9r3MofzTy7SdBSm60061OMKDKvLw14V4wDjoNUG6YKqQFY GIJ0YORkv9T6SFFsYp83LYxk07EFv8k= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=rOVb8YD5fFvB47xecqte18KDPoif4B7pWlksRr6JsfI=; b=kCJyVDgeQH6z/iZDM/mUN/yxuH l+QJqoyoBYXERE4spfq9xVrIccMlrvxWLdDmz6SZ20FWbzDgbqIS/2ISZ25MOwtK2vT2QJWJg85Co 2AzFXYbwRZ94isPHXSBRu0QMDiWpdEs+sVHCu7hb0s1bZ7LRjSZyuI009/hsZUr30tqLAENh0nh9G o85uxbt6i/pHHPFD3eAZFoTPqhIQl8T2j64qmTGreeJAZ3C2BDYtAbq55T4Gp3VvRIZOVxcTULeo5 CEIrC5nQ5/kkGfV1nX1Y4ykeAqJGN77iRZZbpY2EirgS8Ha3EgtL0FQuriHiLgUl2egJeow3fxDu5 bK+OowQQ==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81g-00DTmK-Bn; Sat, 21 Jan 2023 07:10:56 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 01/10] mm: reject vmap with VM_FLUSH_RESET_PERMS Date: Sat, 21 Jan 2023 08:10:42 +0100 Message-Id: <20230121071051.1143058-2-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: BF60F140008 X-Stat-Signature: 9e1uteype9bn6s16pymesgkcxngnrzdx X-Rspam-User: X-HE-Tag: 1674285061-376404 X-HE-Meta: U2FsdGVkX1/0XRbVq8cZvEpto6HF0YYMj8N0AO9ckRPGPGrP+QoDD5DVyxK5spCz+o+CfrL4LKiXJ7foD6W5zKcPvQ4ljOTYkVu3+HHK2UuDH2wm8QMHbdEuTUPQfw1YZXbxqJiO05VX+Y4rWWbFTE6dNw0cKgl6nNFcXjklvdTM4Pf+vqDaigLPeWqRTCXJCIkXfneXzU5b3P2j3jeFYQcCgiO6XBLcU4SOeg8IiDXoonr3Jmkk/FCpCKe6B+yn5sjg/jpcAAhZkKHV2psZilmpZBS3MQGn2e2aNOUWYUf+MJWJTRSBwFI0G5qfHFv891ihEEaMoEMjTheaHGICL8LllJ/XUYZjneTi2GaEyKO9Mlyy332A+T9F0BBUxxI4+ey58TVa97NbON4QGgyql1gnC0OL9CT63FRdLBo9yjuJyfrNV7+BzmhdZLtqTDeHyulvLrj0YkSEbh08e5hzZDv2z/cE0P+M38CNZFtEYm3taMcV3Q7N/BC5AE68r0YQuojAKTyrpCN5t8SNg0IL341UaG/Fun9VFkXi2afrfJ7irnWphh5KaDwwJD682QVhmNTY7ZHdz3ykoTV+a80tQJls9XbOReXOEpTyIFi1vhfKvjfv0la45ahLE4y/WmgsDxK6xQBn07TysQyf/li/tHvuJWBjMsYdQaNZN5v4QW/ZyFS980n/v9UPHBpB3JL2kfKy4jr7eTGd7fKiPBWxYAl2eOYQPaVeF4T6DYQVoUch0uo3DnMexaNfq8lWB62b0LxipGdhBYxjLpD1/SfBupZeriD+7UWqoievVAE1M1CAPpBd6/BgNaj7PHsKip7ekzLWY8Dft/KFqc5zsvzBlrCE+o8nQV7eia+wHKK4xhWUkAZrmvcxVVYHjnQE+NR6rpDdhbYc5i+PDUeBUgO+k1HzpINLrYqTR96CGOYeB6Ame5LTAuMs7qDuZObJuy8GDEwo2/q3kSWsRgL49ZY MLd1wfap j+/iAGDAFMo2To2a2sSXc13R1h9Kh6r5GEWRmZFvlsm55rBt7ifrpTg73U1mQxsm0ezuh1S/rgERqAek8MWoiGJB4YPXJha33SUEX7bVIr08rSfA0R7R05WDi9f0Pua/E6F4hWrpcma+tOTQRvGyp1ZBvytxgumJ0wN4iF7qHShedUWpk8XKLGRkJoG/TaA/UIMoTwv2VdpfpLjpjVOnlho1U1Kd2aIe21FiMBpE8vB2/vqjDwQaxTgQq+BjOdugpoYOhrBIOUgYwwu+47FSmPWprYETnSbXw0oPMsgoKruiNEkNbbQtTbu2IeCcPRTZJrBWhMaTDLZpSKzY1fv9ng/nvipAFd6+0XhQpuj5M4PsSP4S8G3nyJgzctiN2pi6TublJqePNx5tXJtA+0aQyP4GZb6ZC2kfS82TRQMJsg66FZx1V2x+hEK0vjYjLR4B6fD3/qV96ZuY7lulTSQNn7gQLEuxX/K5ec/PI66kSyZWL4jOORx8NWGuRJkulPQtFpt2CZV9P+jmiV6MoOGjQ5mROOIQb7sFD8c5C3YdAlUUAKbB7ZoqiwAjPtH2PA0x6cM1p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: VM_FLUSH_RESET_PERMS is just for use with vmalloc as it is tied to freeing the underlying pages. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 0781c5a8e0e73d..6957d15d526e46 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2883,6 +2883,9 @@ void *vmap(struct page **pages, unsigned int count, might_sleep(); + if (WARN_ON_ONCE(flags & VM_FLUSH_RESET_PERMS)) + return NULL; + /* * Your top guard is someone else's bottom guard. Not having a top * guard compromises someone else's mappings too. From patchwork Sat Jan 21 07:10:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB08AC004D4 for ; Sat, 21 Jan 2023 07:11:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 926466B0074; Sat, 21 Jan 2023 02:11:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D5EE6B0075; Sat, 21 Jan 2023 02:11:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79E166B0078; Sat, 21 Jan 2023 02:11:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6ACC06B0074 for ; Sat, 21 Jan 2023 02:11:06 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 43853160F6C for ; Sat, 21 Jan 2023 07:11:06 +0000 (UTC) X-FDA: 80377934532.07.0B75FBA Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf05.hostedemail.com (Postfix) with ESMTP id 8E3A010000D for ; Sat, 21 Jan 2023 07:11:04 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b="ovyG/4CM"; spf=none (imf05.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5+yIJbbq41O5DQj1Bid/P+oY4/gY3/DwnrWQQ9o5sRM=; b=xkVfPBunDtXtIZCECLO/TwWU6OQvgr2gROMk3QA2qE+y2JjA9kdty3aJYr0zuW2pObC/hA RXD+wcUOD/5fth7a+4jvxIIhvvL9vQV/sm0QBE6F5LbEoPVdeD/xsWDFFnz/9W6iwvYjX4 C5kdz5TKGItM5/SWljnS+m4fveYPxcM= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b="ovyG/4CM"; spf=none (imf05.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285064; a=rsa-sha256; cv=none; b=vB3L08pNPihCePksm8HiWcLfd1oEy8is3x/issRl7o1pLDhE5tD8SrIVhWFUCwRFGM3iNe XzNgRd6zSuorR17veN9AgL6v1FoG5ZiMhM8Y0ol5574h/kOb/ga1OdGptSQ1mSYxSJYAvh W8CcN0Hukk4P7vKg9Oj8xtU0mX5QHg4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=5+yIJbbq41O5DQj1Bid/P+oY4/gY3/DwnrWQQ9o5sRM=; b=ovyG/4CMuYwsiX9e0jV8DpYMTM l4eDRY3sfXqwakMNYgYBLwppN9lG2Jm/at9dKrY/Vv7E7C3vCmR9P9CZLwEmoNlVBgyP7ahs7ri49 4kEOTrfK79qSI1gFOJuijAQepIBDCY2Jhhtbp6bMZjftR8jzV7WE4z2D9s8tWDF+CxrCujhCYrjlf ovi/aJW2MKeHy9Sh5IHPkAoGr53Y3QWKYwjixOsPdLKBaOtBQClD2yRiKHXB7frqOucAf4HGvIqRC PQhMmtBP7kpuRBaxUj5/MGDCl7cTYaWs0Evryo0iuAH5PRQS6s74TP3TaARaDeWgaST9rfX8WDw6z xNWyYgwQ==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81j-00DTmf-0Z; Sat, 21 Jan 2023 07:10:59 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 02/10] mm: remove __vfree Date: Sat, 21 Jan 2023 08:10:43 +0100 Message-Id: <20230121071051.1143058-3-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 8E3A010000D X-Stat-Signature: gzn4rtmwo6gug7n53ghusa6nqe8xnd7u X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1674285064-871299 X-HE-Meta: U2FsdGVkX19ckAnM0RlPk+KnA1RL03pMZJS9bhIERNVLu31NRvA2vx6nNKSOB1GLMQ0uNFRIICoC5OB5eIGE81jV5PnHNJtnZJozsdPkECQVLwsYVwM9W/KnJ+sY9xfywFbcJOQqAoKMh2eWwRFqT4bUaqecTo0EsRyQ03Ko6gZkRnAB4sYO4lSVReFQH+UY+JaLJ2XbxjVJ4+8T3mDV6RSM1FisPzGsIPFMw07Xfk2Rtc/hr2Q9IU1ijm2owUxSNNEa2u+7JbjRbL6BT2OO2SYAWCEF83KHUQt3vc43mHKow1VEeaQmaQ/2UoZStDp9yIEr+Gfzpi/YJLB064nhm8M7HEHRpvbTkGlFmT93M9PRhaLtnbGOMeHeADBW1dhhJpbqS7uVKf9GhPxPKYRzI8da3s8Ci6++VIrk7cAdeW6yhjTOox3FinkiOmjb4gcT3LEZkNI83ATey3rVYDkWbCPmzEoHJj8wXX77p0b10jc2ugDho4Ixk+JeZA9xk21cQI2VL8JOZ1BD6mGXv60l8UjCZ54djxUd+Jxr/Ajb2S7UxR3zqSM496jgA39z08tgAlChvwIr7t89v3X4p6flutbGivwVhLt8fX5ewBVBtgzFJQPVVevuJt7yHIJ1K4Dpaoc9EWSGsqpE9zdJNtInJi3EQxRkgQqjEUYtF2xYuYTeG857WlbB5zIt3nlp4E9XGCaChk4kosgn/uj5hVBk9XrdI8MSTYfRQXi/uMPEWeNZ/NghfaX3c2AubVAh7eNqtIcAeALhYJMVOdR2dezCVE3VVURuVKLH9kXLgIkLCXds2hG/oO6tAIfzp3QyitOt0OkPi4Oe+EAd8bSLyp+ow/jzOhPV7dh50KPrKv4k4JYK7jqc+Jeq86j0ckH6OJ7SgoDKD8g48oQnqZayI7UQN8nFwxYvbI8K2IGohywlAGucJfdhIQ/W3INqOgqCRGKdubJ7AAU9KJmzFW+68Z3 43m7q9gp Q3Sh8OdMnjEOEQWQ2ZPEV3enlOA2V66l2Z6EdiK4AWxPb9IeRAN6qefoteTV0ZwKkN+0Vd5mdouRjkdjSmNym9Q64cLPomGwDsr2txFFpe7caFuOj52gCD9OXkSaibeMAS3urTc5siTbFEO64ZN0csoDWilnML1fcbXhJ0PfNQEe/o+WbpxycmfG2tJQx0yNChRz2D1GLVCn5MXlV7/7Kf6KS/Q4rxM9ZZ6iA6k5/brQ2neVEEQJFsldV7izFbnlDhvxJW/g3gjbYiwZAgStUo2c0fgNpjIvE5BOP2ALBTO1dWhD6ZJg2wzTb3vAAsougw1izOVMrPM3anagZ4WOiSl9f5vb/m8Sz6W4jh4qyjpzd6bfGovWCxUWJXuIRfeIip6ig70bx1UQjPj3VUOLefIIN+b/3vqs2d7LG1Gsx/UUqmJs+raFQApE7qS69hxUzdbkJRYRJPsGRshYWiEml7IDERC0ToMM/RhVfHswCk3PGGhE5UErcl8Mjn2e8PfIHV1giSGDPHJ45o/rZOkGvnDchYZQaOPBhi/cmWyAHDGkRMvW0d0AYxlkxOP5V3LwWb0Kf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __vfree is a subset of vfree that just skips a few checks, and which is only used by vfree and an error cleanup path. Fold __vfree into vfree and switch the only other caller to call vfree() instead. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 6957d15d526e46..b989828b45109a 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2801,14 +2801,6 @@ void vfree_atomic(const void *addr) __vfree_deferred(addr); } -static void __vfree(const void *addr) -{ - if (unlikely(in_interrupt())) - __vfree_deferred(addr); - else - __vunmap(addr, 1); -} - /** * vfree - Release memory allocated by vmalloc() * @addr: Memory base address @@ -2836,8 +2828,10 @@ void vfree(const void *addr) if (!addr) return; - - __vfree(addr); + if (unlikely(in_interrupt())) + __vfree_deferred(addr); + else + __vunmap(addr, 1); } EXPORT_SYMBOL(vfree); @@ -3104,7 +3098,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, /* * If not enough pages were obtained to accomplish an - * allocation request, free them via __vfree() if any. + * allocation request, free them via vfree() if any. */ if (area->nr_pages != nr_small_pages) { warn_alloc(gfp_mask, NULL, @@ -3144,7 +3138,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, return area->addr; fail: - __vfree(area->addr); + vfree(area->addr); return NULL; } From patchwork Sat Jan 21 07:10:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A02B8C004D4 for ; Sat, 21 Jan 2023 07:11:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46CC26B0075; Sat, 21 Jan 2023 02:11:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 41BD46B0078; Sat, 21 Jan 2023 02:11:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E2F06B007B; Sat, 21 Jan 2023 02:11:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1ECE46B0075 for ; Sat, 21 Jan 2023 02:11:09 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EA0E0C0A87 for ; Sat, 21 Jan 2023 07:11:08 +0000 (UTC) X-FDA: 80377934616.24.F34B1B6 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf21.hostedemail.com (Postfix) with ESMTP id 406161C000E for ; Sat, 21 Jan 2023 07:11:07 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BhY3pELB; spf=none (imf21.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285067; a=rsa-sha256; cv=none; b=OR0IT0uhXhRwVkBnVljeYb8lDI6Wp+Xu3MNzewiY4UmXlalG54MaQjbqm0P+npk+RY2R2O aYua88gFhFzIcrTzA50RFjLv796MrLaHzU029M4Jy2XULVfVL/jj87jOJVOB74AeuSRZWL o4z85y43P6BRE1YEb7ek4r77OsnDp3I= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BhY3pELB; spf=none (imf21.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eBVOWMkt3qqfmS1eY0rzR8noKiZ5EvmTGPqU8LqtGbE=; b=ZVhnD9zaK+5LxIoJf6xAt06u4Fay7cRNe+HwT9oEUi98Linh0bBm1R1scgloiNhcxY4xPb v7OwwZ+svIBtcCfyGCr981p5ZX0ztoBPVFQUbgZVeAM9nBLVnYlTJp+hXRyemp3CVfIXtI IAp6bwbQ3jV4E8zG98CrMK/wujycQX8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=eBVOWMkt3qqfmS1eY0rzR8noKiZ5EvmTGPqU8LqtGbE=; b=BhY3pELBYuBpORH+zQ3WgTL2AR JaRJ0lhjYYmdhki0EtCux+XcwxeANDK6l/NNLWIweICq4URG6gHCfIb6EhstJJm01jklNiXK6ICm8 LYj7GVOuI8GOhuuWel2M0FWE8LG7zB/HEdsvHIwOfAGLprWnm9e3EI/7kmLiXdCfXdyV4lt9gmXRx 6Epu60uJgHnQVQf/WB4EZg01h3RnHo4Fqyb8ZZT5xUs9H9Yqk/Aw+dPhcHKHwhDhWiycLpBD/UC7K QbCFYCh8XW/6+NU09mSESac4jMPhDLCWenfB9SRVC93wqVGPebIJKU/l84hVIloKOIm3Bd5mILMo/ SpC8YV+Q==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81l-00DTnB-Jz; Sat, 21 Jan 2023 07:11:02 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 03/10] mm: remove __vfree_deferred Date: Sat, 21 Jan 2023 08:10:44 +0100 Message-Id: <20230121071051.1143058-4-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Queue-Id: 406161C000E X-Rspamd-Server: rspam01 X-Stat-Signature: e4dkkxa6m4um148g9pu9hefes4p8tgeb X-HE-Tag: 1674285067-632259 X-HE-Meta: U2FsdGVkX19LqDSAxQx6dbGa8fuPMpZz4CmW+79i0ODZDIoqbO0uNqI85IbRY6HyyjkY7n5xTwvptfe6wzeOYPkflw922irxzCWZIMfhssb5pIBkDlQAnRqlbXR6pLKl+dqWL/Eo2TuTB9HXT7SPM92if+4cagDaLG3sws3sdGwVzT6LLqyWMX067Kr1t/ZxVoiej0SqWH1Q2KFHikTIDJOgUPwjH046WuCeEQ10aNns3VyUoAXo43Mnoweo03kDSQMSOZwBkCmR/ATP/n4YJGgipWvu3O+OV+MjAoE8x5O24wCbZwiqulsqwL5xQXSoibSvtqssfB1BK7LuoUnkSlBKfjlFJSAxGVOfFIUjXX8yxDPIB5Jc7re1PBwQaAj//Ia1w5LKfyiwRlN3GkpiokR7+D3RnjtkrtGRyvZs1tODL9oTEspG1CJl2qqZTITwhqxMrffGAMkd2LF7HqoE95wCvx2an4VpHZ+K7/KyWhoktcbZneGkNf5ymEYLGccld+cnEwIUcF83nxFz8bfM2M2zP4t+1BhobSdxV5fmvh7GiCYUhCRq4sSK7P0RMyJXgYrvV5uTW7GTWnBoLG/oq+eb2hq56VH5o/r832STpDur7LWkVnwJUnsGAjRMpD/JRwYESJZdbyE8QcZDDDnc/fEqToScZYQ2vrpLLdwJbSVSkDBRpOB0jd6jIPBbzN5hdHPsjCiqqpj4v2MVyKiM91GFT6skPcYXOYwY6q/HJW7ARN1viS8QwZkHyPanP/qonBINmjX+37z/aB96N+nIyWaKtBF4hIefgbzd6jw/+CsJZZLJv7u0JWIsUEA9525BHi0hhFuIgDzMEhp/I7VcEDn9Y9ljrckCp6jyqAXxFYZvFmYYa6NMAPF+9Q6nJVQsB51q0/XjnMfrB9P/x9npESRLHy4bl/kjraZ9fqbQm3/PhgAwLSv7jjovb2jpOMO8qYPiYoANmbHcV3e/Vrz Fcqfeteg oR37W8hEMuzf1O3g8KTjZbD4FUIQRYytIW06U6rKDHWZCSldICcQmnGniqbhdqPN5CbpBfLh6iBb1xheDkevQAlBtS1bqE5LcUmcpf9Ebg1OD5gxGfIebhyBg4FBohwGHDygCG+VnpTtgRu1H4b9VQXqhIiZIw9567Ex7cUlCzBNbPihccVU18cfwTGYp0dM0vJTi98NOkFeTI2x77PrJyDwB/MzZvW37e2QHq2RQkIWV0figBxIKv3c38hyoUJALgIbQFDqEEkUmL64zDW28U3F/jUHvPcXVShOS+J0KSTO7dVoONvhKD0Hugtnh6HLPIFeBZ2K++7X3jABDRzQ9f5idG5NEx+0BQrxoWhU/139eksW7EEmH+QQg85f4kfHNAsRpFTwuujJ9nSnrBVQBHPzmrJ/405JXY67h2fdFV3PrNbg/rcge4cQ/Xu9WkSS8kKyfGSyIXngSwExzw+kDL3SgbQgYl5TDqfv9Ajunei+VS9U+/ytUrnsaDjAv1vWD5U3DNmw95WPeZovyjMiOu9h3pj4F6v1rAXz0d0aYGfgM0bSDlCp2JLyi5fM0+8xWZrfG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fold __vfree_deferred into vfree_atomic, and call vfree_atomic early on from vfree if called from interrupt context so that the extra low-level helper can be avoided. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 43 +++++++++++++++++-------------------------- 1 file changed, 17 insertions(+), 26 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index b989828b45109a..fafb6227f4428f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2769,20 +2769,6 @@ static void __vunmap(const void *addr, int deallocate_pages) kfree(area); } -static inline void __vfree_deferred(const void *addr) -{ - /* - * Use raw_cpu_ptr() because this can be called from preemptible - * context. Preemption is absolutely fine here, because the llist_add() - * implementation is lockless, so it works even if we are adding to - * another cpu's list. schedule_work() should be fine with this too. - */ - struct vfree_deferred *p = raw_cpu_ptr(&vfree_deferred); - - if (llist_add((struct llist_node *)addr, &p->list)) - schedule_work(&p->wq); -} - /** * vfree_atomic - release memory allocated by vmalloc() * @addr: memory base address @@ -2792,13 +2778,19 @@ static inline void __vfree_deferred(const void *addr) */ void vfree_atomic(const void *addr) { - BUG_ON(in_nmi()); + struct vfree_deferred *p = raw_cpu_ptr(&vfree_deferred); + BUG_ON(in_nmi()); kmemleak_free(addr); - if (!addr) - return; - __vfree_deferred(addr); + /* + * Use raw_cpu_ptr() because this can be called from preemptible + * context. Preemption is absolutely fine here, because the llist_add() + * implementation is lockless, so it works even if we are adding to + * another cpu's list. schedule_work() should be fine with this too. + */ + if (addr && llist_add((struct llist_node *)addr, &p->list)) + schedule_work(&p->wq); } /** @@ -2820,17 +2812,16 @@ void vfree_atomic(const void *addr) */ void vfree(const void *addr) { - BUG_ON(in_nmi()); + if (unlikely(in_interrupt())) { + vfree_atomic(addr); + return; + } + BUG_ON(in_nmi()); kmemleak_free(addr); + might_sleep(); - might_sleep_if(!in_interrupt()); - - if (!addr) - return; - if (unlikely(in_interrupt())) - __vfree_deferred(addr); - else + if (addr) __vunmap(addr, 1); } EXPORT_SYMBOL(vfree); From patchwork Sat Jan 21 07:10:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10D6BC38141 for ; Sat, 21 Jan 2023 07:11:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A104B6B0078; Sat, 21 Jan 2023 02:11:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C0BE6B007B; Sat, 21 Jan 2023 02:11:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 888FA6B007D; Sat, 21 Jan 2023 02:11:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7A1926B0078 for ; Sat, 21 Jan 2023 02:11:11 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4F094A0FC8 for ; Sat, 21 Jan 2023 07:11:11 +0000 (UTC) X-FDA: 80377934742.02.E60DEB8 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf21.hostedemail.com (Postfix) with ESMTP id A3B2E1C0006 for ; Sat, 21 Jan 2023 07:11:09 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=mCKo0YMQ; spf=none (imf21.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2b7jwqEZ/VORicHVoAJulj/DOUixU6O3VS17e+UwuxA=; b=rW/zB3cae++2dBPibKANptRmP6eHsjS1qbFjAawQyBs6s9JZz/FAbWmaATIpEGNokrP1FK oXBmlY5UyHctQ0W43c6Cvzn4KfFkwFVRXI90kH/qPx/CzIGLxLrWB6eUhHV72btRzct0KT iHnaNoYSuDY/f88sjqQiE79VZQwJQbU= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=mCKo0YMQ; spf=none (imf21.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285069; a=rsa-sha256; cv=none; b=OBEA1QMSbjdcWMGXrNOCsl249va6yNc1h3DG1Cv202Yg/iJH/CUF5FOOplqT8AQwIXfhn8 1RstL8+Ru+fiKHgMjWOB/YDUEC7up0H6pJ+WUPkq9GKARfamF9Jq09U36tJjSaX4JMrdKx 96QghLDDI0FpzhUq2fevlv++4PWDdgA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=2b7jwqEZ/VORicHVoAJulj/DOUixU6O3VS17e+UwuxA=; b=mCKo0YMQ7Gz/166FUHw6DQQRng Vzb7WVMDSW8/Sho6QXe90Gmf/kwRiMvqtvDuchmvfsVA4P1tlC0OZqSILJUCC4m3aFli06L/WIsjj he0EkHwVqvpJYEm43EE2149vGXx9DCritYQQIZIuQhcl3Co6H7FfWJfxBkKgDQaOTmIbdvUJN1xcL aoaMBD+Wwo1VvnddDBtwvE5sQ9iYqPpgIy8C4bnEP2cYzFbw1NkvmOJM5EJ/SRhnzNWAX5Mg4tguG 4QtRF0epFSv0VjT5TDXp7auYlODKQO0YiCZD2YsHvzrFx+KCQh7slVrw2LMOJEZvCf1ulGocwIS0r qVXPzGWA==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81o-00DTnU-2G; Sat, 21 Jan 2023 07:11:04 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 04/10] mm: move vmalloc_init and free_work down in vmalloc.c Date: Sat, 21 Jan 2023 08:10:45 +0100 Message-Id: <20230121071051.1143058-5-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: biw9mraqnku15mcwa1rkqzwmhb6kg8nb X-Rspamd-Queue-Id: A3B2E1C0006 X-HE-Tag: 1674285069-833198 X-HE-Meta: U2FsdGVkX1+APfb/ItZuQvFaRDUQ7J32EnSmHPyS6RBV7tf8/Rj/XE/AMlHzco5o8bE9/LQLFVKpYWO9z7K70as/wszAami3NCnpV+Wl5K3Vv8g64Sl6wgHusM5M0fX11Gq+5K3rqYRT3ERn0dmeJIAYv9eRuE1GhU3CdZeBAu0ZIib2I1c60BaUa1YmuiYrxtt7j/KzDrNyWCHPgU56DgO/fdDsIoVW4aIU2co/vM/Efu2tWzDhcv37teE4OZmJiSCwzUo/WItg0Eey1l50PY6v8uu0u1+DtNc9emiH9pS16Qkb/5kHAWHsA4ltmKB5c667oY86t6m7T6H2/JAsSBDu90UKBW2pTL2ArDYCrC2Kiv400XZ6oSac8EiBtsbh87iWc7fXpczVSP7gIOshF8bt56/u3V0MWXKfxY5FmlPM+mUa8NdZ5xXmjiIYfnkKb0xy964M2cSpW7zRYYlMXRAm3ppEGW4LFDYNA3QAGzK4OZse40IgonUFD9wg3ODy6lG4m/cbCSttv44a1wg4JXNv8HdRZ4staTe1F4GsQ/x03WSohvDPcSEWO3mDu4uXKvzWxR0zZyhUl0M8ng4dY9rfr7forw60oVyucXANxoBDu9XoocgUA3nIk3Bm2jGBTkEBTu4B9X3Ve8zrbhSYbls0O28W1ouoI7S2ksq1OE/i6BOAbtYC0IqdnBm8iketA+otV0PyHCzTt/drcKJonSxhia/yBAis/4WvDVXIKPTqqKxSWji2pGhgYKezsKHeEt76wSvyOXDXFDA7vnux+5MxfArGDgLuHn7H1WvDT4GAkevHUCa6qA/59knJjolEcuahFbnckoqbVRfMjuycvg2VArhwtdYFhuH9o4NgruCaLyfw9erPvbqgVvVHhmhnf8073fi1dMKRTz9mix5+KelzAoM8VBcqwftJx+lqF+4P1adbYUfCkco/Bt5n0dyoTjghk/h7AWF1GMHKHbM Mm3kviIo pm/bHn20vB7S/ZwDb5iYACRbXlZ+hV/Aoiy8jlRtBc9b4T3C7N6lnqiais+KfJNTV2nUhrSYGmW2X4rC3V4kg7vaEJq+lrpGYGFB9dsOiXJ5p8qZZp7UnGXNHtGI7OFEfaDoLNV+T1Xutqo8aXeD4sB3yaUTgFPea1KUp7GBylRmTF2PC/M+P+pCVz0Ostw6p9mtnVZuDc3ATEjIHp43+17N0RaJdgWHKxuJoJo+z9U9fQX/JvP4Q9yUlgXo27xfGdTGy8F40DhiSyf18RPJl8p/pBDtyfdn3a46zOO8d2ObRNoRFmTk/419w79mIN2ifSF33d3ius6IUO9HK01OoBQDISDXT0Rzr9Bkzt2aiGdTf2alJc7JfKvA0DTWPGKK/TGfeWrztPfpUSw0FDiNiBhNc7E3NoOkKMNwcPL7efm3dB5fco3UsMnQu2OX/BwPpGB834At86lbKJDFE3PJ26j5zwgJcDLfMORhiFpT+V4BQUxxP2QKwTN4WY1/PNNAHccxfB6rXaOhDyd5nDQh/FEXfRhUQAk6cCSfDGUocFOZPvuNeTQ/k6tmh0obxZPIhsBq4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move these two functions around a bit to avoid forward declarations. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 105 +++++++++++++++++++++++++-------------------------- 1 file changed, 52 insertions(+), 53 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index fafb6227f4428f..daeb28b54663d5 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -89,17 +89,6 @@ struct vfree_deferred { }; static DEFINE_PER_CPU(struct vfree_deferred, vfree_deferred); -static void __vunmap(const void *, int); - -static void free_work(struct work_struct *w) -{ - struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); - struct llist_node *t, *llnode; - - llist_for_each_safe(llnode, t, llist_del_all(&p->list)) - __vunmap((void *)llnode, 1); -} - /*** Page table manipulation functions ***/ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot, @@ -2449,48 +2438,6 @@ static void vmap_init_free_space(void) } } -void __init vmalloc_init(void) -{ - struct vmap_area *va; - struct vm_struct *tmp; - int i; - - /* - * Create the cache for vmap_area objects. - */ - vmap_area_cachep = KMEM_CACHE(vmap_area, SLAB_PANIC); - - for_each_possible_cpu(i) { - struct vmap_block_queue *vbq; - struct vfree_deferred *p; - - vbq = &per_cpu(vmap_block_queue, i); - spin_lock_init(&vbq->lock); - INIT_LIST_HEAD(&vbq->free); - p = &per_cpu(vfree_deferred, i); - init_llist_head(&p->list); - INIT_WORK(&p->wq, free_work); - } - - /* Import existing vmlist entries. */ - for (tmp = vmlist; tmp; tmp = tmp->next) { - va = kmem_cache_zalloc(vmap_area_cachep, GFP_NOWAIT); - if (WARN_ON_ONCE(!va)) - continue; - - va->va_start = (unsigned long)tmp->addr; - va->va_end = va->va_start + tmp->size; - va->vm = tmp; - insert_vmap_area(va, &vmap_area_root, &vmap_area_list); - } - - /* - * Now we can initialize a free vmap space. - */ - vmap_init_free_space(); - vmap_initialized = true; -} - static inline void setup_vmalloc_vm_locked(struct vm_struct *vm, struct vmap_area *va, unsigned long flags, const void *caller) { @@ -2769,6 +2716,15 @@ static void __vunmap(const void *addr, int deallocate_pages) kfree(area); } +static void delayed_vfree_work(struct work_struct *w) +{ + struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); + struct llist_node *t, *llnode; + + llist_for_each_safe(llnode, t, llist_del_all(&p->list)) + __vunmap((void *)llnode, 1); +} + /** * vfree_atomic - release memory allocated by vmalloc() * @addr: memory base address @@ -4315,3 +4271,46 @@ static int __init proc_vmalloc_init(void) module_init(proc_vmalloc_init); #endif + +void __init vmalloc_init(void) +{ + struct vmap_area *va; + struct vm_struct *tmp; + int i; + + /* + * Create the cache for vmap_area objects. + */ + vmap_area_cachep = KMEM_CACHE(vmap_area, SLAB_PANIC); + + for_each_possible_cpu(i) { + struct vmap_block_queue *vbq; + struct vfree_deferred *p; + + vbq = &per_cpu(vmap_block_queue, i); + spin_lock_init(&vbq->lock); + INIT_LIST_HEAD(&vbq->free); + p = &per_cpu(vfree_deferred, i); + init_llist_head(&p->list); + INIT_WORK(&p->wq, delayed_vfree_work); + } + + /* Import existing vmlist entries. */ + for (tmp = vmlist; tmp; tmp = tmp->next) { + va = kmem_cache_zalloc(vmap_area_cachep, GFP_NOWAIT); + if (WARN_ON_ONCE(!va)) + continue; + + va->va_start = (unsigned long)tmp->addr; + va->va_end = va->va_start + tmp->size; + va->vm = tmp; + insert_vmap_area(va, &vmap_area_root, &vmap_area_list); + } + + /* + * Now we can initialize a free vmap space. + */ + vmap_init_free_space(); + vmap_initialized = true; +} + From patchwork Sat Jan 21 07:10:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA999C004D4 for ; Sat, 21 Jan 2023 07:11:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 702FB6B007B; Sat, 21 Jan 2023 02:11:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B2C86B007D; Sat, 21 Jan 2023 02:11:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57C546B007E; Sat, 21 Jan 2023 02:11:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4861B6B007B for ; Sat, 21 Jan 2023 02:11:15 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2A8B080DF6 for ; Sat, 21 Jan 2023 07:11:15 +0000 (UTC) X-FDA: 80377934910.13.DD3BF64 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf06.hostedemail.com (Postfix) with ESMTP id 790B318000F for ; Sat, 21 Jan 2023 07:11:13 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=uZDwXXMJ; spf=none (imf06.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285073; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Bku1701L1RVMz0DvJO8KOhu9YUJbNSRasY8+kRaBijA=; b=UXo8Tcd3DrF2V7pgBmRTqKfbaYBQ5CPcdIfTOdwcBWFN6ngya2xEIKIdbtRG7EZvxa0n1H xLxZfXf8NgFP+1oiTThtDel9EKcO+RKbSlZQdGl/ujBLgIUId15I334bMPmVABcdfWYzZi eVNTCaD1/fnBiYzB6JyD/uge4cukMvA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=uZDwXXMJ; spf=none (imf06.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285073; a=rsa-sha256; cv=none; b=33fxYs4FaGs5H/XgIp9toXtD8VtXqWKjYTFfU8553c8rflTPFyVs63W69gXUq5f+SQYa8n nxHxUPw2ApR80UF9dcD4T2JvOyZfQ+6hmgUlP4YOdzdoajqSmRcbCa12xXd55CAR5MAfOd Og4/2TXmdmzEQdStsLrT1RjE9rNEpw0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Bku1701L1RVMz0DvJO8KOhu9YUJbNSRasY8+kRaBijA=; b=uZDwXXMJEwPtdungJlU1S8TMXc AaBWPiajV1qmHuTJH5nHHxF4ndCPSScuBFsNztT5Z2CD3Gvk4KP1/+i6sHUUWMsx15ipICBnWlZpR y4SF/h//K9ROJlWkGg+S7b+qMtJcMCGJeOZRA3p0n0nJEOOnxw0V6+C4i4qPSMkZfmYJuiP0hOmpf BPmAN+sLjxU3iKKDQBUr0WgDmFQ4xpf1u8Bf1h4rF9X8ZoimwYYvjCgmGikoloU+x0xtbXdGa8rhI OGllWbcfez46BwRuRo7Vntu9AysIeebS5vUzbC4sIKDvGqtv8sH2bPykPlKQ2tN8MeC1M0JU6HFji 658ILgKw==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81q-00DToV-Lo; Sat, 21 Jan 2023 07:11:07 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 05/10] mm: call vfree instead of __vunmap from delayed_vfree_work Date: Sat, 21 Jan 2023 08:10:46 +0100 Message-Id: <20230121071051.1143058-6-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 790B318000F X-Stat-Signature: rya81j69r9uhon6f4kjb74q477u9ugji X-HE-Tag: 1674285073-117035 X-HE-Meta: U2FsdGVkX1/rxyonVsRYjRkgYHYogQq78msztymMyGx+uzLkc3bW07ABFR+/r/1uW7tHCOqVwgWvdKOflcjm/SYNjPg50M7NfAtd3J1nzVayEkfdjAhNejYxyhBFohKKVnZXPlRUGrIKGVlDnD1zOCeMR6okemHOlenm04LypBSTj8CC9xgJ/O3tiN2AQ/xT+OeNeyfp1L5pxqU4w46tGdlWm8S+RhEUODwWJ+DZbyuxVeaBXRYv59+eVYAXW23ZyVhKzkb6fU4InONwyxyky8Lxkj8VY/gppjGj+AihpaPYJDasoic5Kc7l99dxscVnlTfAonp2miX5lFNLPyteqy+RvCJWmbJl4dE0yjqEsGAny9KYviv4XvGW6Q89lIHSRpW2WJmj7m8e89qptZ9rtFD+YJUkwN+Zybe6F/+jxS4TLrdnmDc7tGmtditJPH6wVCbCIx/LwGXHcDholpYmAfehwqQjP9g3U/99AFeQ6sGJxOXWdsCHTXUmCPB0GWwpvVo2yzeDwd3on+ZZK4GlcsN9YjCbq7v17g6AIj5Bs9qcdMGtWOvCULvxpzVVq2UYazuiaLH1OO9G4dk7rnio4Tr+MPgn5Di3TmKs1GWbt/iMuoEsMf07eCxhJY9GjBX97wiNSm91sAhfhOe2gR+j8hr/aSMb15T+xk0H2Ny4f8FxR0S6q9BAoAblUOkgU3VnL19qGvy5Uu9Iz3ToYFnnIEHO+WieCf5BiA8Q2ukMkHPmEi0EdFZ7EG2jj6qiYIzLZh6cx2RnkyltAXUM0pOxpensgieLW7Dg3Mv0l5L5tAv9DDrS2EoN4ayrP9B0ZQb5t91P342nJ/sgnTYwXuN3HBzMOfMTKzQKyq+i8H2fLV5o6Fcsrrw2XxLRWvVc3MpVg0nzi4tqyNP/4mVKog2gS6G3i+mhb++vOQTp+nEWlyHVlYiM/9r6v5cKq68Rb+ht3MXDiZ1fhxjmg+9Xd0K hY9A7Pw+ E0WgAcuTjjoQxWZhx6tKpbUZW+KYPJQShXRk2d40sVczW0TvWzECsO8vKMOpYl43RYqj5igm+Jcf2XGnQNvdlnWEiXAp50juWEnhreK0R3MB1jXBOEyi41qVuUiXpUJPf8LQYI1xK/yiwWpRMSVC/RUXz7mg8as1MlYK44Ob++B0st8RFmhkVBKlfZDHcjRYaIIhPxBgDa9xPBYaX097xlmXX8sYj4tEJ4lF4PXUtu+oRDzqpESusMu35ymvfasJmRqg+neMtCegrN2NzFaPJ1MKM0UyemjTWTcekAWje4XqeOSb1cWwVvoHn5lA2NNHqRGv+MOv+A+LxBAtTH6NJT8dLP3BEKIK9iPDHZxTwyMFOvgjmMFzPEihFVZ4MXHlQtpOYRMYoFqMr4+Q2YaPqzBHu9sVxolzGPIgG3ogE9InxFexHhC9e4c8Txfof6tMP+S9zzs8/gHWWM53AOh9FCkW5ME95uwP+kIdTZgTf3GLeZCmB4IBCOEsvaF4Rr2VPrMZlaV75SvF1yF4zXbXIbAXbcnuAJKGbsr2MP9hOv71vk5BopwbDXUeSuJi7EaxQz1ZF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds an extra, never taken, in_interrupt() branch, but will allow to cut down the maze of vfree helpers. Reviewed-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand Signed-off-by: Christoph Hellwig --- mm/vmalloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index daeb28b54663d5..3c07520b8b821b 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2722,7 +2722,7 @@ static void delayed_vfree_work(struct work_struct *w) struct llist_node *t, *llnode; llist_for_each_safe(llnode, t, llist_del_all(&p->list)) - __vunmap((void *)llnode, 1); + vfree(llnode); } /** From patchwork Sat Jan 21 07:10:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 744C3C25B50 for ; Sat, 21 Jan 2023 07:11:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0EB356B007D; Sat, 21 Jan 2023 02:11:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 099246B007E; Sat, 21 Jan 2023 02:11:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E2F066B0080; Sat, 21 Jan 2023 02:11:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D446C6B007D for ; Sat, 21 Jan 2023 02:11:16 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B0B791A05EA for ; Sat, 21 Jan 2023 07:11:16 +0000 (UTC) X-FDA: 80377934952.02.9E293EE Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf11.hostedemail.com (Postfix) with ESMTP id 1E0E340005 for ; Sat, 21 Jan 2023 07:11:14 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=VeEOKHCn; dmarc=none; spf=none (imf11.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285075; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CBsCp/QzNVZu+9wjEps4s2+qTeM/ys5OZl1OD4aXEDs=; b=pil91iPwS+DxvUJKrM9KsDHdrY2TxfN48sV3fNznwfZod7RxmsbHeSkBsZR0U77YZSEyMr tTupVyYsti5F3r/V8qvqZ3fQ5iI9p8e14FJV12fmnye3Cx1q1VGi7KF3Tp+4gmBppLi/+r xYd/7F6r0qaOUtafkbLfhat1mxD2MT0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=VeEOKHCn; dmarc=none; spf=none (imf11.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285075; a=rsa-sha256; cv=none; b=W++2NFnr+3UI4Kl98gwViC0V9vu3WD93FSMa1144uQ4iq2wIz+dX1Eze/gIyj42YmkrPxj X3/aGI68Jrg4e6bPyvfGHFlJJrxKSE7yXeWjw9P8Kmqtav5B7iYRV/tTnSo2uvEpB+y4ST dM3v5hMGzYONrT+7pefK4M8o5oTMlIo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=CBsCp/QzNVZu+9wjEps4s2+qTeM/ys5OZl1OD4aXEDs=; b=VeEOKHCnO6cmqihYY0yoJCG4Bi hHznZSK3OqyifQO3u6IYqonjoYHP1eVgrMXN969MoevSuAj7lAtKAMsZuSfwIr23BGuA9dLHbiPp5 PGiPU5iEA6n9FmTDpCeU4sjX9r9vuTFa0fgpUhX8CAOE/cb3fFdvxdqXtPv6zCkUXAlKSwuR3pzL+ XzUqiHnjd70YVfKCftZIMqFu3L1JUPtpRPaYccbuDg/m87FUFjAwXnio/YtLhusPlx9/n/rwxseHK jg0zY11p/HR85+xDO8X40WESdC4zyXl7lVc2jkytI0sH3/xgs84GrvSMZ/DrYhX7YHfnhIW/IT/qF fwPkA2bw==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81t-00DTpv-53; Sat, 21 Jan 2023 07:11:09 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 06/10] mm: move __remove_vm_area out of va_remove_mappings Date: Sat, 21 Jan 2023 08:10:47 +0100 Message-Id: <20230121071051.1143058-7-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 1E0E340005 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: ajkpandewb6ax9fnw8qa4kra3jukt1xw X-HE-Tag: 1674285074-440241 X-HE-Meta: U2FsdGVkX18WIhwG40AWnxBtG2QazyScCDsi/PSG+X7NboI728pReI0UcHXPqIx2GYkh4LC2pmwzAp5+vO+I9n++daeCcyQXRCGjBirGWDYgNt6aLkmJXVIbYwn94tEewOE6kMvNJWKY8wS9N5AwXY3hpEPJZkmnZGofQw/iXxECztlgpsTto3TmEHNolRRHR04+XLX+UFdNueoYkOgBWpS4UZKCb4NZMB9I+t67vpwFW8f8Mjs1yv5LlTe9CPRGQgXzbqLAO7JiTYlqqwXn49YITnv6n0azbFLjFxcghMUzAhCctJFLLlhwT+8fnr1L0MLgidwFJjSi3py76JRewHApBkZBhV/H2cPRoQ7NctOFCD7hYD+PIOoHoSNa4vH+Am6a6hRuPWXTZNgOaFi0ajdt+r0Nyehd1XjLxKV8b6BAPqJO1t4ds2Yp0CdW+Syx7qF9lIGxYLm4M70Rc5k8iSwvSrDvzLFOAE1RQ1jBkQbWFuGidp9XMkPB6qFvTrj0A776bz/hxM2s/kvVoaH7qXBvDML5yA7bMzkvywDPgjMeXAeKQmazgC8t41Sk1IwRhA/r7ZG662XNF17RRo0BqLdgfnGlx/8kPdKo2QcpxTlusJJFcQFgX5KxFktNg8JW+yNv4x/2gDPrf3vWiQXfRUoFrmg1GOKZz1UsutNhmTK1cQBxO1uDe+gO+Kxakm8NNhswe4Z2VJSLgVxJ1fMFUYm0VOloiuosveeIh9rSc5FiwgXbP+w54X3d+HNwEKlxjwuT9HQTXfm3wnGFPZL2TD8e0/wqMW8/zSsHNHrnkWz9nrjWm0IlccK7FL0FqE8x6mjsocJhznp+KTYPvpMY1pZhTN54iQjwA5IpoI0gQKhwRetpQ47jkvm4Rg/dKAqxc4rTpvNA3JyGZqZ1BvmgOTE6O8Q/kVDiku0Yc2S5+UgEdKDMVs6NpeAO1xrwHtmuZ02+nkeQGfJRNSsaJ8A sxHz7dZE 0R1jIlVOACoFrqGOhiE033CpgytwQ8pE2KBKxxTk9JSBjM2kFZjMnhicJwUO0JZfi4GS8Ix7fmX4zij3HBD4mdv6xs81jc4LNdiXge3liJdaiJWScuM4psShU6khb0ieQykseyn7BvGrikHgTsx9wv14lFx9+f8UPhaV3B2Jl8S1gCBgYr7oqEnY55NCC2gvNXpqqTBN4LIui64oA+p5MUpr21MKhV4P43wA1U/+IQQVeryzmLESY6ezA/zQPvg75w6+UkmoTgiWE1mL1aF45A0Co/UsI36aenNb3TU+c9jCtNpdKSu6rHVr4jppmlM2MRgUDFJV1IF+wLFKuzLioCWwhhKgkw6EI7MdBFuWnJOF4I7Oh3Y72Pt226EL1T3tqARzuoOOZLQPzGbrmXkwAKWat3/conBevTxlyM0CFzOAsTGt2Ea9wO69uzedgLagy7GnfT44n/DJ64zK/lHz4k2WxJPTfpP5A6T/sxuIb2wij4cDc0cvtq5h1hJoFtefa1nGkglPuUgYpL1NnN9zfCtZm5NXxB+hhYYqnO7T1vve6GMmoWAxWq3Y4oC8HOizu9oS9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __remove_vm_area is the only part of va_remove_mappings that requires a vmap_area. Move the call out to the caller and only pass the vm_struct to va_remove_mappings. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 3c07520b8b821b..ee0d641019c30b 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2614,18 +2614,15 @@ static inline void set_area_direct_map(const struct vm_struct *area, set_direct_map(area->pages[i]); } -/* Handle removing and resetting vm mappings related to the VA's vm_struct. */ -static void va_remove_mappings(struct vmap_area *va, int deallocate_pages) +/* Handle removing and resetting vm mappings related to the vm_struct. */ +static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) { - struct vm_struct *area = va->vm; unsigned long start = ULONG_MAX, end = 0; unsigned int page_order = vm_area_page_order(area); int flush_reset = area->flags & VM_FLUSH_RESET_PERMS; int flush_dmap = 0; int i; - __remove_vm_area(va); - /* If this is not VM_FLUSH_RESET_PERMS memory, no need for the below. */ if (!flush_reset) return; @@ -2691,7 +2688,8 @@ static void __vunmap(const void *addr, int deallocate_pages) kasan_poison_vmalloc(area->addr, get_vm_area_size(area)); - va_remove_mappings(va, deallocate_pages); + __remove_vm_area(va); + vm_remove_mappings(area, deallocate_pages); if (deallocate_pages) { int i; From patchwork Sat Jan 21 07:10:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98BB0C38141 for ; Sat, 21 Jan 2023 07:11:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35ADC6B007E; Sat, 21 Jan 2023 02:11:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2956D8E0001; Sat, 21 Jan 2023 02:11:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15C3D6B0081; Sat, 21 Jan 2023 02:11:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 088016B007E for ; Sat, 21 Jan 2023 02:11:19 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C3C51160F84 for ; Sat, 21 Jan 2023 07:11:18 +0000 (UTC) X-FDA: 80377935036.25.1B2A7B5 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf24.hostedemail.com (Postfix) with ESMTP id 2F538180012 for ; Sat, 21 Jan 2023 07:11:16 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=XM+vtBXe; spf=none (imf24.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285077; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/LORA17lyKlGYgS2Flqkv2l85bKFZsz0OAvq3/+VRvg=; b=PjRH+9Jxm3iQQVyIc89uKS8TrngdPF3myDFTCQYS5BgBujItmbgEZdVyHMjz/i8hEsv9uF MLIf7gk+geb5flrKJ1dbBXza9welBbthg1ZuhiVbNagTRKj92j8z4qGCSJOdMOXWUe1ZDA xttcudV0SfO7YDHQ//ulkC+Fxve+M5s= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=XM+vtBXe; spf=none (imf24.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285077; a=rsa-sha256; cv=none; b=GhKrM5Cho/MSHqUU0QGam39IOoT191n6oGPVu8X7Cq1ydNAfBNq0Au3e7QjxXNk67xRpmE j5k1MzrUxdxWomuu6kAftF0ZFIjaxzY46cLBBBzUrqODd74axA2qjtdhP9En7mGlm8eafR rLryccJHeVyFpawupHeGNSMj5ypEn3k= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=/LORA17lyKlGYgS2Flqkv2l85bKFZsz0OAvq3/+VRvg=; b=XM+vtBXeQfHIdNOqhrRFIiQpVS as9nNsLzVhrL08NSOV1H9gmav0bvUe3POeb/f3XAmlet6O+Ix6L6o9MbYT+A0mDQFVc9CNw85j+mP NERzLlXmIFRpcIF6TyLrvj+CSmnsLkAMCCr9tZH85mXyIYuas+7dHo/5XtUDa01aAH3S9+j5JvbC1 kEmrBLUnVJkZl2i464hgzmRvAWEDjM/zFO+aJTEX1pSxpnrAPxtoV7C6pYzuMWsMZkF9uGZhEFupU JBOfMQNwevrkxKwrCCOX8UdmwyiezK2eYtxHpuW8dnEzOjlQUb2uR+3ZpkZOtuMtVNPNuO2RmEIrx wnRPmJLw==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81v-00DTqj-OJ; Sat, 21 Jan 2023 07:11:12 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 07/10] mm: use remove_vm_area in __vunmap Date: Sat, 21 Jan 2023 08:10:48 +0100 Message-Id: <20230121071051.1143058-8-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2F538180012 X-Stat-Signature: ht4o6c54kfbw3xco1ox1r4oddsk8kqsg X-HE-Tag: 1674285076-134156 X-HE-Meta: U2FsdGVkX18NwP4dkbwW0KgVQ4dQqUAufK2MIttk2pFoMpRihUhLmHLPHWpG/+optZRQsynvrQmokPBsPyKz5tAlrC1IRS3QxI6ktnaoQmVhptxwORUi4L/5IQrb0OmhmYQgsUlERGb7HzRlARD4eNiXC8ncDapVfGWhYQ3uLBSDPbPXVT1UUY+YMLVDAat5FI9LbuQ9h1zJMSCHLB+K7vv6/RPMMeoz1tATxkhJdtSKHrFXrClRYbQVeAZJ478Q4CiLq9r4hIBic7j43TVqVV6SXkII9Vuznh4g1zjVdnGCMR/AkzsPm6AEH+buYzeNBHle5us1j0V9S3FFVfGJ9NRJjKq8x21HltuX8Gi77yrfFhpNBzD6F2qkW/fbMpQgrCYyzcIHiMfUmBmZ5T4J7ftXEr4xvmYvigd1CHk0Z26GcVOhb7vE7m0jrkIJZA/HvdmfDTTd4VwslYV+fs111z13j/KM4t1sNXeD18GBMEKiUc0RdvHnQFm/l7cNBPrM8TWl0OUwowHf9OAZOCcPYrMHy3wZlRvdt52yQnvibeUDiWRWbvr98n1hXGjrTaikGxEj3gVUO2dq99neKaDmA9RaHY5tJiF+Gq5+O+gfCZU60y+LSDTQSq7oQEBsgZ0TlRZKBvTEjzoiASCHQNSNfQsAKxqQSVukXFYswF1b6DlZAPtPZfI3i+yOqqk+Re+RGMPBxyIpwbK13PwU2BC565oUxh3BcIvLemVz2suEHr0jZwcxiSGNtIowYGnlEGYzCoj1ebFavKpwdoMki4F0mfvlPV5TOmXGg0EncMUWIgFKdKqQlSyuEdj0LwZ6e63T2Mw2x/D9IfTXhqWLPdwnCqrU/opAFBi8cP2SYXNBs11hkiUoUgs0rwT7HfyHkXbG16+vdvHOxRf+iVltkDcByipXytni6Lub2Lh2ikHb12bZOKcfGxitVqRN7xD3VhEbKn2cpp0APsDMGDpEjez IAnHG4oT PQLjl35OqQGKkd02FI8PQMKcmWhnDOkInmZN7jk+nzr+PRqNuSVzkhL/YyjyGCf3JYmT72Jssz9x3L6Xa/HEIPi+1/OhaZxltukb2bBKBWaJfikd24G816ZuH0SR4iKiaUiZ6mZmiHoUBpjvVzVdeCnm7rJgHoYvPenxeH/zG7YuVV3N3UHi86JBVjFdDSHQPA9t/B7aw0d7UyX5Mu0de7m+JIA0oeXAAphbT58ftasBEhkR1JegqjXSiTDpGw0+MMUO1j7vyRh9Q2+PuX8hhskYzlc+95Sr7xvo0qaK5t7auYvrpq9N0yDmkAG3cjJ8R1MvgqKGUGdpLm1XVt+FkUmNeziRjvdrGLvbeTKc15+R3mA+jBS06fpMCzbwiTaiGWgHRgYoAi/ylJ5IVlRRD3ZMoePAjaURmV5Futqd9Vda+ABV+vjER6bRN5lWkewf2h9PKaef1/lbzpeXbUGSkhEOvMQtesW7tio1x5pYnK9m0RQdbU1CCkqxdqvtZSBcS8aIzQzHg4XmI8AvtBabtgM9Ayl0/KCD3zKX2UBLzZgbAFQlW5KDOlBv2IuC6ntyEZTQ/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the common helper to find and remove a vmap_area instead of open coding it. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 33 ++++++++++++--------------------- 1 file changed, 12 insertions(+), 21 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index ee0d641019c30b..97156eab6fe581 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2571,20 +2571,6 @@ struct vm_struct *find_vm_area(const void *addr) return va->vm; } -static struct vm_struct *__remove_vm_area(struct vmap_area *va) -{ - struct vm_struct *vm; - - if (!va || !va->vm) - return NULL; - - vm = va->vm; - kasan_free_module_shadow(vm); - free_unmap_vmap_area(va); - - return vm; -} - /** * remove_vm_area - find and remove a continuous kernel virtual area * @addr: base address @@ -2597,10 +2583,18 @@ static struct vm_struct *__remove_vm_area(struct vmap_area *va) */ struct vm_struct *remove_vm_area(const void *addr) { + struct vmap_area *va; + struct vm_struct *vm; + might_sleep(); - return __remove_vm_area( - find_unlink_vmap_area((unsigned long) addr)); + va = find_unlink_vmap_area((unsigned long)addr); + if (!va || !va->vm) + return NULL; + vm = va->vm; + kasan_free_module_shadow(vm); + free_unmap_vmap_area(va); + return vm; } static inline void set_area_direct_map(const struct vm_struct *area, @@ -2666,7 +2660,6 @@ static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) static void __vunmap(const void *addr, int deallocate_pages) { struct vm_struct *area; - struct vmap_area *va; if (!addr) return; @@ -2675,20 +2668,18 @@ static void __vunmap(const void *addr, int deallocate_pages) addr)) return; - va = find_unlink_vmap_area((unsigned long)addr); - if (unlikely(!va)) { + area = remove_vm_area(addr); + if (unlikely(!area)) { WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", addr); return; } - area = va->vm; debug_check_no_locks_freed(area->addr, get_vm_area_size(area)); debug_check_no_obj_freed(area->addr, get_vm_area_size(area)); kasan_poison_vmalloc(area->addr, get_vm_area_size(area)); - __remove_vm_area(va); vm_remove_mappings(area, deallocate_pages); if (deallocate_pages) { From patchwork Sat Jan 21 07:10:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AA3BC25B50 for ; Sat, 21 Jan 2023 07:11:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C18A8E0002; Sat, 21 Jan 2023 02:11:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 971D98E0001; Sat, 21 Jan 2023 02:11:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 812D08E0002; Sat, 21 Jan 2023 02:11:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 722F78E0001 for ; Sat, 21 Jan 2023 02:11:22 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4DABF1C418F for ; Sat, 21 Jan 2023 07:11:22 +0000 (UTC) X-FDA: 80377935204.02.44E5735 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf15.hostedemail.com (Postfix) with ESMTP id A6BF1A000B for ; Sat, 21 Jan 2023 07:11:20 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=1nYfA6Iq; spf=none (imf15.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285080; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2AhZtCg8WcctCTduxx8cDUSUIZL38/xwLWyYwxGa4ko=; b=pYd2Jy8QC426WtRJ589KboRY9gejTIkfg24tOKRJn1SgIBlH3vFVh8LZ0Kqfce21FnTmPy iMdD7hdp9rBeiufztrHSdh+YsLWM8Ks+by4TEuqWAsQ000AqfRqz9Tck2jsV20mOLYUxQT OHqXn8PCm3OoMIMmkuamXmpIFXx4L4Q= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=1nYfA6Iq; spf=none (imf15.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285080; a=rsa-sha256; cv=none; b=Mkq/Z6RPrHbJ9LKgnzY1udQTjjfMv04b6zVkVn4/JUNS4Hk3tdmJa6dO63RpoCiKCaHxdu 55eAJgDzPq9Ggm8utakBTdq/dktWvm/AJCtHRGfGl4vwEDlmxZvkQMl7wULsmsmuEt1Bfa geSArKnp900wdgXuLSTqKP2EAfcl2FQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=2AhZtCg8WcctCTduxx8cDUSUIZL38/xwLWyYwxGa4ko=; b=1nYfA6IqZ22RIVP4VNjWYursgd Wzd52qurYBoiY5lhthB1Ey9UZpv4RdZ84MHk0oqSJLDXtuod+IJpqxtmg7d6E445XUHVK9DVJlQa8 wW5Pnl4Dvc27/w7DWkm2DV5OtHrWUvqjSB1FfMIXuxdMaEQ3uaTQTgztjizbgxOMf/tLq1nMFCsOn lE6ZJgsTna49sB9nbnKnkuCTjkifirZSs12RAmr+eKkITv/HI7JOWUPaSChQY0ibJ4N9BkHkeKL+Z Jpx8qbvE2Ft05cIOPi440FOdkone7RPDFJh+Qu/2mNbwQmzQXvQ5fPsyXEaOulfSOHONCIeSwseUI /fpcQgAA==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81y-00DTre-Kp; Sat, 21 Jan 2023 07:11:15 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 08/10] mm: move debug checks from __vunmap to remove_vm_area Date: Sat, 21 Jan 2023 08:10:49 +0100 Message-Id: <20230121071051.1143058-9-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A6BF1A000B X-Stat-Signature: 18ecn8otit9gcdemzopruz9q1gi1pctq X-HE-Tag: 1674285080-704307 X-HE-Meta: U2FsdGVkX18X/YwKOj+35+fRHG8lYpOcJQerAIrak6RPlVnVb1TPAYXoA2T2GiLRwKA7f+praO+VX3XollBv/zuxImFU7i4PejCARfVLS/GSrDFezW9JG9oY6U89mKUVwDp1Yr3WUeF8KC3JmKcWbMELykUwQwKU3HdcCMMzOIjS+k0a29S4ZRliXuHpRHmMk7loa555JuPYmbO8KTFNApZSDC3xocstwP99v+S4qhgk+NCYeeOPgfW0UGeO7JuAihEcWOED+HQYSFuagQx4ORM/Ex6YKeQUSAShOSRr7W/m9rjOekSVXi1Erz0K3hHS3VwvsR8OHDOOqPECkp11sswoEGeMrbdMNdpRVuvUeXR/ZCskyrt6w7k6/hdJ69P6BCKE+3aAk/FihQOpP57fywx58biU1hazbQjy0Kw2pekAhsXGbW/S8FyuUXA6XiH7OhyS8DUoFqfAd+7IVT+diTyWzZ4NMGlSUDBUbvAnnvUIXOkpO6O40SrF3t1OdW2GwYPM5Uv5qr026vmZdc1HDPjYOu9s4vIODeGmfke60/v/BPYRRet9XtMkzAKuk4chbDun0SU9VOdFAezxCxp+XD2bGyX20UZdhubNc5JRhym3/NPcJ6pwqD5hV8vRds1PXPQ2GYyja/iikHRsXXeMzVoSPfAGsYcxl3pp4Zq4HJ7gDQojwj0w2YjKX0PZMHzJ8m7UzvgRRnM3XDUIBX7RPg56yKB6DRkb2RJiAQVa3zQ/IuoMBvMKBE+9yDf/Qr4TPMY57gm25O+KCk9wU/GMNifgwd08M2lS/37cbeY5h4inyprk7tLtkkmD3gflIp7Kmcrgd8SB116TddBcTWBk/Q3e6Sd4UJNWvwk/5gPBDEUWZHUPpUiBix0s9UnWU6B4lQvPCCeMxFsKBJS+AXkegXzz5xv2uzz4XCrO/ldVQo1xBSH1Dxag8Vi/e25pKwPWKdg+HARzTdX+jXN1TNC Z8kXaOoP FZEahC53fkB9GcrPsffUTTIKkb/L2QySe2wa+x8fFt/7dIBRWXFMIwb16QMKme/1zXN6RUo9zpGLjxhGZR0jFVw1+VV97/tRzBBnwM8q58hZ5MP8o+mWJBab8ssRQKWPxptNppOeHgOPEUCIVvQV4xY8q7UOxtDsUTLaqE3YtjJFMdkhuWUMb5BGoaLMROcdeJlyqa/6KmxU0WpA+WyT+l/WXSTkf7Dk8q4u+TSjHQDEmMMOvpX2DVhtN/6LOJu9JXJEsckY4fNg6YDtlhuI6h356fgnN0nVJvB16lSR5C+ERWog8sqEFWkUKVjoR3cvENMn/c4a3Lx3QEIqlKlW4OPRGyQClvyMBup3w5aFX5EOBKhFpxrLqLoZC5FkQkKEANNB6A2suyYFGvJt5hkKU/I5tUN+YOIs6rzrwl0m1O3h9jovI0KlsEM9QerHr5YoLNo1fjP9mPX+xgcZ2Dt9JX4SIXlAFP3V18MIMPEtl/f6L9geMzQUZKIQYANn+owdBhIkgmSpNQZnfbwQmEtCG7R2dFPjLUpuIMspQEM0Gp2Wsx1QxkWwWyxzcbaIq6EPraf4V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All these checks apply to the free_vm_area interface as well, so move them to the common routine. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 97156eab6fe581..5b432508319a4f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2588,11 +2588,20 @@ struct vm_struct *remove_vm_area(const void *addr) might_sleep(); + if (WARN(!PAGE_ALIGNED(addr), "Trying to vfree() bad address (%p)\n", + addr)) + return NULL; + va = find_unlink_vmap_area((unsigned long)addr); if (!va || !va->vm) return NULL; vm = va->vm; + + debug_check_no_locks_freed(vm->addr, get_vm_area_size(vm)); + debug_check_no_obj_freed(vm->addr, get_vm_area_size(vm)); kasan_free_module_shadow(vm); + kasan_poison_vmalloc(vm->addr, get_vm_area_size(vm)); + free_unmap_vmap_area(va); return vm; } @@ -2664,10 +2673,6 @@ static void __vunmap(const void *addr, int deallocate_pages) if (!addr) return; - if (WARN(!PAGE_ALIGNED(addr), "Trying to vfree() bad address (%p)\n", - addr)) - return; - area = remove_vm_area(addr); if (unlikely(!area)) { WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", @@ -2675,11 +2680,6 @@ static void __vunmap(const void *addr, int deallocate_pages) return; } - debug_check_no_locks_freed(area->addr, get_vm_area_size(area)); - debug_check_no_obj_freed(area->addr, get_vm_area_size(area)); - - kasan_poison_vmalloc(area->addr, get_vm_area_size(area)); - vm_remove_mappings(area, deallocate_pages); if (deallocate_pages) { From patchwork Sat Jan 21 07:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C27CBC004D4 for ; Sat, 21 Jan 2023 07:11:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5CA2E8E0003; Sat, 21 Jan 2023 02:11:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 57B848E0001; Sat, 21 Jan 2023 02:11:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 444198E0003; Sat, 21 Jan 2023 02:11:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2E9848E0001 for ; Sat, 21 Jan 2023 02:11:24 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1401540F88 for ; Sat, 21 Jan 2023 07:11:24 +0000 (UTC) X-FDA: 80377935288.15.75EFDE8 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf18.hostedemail.com (Postfix) with ESMTP id 6C03F1C000B for ; Sat, 21 Jan 2023 07:11:22 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=vk156WbC; spf=none (imf18.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285082; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2FZpTb+3bZJMUpb6YA7LoDb6w4aUjCraFIBMJjAahss=; b=0NZxR9ndk07U8l5qPDL1RpFFyMA7j3/jFHKMCHpnh5nST6BUMMFt1jamLJDK2WVTGFV3UT K0po480plG6dhRkBoYY9+XQyCl+Qvgb+76TSmLVHwrshpBjhmrOoWc5CZNHyTydrHs6mST 1OM2K5ZKV0k45VGPgksoSf1/UnbRYdg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=vk156WbC; spf=none (imf18.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285082; a=rsa-sha256; cv=none; b=qWLyCPO8uMYWqPUi6Pa/QBDuznHxZpEruW14KDI2nZjwFz5Jwa9IcXMq+cV1o+rx1/YHKl W+F+e3g2fY7LRZ5FWlfTeqVM+UbNjeSrOhPpKeP1iWu7nM09o6Ci2hno6YjJMAXEhc3bB2 qrk6FOumKq7McFGC/ohjZbNTpIceFXM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=2FZpTb+3bZJMUpb6YA7LoDb6w4aUjCraFIBMJjAahss=; b=vk156WbCNmHsZu4P/efUcpqIgq e4L1tNlJXOHKIraBOpmRmbAlKkiW6f42LHRLqxAKsLV05UPGq05Z9DvX2G4VT4KMTJ3UpsfaV2wcS nCNKXGyderjQ87xrHOZ2rPFXm7hIvnjk7GfNF9XGfSTVUcDCkdAHdr4l+5RhNCqfjnBeu9HQfW3fj CSufSOGC/oC6ozUgSD90d8zTHXpqIm6G/qvNELnzGY44tP7P9DQ8iJh3K7BwjChgKrql/CK/XdHGw 3hE1vZOdDEszNp76ffJGRv+kcYGwlMJONWpH76zXC187SEkyP7l5Iyu6pcQQVUjHjCDi6CTYyJPYn ExbvTVhw==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ821-00DTsL-5M; Sat, 21 Jan 2023 07:11:17 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 09/10] mm: split __vunmap Date: Sat, 21 Jan 2023 08:10:50 +0100 Message-Id: <20230121071051.1143058-10-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6C03F1C000B X-Rspam-User: X-Stat-Signature: 75n17tdeeze9tccagq5dpc4396rqak8a X-HE-Tag: 1674285082-184560 X-HE-Meta: U2FsdGVkX1+Ud8UfHqY9WHqrPmPAcuWr5E0dEE6/dLKgwNgWKSzCBBi0db1jXQlcVLzcpQUwyAUvAtBZCOjD9ZFSFIw8SrX491MyFfF5df6WCXxFLku6WpDcKl0iHxaQ9TOQqwyzkFrYJhWSGiglR5jJxiPPJoKlcz8Obg2XiN9OyimrjlzGxgRUKEUXfCnGnAwZK15yMJfYBjX5DQ0ZdtQ+QPnNczEtVNGNnxOmseTkmhYbf++PoaelY0l+pjdZzVuXnTdpufZqlOkTFAZ/5eaarPb12P+I+N3TibqWQQAPb5WatI9vPCzzFGOGTqAy1kZFeydgxmas165YOSWtW5IxX6QpQoD9JTvAJuU6ivS1Yyax3zjaAdj/EgmBK24muc8KvMO3TPmRr2Vr8UMN5/dlXQIRaBvrjWGkagL/pDDOK7d93T/YZYN784KERJc+LdBK/Fm9w+d6/qodTEXqg25sx6eeaqMtsUwR5VNKRDW5rxgl2Cg3nI10CKbHyBCokyDr5qE1NxdEPVXXijoOdbddBMIBTbcGscSg9Mqojuj+WBHMal8M+oEhQ/mGwfY0DzghpBI9N3Zp5bRi2L/H6VE9vm9JYN64Lu8YRfwGNUyDIa+E8yCjUM+Dzxk4fBg8GHPx4I6CBrPugOwwQFLFZ0Y7DrhCfzhdZQu9D/E0AXJGwpkJLYnxxxjJfs5oHT5n4Rg46Z+Fs+guiiwXTxweLGa15k6c/Dh/LFuJOy9eRBOJLwhqOZRfwspUXP+opKjRHcXy5GjEQfAC80XyghpCmVwN6Ola4OC35W0c8M8XLdunZuX/d9R75H9a4xAMFewRYkxI2+NsOFRKA8q21DWinc/qzmnNFdehiAu8/14RoMrKbyOF2FYg/ExYfCCwm2HJKZWdkoMkT8cV6AmBL/uh56ZoJbEw6scBQWlHMeTIs3VYZDOKpWEZt7O7V/JJjkghIPTA/TjKi/+r5VpVBMI V1kZmK2/ Gvfg341oyhBxebx8G7O8iTP2tBQ8dYO+lLgr0CoRXYGD6qwz+6onu6JnbastYi4TCJlMaEvVRc5GpCYsE5AHKvpm3FE1NRN/h6VdfneKXfzlFe6TMWxbiRgYtkXyKAIasq6NpaTG4d1kjSaU8n3txZs3iZlkDBfwuSHIMex7J1ts7zNxdmySjQOLiQj0U/yEtccUlQB0zQrK5IL2aybbx2OXUwKBvzSoBqEEb4I2mAesMOOtEkFXICHkRrkMCK9tSAuofGrHNo7W0HAyhpt2Tn40xVVMCFopYx4z2lBIS5wkvcAfOLEPienmrZSohaTvoamdiOC7Jq7ILF4TIdj59R3OBSClaXGKv3xQFiNg9/mnDc5d56QnVrjFIM3S1oFXVj45VfdQQnMupxhnMrqeGy88Qkdv4yiGJpdzi5q+f9w6PLOkmsz98/hOwJSCe6HZGlTKoy0ygqy9twSC9dp2SBRrnDwnh3VvfDXEur8X7rYUXPoi+MBFHegjOyVwC67Ou5ZgftAR4YlUmisNLdVerUbwureCxuhHEPT6sSFHB9lAa7kA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: vunmap only needs to find and free the vmap_area and vm_strut, so open code that there and merge the rest of the code into vfree. Signed-off-by: Christoph Hellwig Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 84 +++++++++++++++++++++++++--------------------------- 1 file changed, 41 insertions(+), 43 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 5b432508319a4f..6bd811e4b7561d 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2666,45 +2666,6 @@ static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) set_area_direct_map(area, set_direct_map_default_noflush); } -static void __vunmap(const void *addr, int deallocate_pages) -{ - struct vm_struct *area; - - if (!addr) - return; - - area = remove_vm_area(addr); - if (unlikely(!area)) { - WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", - addr); - return; - } - - vm_remove_mappings(area, deallocate_pages); - - if (deallocate_pages) { - int i; - - for (i = 0; i < area->nr_pages; i++) { - struct page *page = area->pages[i]; - - BUG_ON(!page); - mod_memcg_page_state(page, MEMCG_VMALLOC, -1); - /* - * High-order allocs for huge vmallocs are split, so - * can be freed as an array of order-0 allocations - */ - __free_pages(page, 0); - cond_resched(); - } - atomic_long_sub(area->nr_pages, &nr_vmalloc_pages); - - kvfree(area->pages); - } - - kfree(area); -} - static void delayed_vfree_work(struct work_struct *w) { struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); @@ -2757,6 +2718,9 @@ void vfree_atomic(const void *addr) */ void vfree(const void *addr) { + struct vm_struct *vm; + int i; + if (unlikely(in_interrupt())) { vfree_atomic(addr); return; @@ -2766,8 +2730,32 @@ void vfree(const void *addr) kmemleak_free(addr); might_sleep(); - if (addr) - __vunmap(addr, 1); + if (!addr) + return; + + vm = remove_vm_area(addr); + if (unlikely(!vm)) { + WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", + addr); + return; + } + + vm_remove_mappings(vm, true); + for (i = 0; i < vm->nr_pages; i++) { + struct page *page = vm->pages[i]; + + BUG_ON(!page); + mod_memcg_page_state(page, MEMCG_VMALLOC, -1); + /* + * High-order allocs for huge vmallocs are split, so + * can be freed as an array of order-0 allocations + */ + __free_pages(page, 0); + cond_resched(); + } + atomic_long_sub(vm->nr_pages, &nr_vmalloc_pages); + kvfree(vm->pages); + kfree(vm); } EXPORT_SYMBOL(vfree); @@ -2782,10 +2770,20 @@ EXPORT_SYMBOL(vfree); */ void vunmap(const void *addr) { + struct vm_struct *vm; + BUG_ON(in_interrupt()); might_sleep(); - if (addr) - __vunmap(addr, 0); + + if (!addr) + return; + vm = remove_vm_area(addr); + if (unlikely(!vm)) { + WARN(1, KERN_ERR "Trying to vunmap() nonexistent vm area (%p)\n", + addr); + return; + } + kfree(vm); } EXPORT_SYMBOL(vunmap); From patchwork Sat Jan 21 07:10:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57D69C25B50 for ; Sat, 21 Jan 2023 07:11:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F19256B0072; Sat, 21 Jan 2023 02:11:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ECA496B0080; Sat, 21 Jan 2023 02:11:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1BF36B0081; Sat, 21 Jan 2023 02:11:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C4D116B0072 for ; Sat, 21 Jan 2023 02:11:26 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A25861A0F76 for ; Sat, 21 Jan 2023 07:11:26 +0000 (UTC) X-FDA: 80377935372.01.82165B0 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf12.hostedemail.com (Postfix) with ESMTP id 148E540006 for ; Sat, 21 Jan 2023 07:11:24 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BxwUWG+8; spf=none (imf12.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285085; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MVFwvinoUqx5/lg5psoeJPHtHhWtA7mgRAAlRnZDces=; b=BAxBZQhd4MgM37EDyOm/DKPmz0y2IF/IE8CE6AxUI/XH081fTJw26cCyFvwSo+a/w2AkPf pDC6uj7Ivy/+jpuKbbcb9keZgS+FcymjuQA9c628p1uRuwjOCVwHQpk+RdjDKaejsZPe2A XbCo1L7+V7eC4Ptqs28Bj8ecrYfMjSc= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BxwUWG+8; spf=none (imf12.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285085; a=rsa-sha256; cv=none; b=d0+zFjr5lo0boRZSVCmrVtKcD6kQCfpnxYoTQyALlAhM+58TlRbxa/AeudDXBo8zF1GRqb hBkKKhtxAv6rwzd15jkXgM+X7ZLQynbGw8L1+UiMqXXp7b76yKgX0FU5gaC73oDPnxy9nk rN7BuQ+ppEPGGAS/9Gg9ZMRM5rUvqIY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=MVFwvinoUqx5/lg5psoeJPHtHhWtA7mgRAAlRnZDces=; b=BxwUWG+8AyQ3rEml1EmfY53UY3 jW/EOiAn7IqlTNMKUXUHBDOwVMwHW+J1SEgJywBfXznR3TsUsK8VHIJlx/XP1Y2uFhfk5q4ch26MM 7y+8R3biifI9ZO4anC7jzQCQhsQz1ueVyMGEoZdLjBu3UFNnHrbYxLd7Ax/vygdZVrQxbimobqkgN MQq65HflqHD3HuYYp+tS8ILFI00CSdyk0cmGWRPkTw4brDSpV5D2nUoBIZMK5egKsomS4t/kwOMt4 m2O7SKRg094LUFjRiIdW1PtEuwWmbzUL5h7ZduAdeBXj2OnjV0vdvIyZOHo4Ve/gjPjZIeBaAPJva YhBnW1eg==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ823-00DTtM-NY; Sat, 21 Jan 2023 07:11:20 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 10/10] mm: refactor va_remove_mappings Date: Sat, 21 Jan 2023 08:10:51 +0100 Message-Id: <20230121071051.1143058-11-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Stat-Signature: es9894if4fozbsrp5qkrn3akc5p8oncg X-Rspam-User: X-Rspamd-Queue-Id: 148E540006 X-Rspamd-Server: rspam06 X-HE-Tag: 1674285084-990436 X-HE-Meta: U2FsdGVkX18RDox2zcO9hQAySsSYpZeZ9HBMhu5uJ+gCOL0fXztKf2hF4uHiUWP4NfDHBYzywDQfLmkB8MMnV4h5Irk4GUeVDladycCfGm4EsnrEXs9oSWVNIRoKh2WuFwdc7f7za+xIhpt6Ia+l3aPrasNXw3sono8j6VQJqW19hEwKQwc6qRm6J21GT6LVpIGolNuVW0rqNZT3b6DKe0PNbIVxxHF7APqOSTSCqJe0e7oXaeyTB4c+QZfSbANDITEJD/OWDTkOK+zyoqODvrHgWNuHiK5gfVYnBMsuL3rXf5A51og5qt/iIs9iZ0YjM/pcV3UGSem56nmx/8YOVBgSORKZd4/VtwgjdTPtDzeDijJ3qwZ/rSJzuhIgBPd94LKlgRlYRUsAGYKTg9uL8J9Ll9iny25qJV+M2tWmXN4Zb/nVzMQaOVwEeMvK9ICZwim0zekeP2OTIdtqt6CTB8vNuVO2Q/DOB9O5IBW3O1s3WcbvZHzVSjXKxMHrSpjbUomwksYZaySJ4nBb/5xFKsSuxfAP2H9N+W9aLzP20PRE1DV0uNTELqhiAgit4q1Us2WLMhb0guVgHzzwBmGyF5v6+U9zFYsEsKGNNL5NY3e+sB7OAcMyf4OXOP1+XTIE+7BNCkOLEAua55i7LFkGjGm/XXnLskBHOtI7o5xEv1R55LNfY7iKwrCEmwFvk7EK9/yZZke0F/NXfx0e634t51gmJMacwO+z4AYPprgW0uWrEIBpBnpG/695yEEnN0GVfobUWTqumQ3XttzdVIyOLCPEPhlWpWX4PK3SGzpJlnqj8EpHXfnQIj8/+KR7OWBf1Z/I3qCQQnCjDx6jQRqbPMTep7GAs42llWmO1ndaSkUwbSy5ZbBeAm9mjlelruI5z1+OMKkvvcpo2HJp+b0PjJSSs39Z6M7ebCiYSXPv4pi6dU1y6spSQxiMjZX8P2EBkuCUn61MbLdU4ZCNFzU ZoSxOI9T EEEAT1W67f62J71Nv2YzpIagLB4IRB7dpUxwgQopEsDr5e0Llzi6wCvxRLIx4AEwk/DSZTarArGMTn00AFS8zFEfCoJ2RGzdp6nCsrGZ6t6K9ZoNAbaCpi6fuoMcXgxhCH4yvQhkS68fR2/z9QOP0EKyJyjPF1ThKku++gVML9DSbhR/CFI9AX/u0WkHPGOv0mnJVeA9JnSJqzXJHSEdP8eV4L6nSBpTuZFPKbqZsAf5KLw0FBRWgM8vuiq/bKkMtrp5NUu9dYfphSjiuLlm0CRCFSlSBTyNsdF9u8Ki+UwSuDn8ha2q3OqeMnmapx2G6OZQwVwGvV8vml2oxExGa0tEbQ4a+Cpt0y1Jwo99NKzxINOjEAP9nzNL3ImSn/P7egAv00XrxGBGHM8FVEL8NC6qgorPjz8wG5R5HjjYisKYD1usdHivVtTIKm4+tOeYVlUqsnRRYQt4cFT/apfG2ynjsVqNEysfnwhHrwbjwEFrfPJ3RuKvnrh9kZ9G0mzJOhqvUDa8N5RnTv1eaAIruxWdduFnS+glGvCH0UrGp/52pmBD3h9reTBu/nT9sBZgU8cbU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move the VM_FLUSH_RESET_PERMS to the caller and rename the function to better describe what it is doing. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 27 ++++++++------------------- 1 file changed, 8 insertions(+), 19 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 6bd811e4b7561d..dfde5324e4803d 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2617,35 +2617,23 @@ static inline void set_area_direct_map(const struct vm_struct *area, set_direct_map(area->pages[i]); } -/* Handle removing and resetting vm mappings related to the vm_struct. */ -static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) +/* + * Flush the vm mapping and reset the direct map. + */ +static void vm_reset_perms(struct vm_struct *area) { unsigned long start = ULONG_MAX, end = 0; unsigned int page_order = vm_area_page_order(area); - int flush_reset = area->flags & VM_FLUSH_RESET_PERMS; int flush_dmap = 0; int i; - /* If this is not VM_FLUSH_RESET_PERMS memory, no need for the below. */ - if (!flush_reset) - return; - - /* - * If not deallocating pages, just do the flush of the VM area and - * return. - */ - if (!deallocate_pages) { - vm_unmap_aliases(); - return; - } - /* - * If execution gets here, flush the vm mapping and reset the direct - * map. Find the start and end range of the direct mappings to make sure + * Find the start and end range of the direct mappings to make sure that * the vm_unmap_aliases() flush includes the direct map. */ for (i = 0; i < area->nr_pages; i += 1U << page_order) { unsigned long addr = (unsigned long)page_address(area->pages[i]); + if (addr) { unsigned long page_size; @@ -2740,7 +2728,8 @@ void vfree(const void *addr) return; } - vm_remove_mappings(vm, true); + if (unlikely(vm->flags & VM_FLUSH_RESET_PERMS)) + vm_reset_perms(vm); for (i = 0; i < vm->nr_pages; i++) { struct page *page = vm->pages[i];