From patchwork Thu Jan 19 10:02:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13107657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA70C004D4 for ; Thu, 19 Jan 2023 10:06:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA4828E0003; Thu, 19 Jan 2023 05:06:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A551C8E0002; Thu, 19 Jan 2023 05:06:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91CF58E0003; Thu, 19 Jan 2023 05:06:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 849388E0002 for ; Thu, 19 Jan 2023 05:06:44 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 57DAE160D1A for ; Thu, 19 Jan 2023 10:06:44 +0000 (UTC) X-FDA: 80371119528.04.881A679 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf10.hostedemail.com (Postfix) with ESMTP id E2375C0012 for ; Thu, 19 Jan 2023 10:06:42 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=Dbnq8ckF; spf=none (imf10.hostedemail.com: domain of BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674122802; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iyd8tqGiY9j8oDNLq7gNrgqmUQgcTuGZnKBX+j0mJu4=; b=4bOblmdcY04g1ScCb4BoWAGbvuOZPVDTuEuYZiyQ3zQYT4e16vXmbClF0THcQ8ZtkO0qCb 0uCdW9RTJhRd6rMmKzavPh3ZRpJg8iB4vNqSFXj1ZBa3CmG33aLNm4IbvGBvRouofplqz8 AHIH75/s+N7S5EB73xeVvQ1ty/j148M= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=Dbnq8ckF; spf=none (imf10.hostedemail.com: domain of BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+d6c99cb0ec1a2cfa7d20+7088+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674122803; a=rsa-sha256; cv=none; b=0PLsM+aDcUoR4L3mFWNzIGoMlj+859t8tsTdqUwf0LA27XgZ7ozaRFkTACLFlVk82unI8Q RixJVK185f0yr+iLaFket53tTCHvIcTtDrTRwEHZE0QvXBzGl08vfSHCIXf9Ixofae7IRv 2Cwh0pUERth/+23YU7pU7BeXx471fUY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=iyd8tqGiY9j8oDNLq7gNrgqmUQgcTuGZnKBX+j0mJu4=; b=Dbnq8ckFqYSFSkdw/O55JZYdU+ u+dJyDawMnfEO+4NsxlWT7hRoFL8OMJYd6SmxDFF/TukEDZ8sHRtuMPS8BC3tGpesWKEss9honOEv AzfpUcfkZuKlMIX0rTducW5OFmR1ynNht/U2gRNRHclYc1s5eW3y6toEXvpuPuhUnLl+iOizKKOb+ Ep+EbN1ZePYQKAcbbD5dJpfZOfi6YKuqmsCaSxrIQyUXEM0Lk1uMGJWRbofXG5pNlCnSJ1q1Hkply RyRqZmY22UA7KjaJLW5lSagZAKOAlUwt3u0NnjXoXW5YE3J1DkOAIVWmAHo3mpyhC2Q3YnblQF3+t 83Z//cAA==; Received: from 089144197172.atnat0006.highway.a1.net ([89.144.197.172] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pIRoe-004NB5-9M; Thu, 19 Jan 2023 10:06:41 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: linux-mm@kvack.org Subject: [PATCH 09/10] mm: split __vunmap Date: Thu, 19 Jan 2023 11:02:25 +0100 Message-Id: <20230119100226.789506-10-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230119100226.789506-1-hch@lst.de> References: <20230119100226.789506-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: E2375C0012 X-Stat-Signature: rgj69wx4kocmoniwu9g4runstp395kx3 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1674122802-972736 X-HE-Meta: U2FsdGVkX19IoJOYt8PwrLK9SbVn4bG2iNTKqNb2hZ5DwW4XqwbkLH4Tl6OfxdCapUxReudwEyatmYnGkF18OpGlNktnGT3WWbl2rLpb5K2+R1qyjxKDwAjS2bMnE3t+TWU4VrwAbFWJ3FEEuzxW1CesCqwx6/TlZ7wv6LBNgm9y9zX+oDO7Upwwx5UqbQj6aXjvNSdpSSOLuUUpBxjO/rJV8iHK+FggTlT05rjF35zCn6L7iER/r7xlrqdiN9XDD9ctXpsjb7a/gBG4/lawfXV0maKbJx2vjOfh9tmqkrnCYs1uPTGkYffyFFdyQka2cwu181RQL2ZcS1WcwsIMpg/fcPSKcPhwDl+lqWFP4yrZQN/V9nkGyqtlMLQ6S4AX6QxsNMvRZTFSuqBlyYZke/bPbacOH6k5KsH+IWwPlNDe2L1XNQL8A1e2wT1fyrOCmOMGJs/6TNPcE7aoCvvhVMNhmXjHbh+2Q9dc17xE8eYfCnM1Wt3n80sl/LmNmKwr5Opmu6+k8n0z+cx3EcJTyh5E+xRd7peIGirizFeDk60MCmUmzkjouXBA7A7PGN83SftOEiXNgG9TduSrYahSBidZYJjkr2VsJlX58vq+EFIlZcsaWmrMYWlkukOUrZZyhg6v9vWOxacBO3F7xZC93iyjw4JbeQ7XJzAl3MNs8Iy7i8v0RNdwmqTOmuSH/vexQXP+n8uR0/w101Ncz7PhW8PPAa8gCee7qEuWP3oBjxuPZiTuSWwtg93a4nqPZNzCg4lSMQLNExkQ8X3C9fMV9xQsdMd69bpJlyloL2iNUfSIdVePlcbe1OOelQpMe4D2dR/Lo+yXLcZbxfThUccdaVORXe//dGctcTUYOrATQ9+LRDvv3HhBqUDiSACgfpB0VAnNgQKu73Mm4XzlqXZIM3vRnQCjuj3EKARstYdp2OAz1erBfibEMVuCPU5oxRDlACSvjjtMQYg3MZpE5lc gvHxHhNs bVCQLPk69Xm3pMBZBcSEaBXl9NegMCdH9EZbwbRdaGYH+vZ4jpRyiT1T8vFUp5oOY1X6KPxpUEZ0MDV31UhehjfoqpuZrueFboMehhK17iAqSE3jkl6gVSmFuFNmNljpUEBqLRj3QNDFXqgjSZWfO2EZCz8gcJXrsnzscXIDsFnIKMS5/7E5vExx3KzgWq6g1uwMZBsl20ZtH3Z0PQywNTs4tCp8BHm4Zc7YE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: vunmap only needs to find and free the vmap_area and vm_strut, so open code that there and merge the rest of the code into vfree. Signed-off-by: Christoph Hellwig --- mm/vmalloc.c | 85 ++++++++++++++++++++++++++-------------------------- 1 file changed, 42 insertions(+), 43 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 4cb189bdd51499..791d906d7e407c 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2666,45 +2666,6 @@ static void va_remove_mappings(struct vm_struct *area, int deallocate_pages) set_area_direct_map(area, set_direct_map_default_noflush); } -static void __vunmap(const void *addr, int deallocate_pages) -{ - struct vm_struct *area; - - if (!addr) - return; - - area = remove_vm_area(addr); - if (unlikely(!area)) { - WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", - addr); - return; - } - - va_remove_mappings(area, deallocate_pages); - - if (deallocate_pages) { - int i; - - for (i = 0; i < area->nr_pages; i++) { - struct page *page = area->pages[i]; - - BUG_ON(!page); - mod_memcg_page_state(page, MEMCG_VMALLOC, -1); - /* - * High-order allocs for huge vmallocs are split, so - * can be freed as an array of order-0 allocations - */ - __free_pages(page, 0); - cond_resched(); - } - atomic_long_sub(area->nr_pages, &nr_vmalloc_pages); - - kvfree(area->pages); - } - - kfree(area); -} - static void delayed_vfree_work(struct work_struct *w) { struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); @@ -2757,6 +2718,9 @@ void vfree_atomic(const void *addr) */ void vfree(const void *addr) { + struct vm_struct *vm; + int i; + if (unlikely(in_interrupt())) { vfree_atomic(addr); return; @@ -2766,8 +2730,32 @@ void vfree(const void *addr) kmemleak_free(addr); might_sleep(); - if (addr) - __vunmap(addr, 1); + if (!addr) + return; + + vm = remove_vm_area(addr); + if (unlikely(!vm)) { + WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", + addr); + return; + } + + va_remove_mappings(vm, true); + for (i = 0; i < vm->nr_pages; i++) { + struct page *page = vm->pages[i]; + + BUG_ON(!page); + mod_memcg_page_state(page, MEMCG_VMALLOC, -1); + /* + * High-order allocs for huge vmallocs are split, so + * can be freed as an array of order-0 allocations + */ + __free_pages(page, 0); + cond_resched(); + } + atomic_long_sub(vm->nr_pages, &nr_vmalloc_pages); + kvfree(vm->pages); + kfree(vm); } EXPORT_SYMBOL(vfree); @@ -2782,10 +2770,21 @@ EXPORT_SYMBOL(vfree); */ void vunmap(const void *addr) { + struct vm_struct *vm; + BUG_ON(in_interrupt()); might_sleep(); - if (addr) - __vunmap(addr, 0); + + if (!addr) + return; + vm = remove_vm_area(addr); + if (unlikely(!vm)) { + WARN(1, KERN_ERR "Trying to vunmap() nonexistent vm area (%p)\n", + addr); + return; + } + WARN_ON_ONCE(vm->flags & VM_FLUSH_RESET_PERMS); + kfree(vm); } EXPORT_SYMBOL(vunmap);