From patchwork Fri Jul 25 19:00:37 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Salter X-Patchwork-Id: 4625281 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 915B99F36A for ; Fri, 25 Jul 2014 19:03:13 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C17E52011E for ; Fri, 25 Jul 2014 19:03:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A99C220155 for ; Fri, 25 Jul 2014 19:03:11 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XAkjt-0003Ha-U9; Fri, 25 Jul 2014 19:01:09 +0000 Received: from mx1.redhat.com ([209.132.183.28]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XAkjr-0003At-D5 for linux-arm-kernel@lists.infradead.org; Fri, 25 Jul 2014 19:01:08 +0000 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s6PJ0c7M014231 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Jul 2014 15:00:38 -0400 Received: from [10.3.113.127] (ovpn-113-127.phx2.redhat.com [10.3.113.127]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s6PJ0bJ9004752; Fri, 25 Jul 2014 15:00:37 -0400 Message-ID: <1406314837.27055.25.camel@deneb.redhat.com> Subject: memory leak in arm kvm From: Mark Salter To: christoffer.dall@linaro.org, marc.zyngier@arm.com Date: Fri, 25 Jul 2014 15:00:37 -0400 Organization: Red Hat, Inc Mime-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140725_120107_490186_AE18FFF5 X-CRM114-Status: GOOD ( 14.31 ) X-Spam-Score: -5.0 (-----) Cc: kvmarm@lists.cs.columbia.edu, linux-arm-kernel X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I've been looking into a memory leak in the arm kvm code which has been seen on arm64 platforms. It seems there is a small (2-4MB depending on pagesize) leak when a guest is started and stopped. The leak is happening in arch/arm/kvm/mmu.c when kvm_free_stage2_pgd() tries to unmap everything and free the pgd allocation. The problem I see is that unmap_range() does not remove all the page references to the pgd so when free_pages() is called, the pages are not actually freed because of page->count > 1. Looking further, the call to unmap_range() only ever calls put_page() once for the pgd and then it bails out of the while loop. This happens because of the arm64 kvm_p?d_addr_end() macros. They are defined to the normal p?d_addr_end macros. On arm64 with 3-level page tables, pud_addr_end() simply advances to the end. With 2-level page tables, pud_addr_end() and pmd_addr_end() both advance to end. So when the bottom of the while loop in unmap_range() uses those to advance to next pmd or pud, the loop ends up exiting. I can get around this by open coding the kvm_p?d_addr_end macros thusly: I'm not at all sure this is a correct/complete fix and I'm not really familiar with the design of the arm kvm code, so I'll leave it to others to decide that. diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 7d29847..d7f77ff 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -122,8 +122,16 @@ static inline void kvm_set_s2pmd_writable(pmd_t *pmd) } #define kvm_pgd_addr_end(addr, end) pgd_addr_end(addr, end) -#define kvm_pud_addr_end(addr, end) pud_addr_end(addr, end) -#define kvm_pmd_addr_end(addr, end) pmd_addr_end(addr, end) + +#define kvm_pud_addr_end(addr, end) \ +({ unsigned long __boundary = ((addr) + PUD_SIZE) & PUD_MASK; \ + (__boundary - 1 < (end) - 1)? __boundary: (end); \ +}) + +#define kvm_pmd_addr_end(addr, end) \ +({ unsigned long __boundary = ((addr) + PMD_SIZE) & PMD_MASK; \ + (__boundary - 1 < (end) - 1)? __boundary: (end); \ +}) struct kvm;