From patchwork Wed Dec 11 12:07:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284829 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DFA1138D for ; Wed, 11 Dec 2019 12:31:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4B18420836 for ; Wed, 11 Dec 2019 12:31:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="GQp5EnhZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B18420836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DF8A06B31F0; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D06E96B31F3; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B586C6B31F2; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id 9D8606B31F0 for ; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 4DD118249980 for ; Wed, 11 Dec 2019 12:31:21 +0000 (UTC) X-FDA: 76252795962.17.coil47_45f63565fa92f X-Spam-Summary: 2,0,0,153d326fd23d8a49,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com:geert+renesas@glider.be,RULES_HIT:41:152:355:379:800:960:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3874:4250:4385:5007:6261:6653:6742:8634:9592:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:12679:13069:13221:13229:13311:13357:14181:14394:14659:14721:21080:21433:21451:21627:30003:30054:30055,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Do mainCach X-HE-Tag: coil47_45f63565fa92f X-Filterd-Recvd-Size: 4169 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=LTZ/BPfJTPVp0bgwkMHutu1dARSjSH9NSFpOWuJS0KI=; b=GQp5EnhZlhc0hKF8Vwa6Mskdi1 UfPcurHOEBZni5wRyasGcG4fCkyZjzSvxlJ/Gu5hbQ7dPfUqQxZAY3cP2mhVN9VGplKUhqr1/625g uZJ4qRBnW6m5jD71YcDOU/OiqNWSIJSLWOUBdKmxgTz3/u3BvHJf6oqrgUw79kKNjd4ZsqUMyQS0i BmBH9gqK8L9tSy3O/8zJjvNaZzIxoAmvokMDpoFApIjRyi/Y1B497JToHbMm58kOqKLA04OWT1TmN vHbvLQK60tbSK1LdHDF413P+wBgmlDcjU/4YJSgKobQ2pk/a/bwhfA5Y2QV2A0ITU4R+h58heLQDE 6QqKg4yA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if18z-0001PJ-Tz; Wed, 11 Dec 2019 12:31:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 8800530603E; Wed, 11 Dec 2019 13:29:42 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 1BF152026FBC3; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122955.711253433@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:14 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , Geert Uytterhoeven Subject: [PATCH 01/17] sh/tlb: Fix PGTABLE_LEVELS > 2 References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Geert reported that his SH7722-based Migo-R board failed to boot after commit: c5b27a889da9 ("sh/tlb: Convert SH to generic mmu_gather") That commit fell victim to copying the wrong pattern -- __pmd_free_tlb() used to be implemented with pmd_free(). Fixes: c5b27a889da9 ("sh/tlb: Convert SH to generic mmu_gather") Reported-by: Geert Uytterhoeven Signed-off-by: Peter Zijlstra (Intel) Tested-by: Geert Uytterhoeven --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -12,6 +12,7 @@ extern void pgd_free(struct mm_struct *m extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); +#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, @@ -33,13 +34,4 @@ do { \ tlb_remove_page((tlb), (pte)); \ } while (0) -#if CONFIG_PGTABLE_LEVELS > 2 -#define __pmd_free_tlb(tlb, pmdp, addr) \ -do { \ - struct page *page = virt_to_page(pmdp); \ - pgtable_pmd_page_dtor(page); \ - tlb_remove_page((tlb), page); \ -} while (0); -#endif - #endif /* __ASM_SH_PGALLOC_H */ From patchwork Wed Dec 11 12:07:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284821 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E369138D for ; Wed, 11 Dec 2019 12:31:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4B1A920836 for ; Wed, 11 Dec 2019 12:31:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="K7tCp0EA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B1A920836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 95EDA6B31EC; Wed, 11 Dec 2019 07:31:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8E7D66B31ED; Wed, 11 Dec 2019 07:31:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 789B06B31EE; Wed, 11 Dec 2019 07:31:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id 5E4006B31EC for ; Wed, 11 Dec 2019 07:31:20 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 136172C1F for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) X-FDA: 76252795920.06.death02_45cdf04ff2362 X-Spam-Summary: 2,0,0,5006852e7e640a91,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1539:1568:1593:1594:1711:1714:1730:1747:1777:1792:1981:2194:2199:2393:2559:2562:3138:3139:3140:3141:3142:3865:4605:5007:6261:6653:6742:8634:10004:10400:11026:11658:11914:12048:12296:12297:12517:12519:12679:13069:13311:13357:14096:14097:14181:14394:14659:14721:21080:21451:21627:30003:30054:30055,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-62.8.0.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0, MSF:not X-HE-Tag: death02_45cdf04ff2362 X-Filterd-Recvd-Size: 3526 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=wY/JMK2ksK+TYA9CE9BrGh6vp4ksJO5HTccgxj/n5mg=; b=K7tCp0EABlCPqIMEVXo8+gF9jq wEpLYwCrFWuTkb7GnvHi3zYR1nbobBXwNrTVghC2eEj46vNvvZJVuv8AiqjpjPbdWgO4A4TsAWlCd KSUFCFXXzDDyYrg+fTcGygJJjQkV1toneITfqYfMPJ1yFMHOwJ8tVptRpeeRofO8ghHKC3saP9Qem enqdbanXdjgqOhyePhU6Fs6GWhHZCAyMlRossm4SHmuKzMlaK1ZhSd1v34O0D7bi/svHgEpCp7DDA MPdHsZbou7hVaQQFFC6W2DPVXyDtH0RJHv8DelnEwyDXkWkN7rSjmVqPbQ546GiXJgE9xsvQD9JS+ zGVCmLNw==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0001Pc-QJ; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 83880306011; Wed, 11 Dec 2019 13:29:42 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 1ED1420309147; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122955.769069740@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:15 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 02/17] asm-gemeric/tlb: Remove stray function declarations References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We removed the actual functions a while ago. Fixes: 1808d65b55e4 ("asm-generic/tlb: Remove arch_tlb*_mmu()") Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Geert Uytterhoeven --- include/asm-generic/tlb.h | 4 ---- 1 file changed, 4 deletions(-) --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -277,11 +277,7 @@ struct mmu_gather { #endif }; -void arch_tlb_gather_mmu(struct mmu_gather *tlb, - struct mm_struct *mm, unsigned long start, unsigned long end); void tlb_flush_mmu(struct mmu_gather *tlb); -void arch_tlb_finish_mmu(struct mmu_gather *tlb, - unsigned long start, unsigned long end, bool force); static inline void __tlb_adjust_range(struct mmu_gather *tlb, unsigned long address, From patchwork Wed Dec 11 12:07:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284827 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B317138D for ; Wed, 11 Dec 2019 12:31:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EC8D6214AF for ; Wed, 11 Dec 2019 12:31:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="LAce8X50" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC8D6214AF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7665A6B31EF; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6A0036B31F0; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 569676B31F2; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id 3CDDF6B31F0 for ; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id F30638249980 for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) X-FDA: 76252795920.26.nose40_45f325117d008 X-Spam-Summary: 2,0,0,5450fdbd744eb483,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:334:355:368:369:379:800:960:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1538:1561:1593:1594:1711:1714:1730:1747:1777:1792:1978:1981:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3865:3870:5007:6261:6653:6742:8634:10004:10400:11026:11473:11658:11914:12048:12297:12438:12517:12519:12679:13069:13311:13357:14181:14394:14659:14721:21080:21451:21611:21627:30054:30055:30089,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-62.8.0.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:n one,Doma X-HE-Tag: nose40_45f325117d008 X-Filterd-Recvd-Size: 3284 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=WStPWl7EHmUdstQXR4srCPTzKuR9hsSl99Nmt3uj2dk=; b=LAce8X50AVM8BS3e/9HHGQ0CKf m7rfVkZyFP+8O4hYosBtqAQueS6cDsUUOkyzh7PbDlfU0GuXjAIQvYS+DCJK9epplpQg9jRoT7Kpr ezBEteoupovlta+/3oi0HJFFRyuqNY+z3qlBfq9+5hH2p3G9BNcFE7VRIsENIbFFA2KlKL+4hdirL djY8La6VYEpPziZmaAamU+jkNDEeKb9nipBkukW62zgz+j4rIJRWbPQ8xZfhOm95EL9w1xLGwFppO 6qBGuNihvgQC5pRhZJcXuL2iy9WMjCXy6gLNyhpgonXpkRj6LAv1A9GHrUSKwRB60548MJhWNFjRZ WB0YOOoA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if18z-0001PL-Tl; Wed, 11 Dec 2019 12:31:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 79841300F29; Wed, 11 Dec 2019 13:29:42 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 20C8A20137C8F; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122955.825865931@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:16 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 03/17] asm-generic/tlb: Add missing CONFIG symbol References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Without this the symbol will not actually end up in .config files. Fixes: a30e32bd79e9 ("asm-generic/tlb: Provide generic tlb_flush() based on flush_tlb_mm()") Signed-off-by: Peter Zijlstra (Intel) --- arch/Kconfig | 3 +++ 1 file changed, 3 insertions(+) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -402,6 +402,9 @@ config HAVE_RCU_TABLE_NO_INVALIDATE config HAVE_MMU_GATHER_PAGE_SIZE bool +config MMU_GATHER_NO_RANGE + bool + config HAVE_MMU_GATHER_NO_GATHER bool From patchwork Wed Dec 11 12:07:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284819 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A10E139A for ; Wed, 11 Dec 2019 12:31:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE9FC20836 for ; Wed, 11 Dec 2019 12:31:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="oZmgj+/P" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE9FC20836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D58DF6B31EB; Wed, 11 Dec 2019 07:31:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BD0976B31EC; Wed, 11 Dec 2019 07:31:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9842D6B31ED; Wed, 11 Dec 2019 07:31:19 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id 8266A6B31EB for ; Wed, 11 Dec 2019 07:31:19 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 15230180AD817 for ; Wed, 11 Dec 2019 12:31:19 +0000 (UTC) X-FDA: 76252795878.29.wing48_45a3d0848a960 X-Spam-Summary: 10,1,0,46737b36966c9194,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:1:2:41:152:355:379:404:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1978:1981:2194:2196:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:4050:4250:4385:4390:4395:4605:5007:6119:6261:6653:6691:6742:6755:7903:8603:8634:8660:10004:11026:11473:11657:11658:11914:12043:12048:12219:12295:12296:12297:12438:12517:12519:13148:13230:14096:14097:14394:14659:21080:21451:21627:21772:30012:30054:30055:30074:30089,0,RB L:198.13 X-HE-Tag: wing48_45a3d0848a960 X-Filterd-Recvd-Size: 10233 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=wcM6hHyLxjZAtwtLpkncfN6HnampEr2pkXBPL1E/TmU=; b=oZmgj+/Pkg+g0RJKKS5g+ihq3R uDsRRT0AoQJzUjVUL1pYLVCx9BCVk6XY7gfyAccgOh0Ej2lIztN0dKreB7jWSjuCXi7lMgCqGYrZG TgHTpm8ThK53nuiRUySztt4P6zXFrMXByN/wVS2YAGGPDza88P+ZkBLbyjlarLL31MjrUG4Xi4d2P 3gRHKJJ0ONAaBM+sDXaCNUhOuKvoXcKxp/QfnXJuhBCiiBWDPz23iC/GBnr8ajUDU2TBou+cOWr6I skKWWLLaYR1bVdUG/z7iSmslwrnfDxXILIkd2T7gny36JqS2b4OQrlFcUhQwS7n66geYcSSJy3qw9 ZcKWqiGA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0001PZ-LC; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 82AA8305FFF; Wed, 11 Dec 2019 13:29:42 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 2757D20137CAF; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122955.883347160@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:17 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 04/17] asm-generic/tlb: Rename HAVE_RCU_TABLE_FREE References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Towards a more consistent naming scheme. Signed-off-by: Peter Zijlstra (Intel) --- arch/Kconfig | 2 +- arch/arm/Kconfig | 2 +- arch/arm/include/asm/tlb.h | 2 +- arch/arm64/Kconfig | 2 +- arch/powerpc/Kconfig | 4 ++-- arch/s390/Kconfig | 2 +- arch/sparc/Kconfig | 4 ++-- arch/x86/Kconfig | 2 +- arch/x86/include/asm/tlb.h | 4 ++-- include/asm-generic/tlb.h | 10 +++++----- mm/gup.c | 2 +- mm/mmu_gather.c | 8 ++++---- 12 files changed, 22 insertions(+), 22 deletions(-) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -393,7 +393,7 @@ config HAVE_ARCH_JUMP_LABEL config HAVE_ARCH_JUMP_LABEL_RELATIVE bool -config HAVE_RCU_TABLE_FREE +config MMU_GATHER_RCU_TABLE_FREE bool config HAVE_RCU_TABLE_NO_INVALIDATE --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -101,7 +101,7 @@ config ARM select HAVE_PERF_EVENTS select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP - select HAVE_RCU_TABLE_FREE if SMP && ARM_LPAE + select MMU_GATHER_RCU_TABLE_FREE if SMP && ARM_LPAE select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RSEQ select HAVE_STACKPROTECTOR --- a/arch/arm/include/asm/tlb.h +++ b/arch/arm/include/asm/tlb.h @@ -37,7 +37,7 @@ static inline void __tlb_remove_table(vo #include -#ifndef CONFIG_HAVE_RCU_TABLE_FREE +#ifndef CONFIG_MMU_GATHER_RCU_TABLE_FREE #define tlb_remove_table(tlb, entry) tlb_remove_page(tlb, entry) #endif --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -161,7 +161,7 @@ config ARM64 select HAVE_PERF_USER_STACK_DUMP select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_FUNCTION_ARG_ACCESS_API - select HAVE_RCU_TABLE_FREE + select MMU_GATHER_RCU_TABLE_FREE select HAVE_RSEQ select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -222,8 +222,8 @@ config PPC select HAVE_HARDLOCKUP_DETECTOR_PERF if PERF_EVENTS && HAVE_PERF_EVENTS_NMI && !HAVE_HARDLOCKUP_DETECTOR_ARCH select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP - select HAVE_RCU_TABLE_FREE if SMP - select HAVE_RCU_TABLE_NO_INVALIDATE if HAVE_RCU_TABLE_FREE + select MMU_GATHER_RCU_TABLE_FREE if SMP + select HAVE_RCU_TABLE_NO_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE select HAVE_MMU_GATHER_PAGE_SIZE select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE if PPC_BOOK3S_64 && CPU_LITTLE_ENDIAN --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -168,7 +168,7 @@ config S390 select HAVE_OPROFILE select HAVE_PCI select HAVE_PERF_EVENTS - select HAVE_RCU_TABLE_FREE + select MMU_GATHER_RCU_TABLE_FREE select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE select HAVE_RSEQ --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -64,8 +64,8 @@ config SPARC64 select HAVE_FUNCTION_GRAPH_TRACER select HAVE_KRETPROBES select HAVE_KPROBES - select HAVE_RCU_TABLE_FREE if SMP - select HAVE_RCU_TABLE_NO_INVALIDATE if HAVE_RCU_TABLE_FREE + select MMU_GATHER_RCU_TABLE_FREE if SMP + select HAVE_RCU_TABLE_NO_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE select HAVE_MEMBLOCK_NODE_MAP select HAVE_ARCH_TRANSPARENT_HUGEPAGE select HAVE_DYNAMIC_FTRACE --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -200,7 +200,7 @@ config X86 select HAVE_PCI select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP - select HAVE_RCU_TABLE_FREE if PARAVIRT + select MMU_GATHER_RCU_TABLE_FREE if PARAVIRT select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE if X86_64 && (UNWINDER_FRAME_POINTER || UNWINDER_ORC) && STACK_VALIDATION select HAVE_FUNCTION_ARG_ACCESS_API --- a/arch/x86/include/asm/tlb.h +++ b/arch/x86/include/asm/tlb.h @@ -29,8 +29,8 @@ static inline void tlb_flush(struct mmu_ * shootdown, enablement code for several hypervisors overrides * .flush_tlb_others hook in pv_mmu_ops and implements it by issuing * a hypercall. To keep software pagetable walkers safe in this case we - * switch to RCU based table free (HAVE_RCU_TABLE_FREE). See the comment - * below 'ifdef CONFIG_HAVE_RCU_TABLE_FREE' in include/asm-generic/tlb.h + * switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the comment + * below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h * for more details. */ static inline void __tlb_remove_table(void *table) --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -126,7 +126,7 @@ * This ensures we call tlb_flush() every time tlb_change_page_size() actually * changes the size and provides mmu_gather::page_size to tlb_flush(). * - * HAVE_RCU_TABLE_FREE + * MMU_GATHER_RCU_TABLE_FREE * * This provides tlb_remove_table(), to be used instead of tlb_remove_page() * for page directores (__p*_free_tlb()). This provides separate freeing of @@ -139,9 +139,9 @@ * * HAVE_RCU_TABLE_NO_INVALIDATE * - * This makes HAVE_RCU_TABLE_FREE avoid calling tlb_flush_mmu_tlbonly() before + * This makes MMU_GATHER_RCU_TABLE_FREE avoid calling tlb_flush_mmu_tlbonly() before * freeing the page-table pages. This can be avoided if you use - * HAVE_RCU_TABLE_FREE and your architecture does _NOT_ use the Linux + * MMU_GATHER_RCU_TABLE_FREE and your architecture does _NOT_ use the Linux * page-tables natively. * * MMU_GATHER_NO_RANGE @@ -149,7 +149,7 @@ * Use this if your architecture lacks an efficient flush_tlb_range(). */ -#ifdef CONFIG_HAVE_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* * Semi RCU freeing of the page directories. * @@ -227,7 +227,7 @@ extern bool __tlb_remove_page_size(struc struct mmu_gather { struct mm_struct *mm; -#ifdef CONFIG_HAVE_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE struct mmu_table_batch *batch; #endif --- a/mm/gup.c +++ b/mm/gup.c @@ -1729,7 +1729,7 @@ EXPORT_SYMBOL(get_user_pages_unlocked); * Before activating this code, please be aware that the following assumptions * are currently made: * - * *) Either HAVE_RCU_TABLE_FREE is enabled, and tlb_remove_table() is used to + * *) Either MMU_GATHER_RCU_TABLE_FREE is enabled, and tlb_remove_table() is used to * free pages containing page tables or TLB flushing requires IPI broadcast. * * *) ptes can be read atomically by the architecture. --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -91,7 +91,7 @@ bool __tlb_remove_page_size(struct mmu_g #endif /* HAVE_MMU_GATHER_NO_GATHER */ -#ifdef CONFIG_HAVE_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* * See the comment near struct mmu_table_batch. @@ -173,11 +173,11 @@ void tlb_remove_table(struct mmu_gather tlb_table_flush(tlb); } -#endif /* CONFIG_HAVE_RCU_TABLE_FREE */ +#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ static void tlb_flush_mmu_free(struct mmu_gather *tlb) { -#ifdef CONFIG_HAVE_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE tlb_table_flush(tlb); #endif #ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER @@ -220,7 +220,7 @@ void tlb_gather_mmu(struct mmu_gather *t tlb->batch_count = 0; #endif -#ifdef CONFIG_HAVE_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE tlb->batch = NULL; #endif #ifdef CONFIG_HAVE_MMU_GATHER_PAGE_SIZE From patchwork Wed Dec 11 12:07:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284837 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAE24139A for ; Wed, 11 Dec 2019 12:31:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7836C21556 for ; Wed, 11 Dec 2019 12:31:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="jDRKoJ/S" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7836C21556 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DD9FF6B31F2; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D5AC96B31F7; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE3E16B31F6; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0065.hostedemail.com [216.40.44.65]) by kanga.kvack.org (Postfix) with ESMTP id 5CEF46B31F2 for ; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 16FF6181AEF1D for ; Wed, 11 Dec 2019 12:31:22 +0000 (UTC) X-FDA: 76252796004.24.pigs29_46144f38c9129 X-Spam-Summary: 2,0,0,b129907022e88006,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1978:1981:2194:2196:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3353:3865:3866:3867:3871:3873:4250:4385:4605:5007:6119:6261:6653:6742:7903:8634:10004:10400:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:14096:14097:14181:14394:14659:14721:21080:21324:21451:21611:21627:21772:30054:30055:30089,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-62.8.0.100 64.201.2 X-HE-Tag: pigs29_46144f38c9129 X-Filterd-Recvd-Size: 5227 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=EAd4sniYvQfVFa7iAd7B3Jx5jzSbOy6mN0wfPQQPFoA=; b=jDRKoJ/SUad35TrcNy3bpq3pYH G2Li0tKMTO5RKQEokjs6Kt5t3F4sEI8aW20OunEnMRuBkdooU9uC1XhpMds6k9iSYha6xCF6EdSpk GiYm7n+WLSb7vhnB2tsxfToJZ28TGmHbPdUtPp0LCV/YElfkzhjWOmrDMH45SS4cLqnMhWXOrr7No IxaISBT8TTbFppEQrfPiM85BEzqj8uSU/qjaPaQhjq0JRts/D1tgg8o0o+Gy9dFXcimcjWTCVmP7a iAWI8Q2RWrvZ0rTdT7aYnJ44sHlwijTP5AJydoumdsWroeisevxXzV/2gOt52vlaS2apwXEfVcsr9 arHuMVJA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if18z-0001PM-TU; Wed, 11 Dec 2019 12:31:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 98CCD30604B; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 2E4AA20137CB0; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122955.940455408@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:18 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 05/17] asm-generic/tlb: Rename HAVE_RCU_TABLE_NO_INVALIDATE References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Towards a more consistent naming scheme. Signed-off-by: Peter Zijlstra (Intel) --- arch/Kconfig | 3 ++- arch/powerpc/Kconfig | 2 +- arch/sparc/Kconfig | 2 +- include/asm-generic/tlb.h | 2 +- mm/mmu_gather.c | 2 +- 5 files changed, 6 insertions(+), 5 deletions(-) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -396,8 +396,9 @@ config HAVE_ARCH_JUMP_LABEL_RELATIVE config MMU_GATHER_RCU_TABLE_FREE bool -config HAVE_RCU_TABLE_NO_INVALIDATE +config MMU_GATHER_NO_TABLE_INVALIDATE bool + depends on MMU_GATHER_RCU_TABLE_FREE config HAVE_MMU_GATHER_PAGE_SIZE bool --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -223,7 +223,7 @@ config PPC select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP select MMU_GATHER_RCU_TABLE_FREE if SMP - select HAVE_RCU_TABLE_NO_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE + select MMU_GATHER_NO_TABLE_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE select HAVE_MMU_GATHER_PAGE_SIZE select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE if PPC_BOOK3S_64 && CPU_LITTLE_ENDIAN --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -65,7 +65,7 @@ config SPARC64 select HAVE_KRETPROBES select HAVE_KPROBES select MMU_GATHER_RCU_TABLE_FREE if SMP - select HAVE_RCU_TABLE_NO_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE + select MMU_GATHER_NO_TABLE_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE select HAVE_MEMBLOCK_NODE_MAP select HAVE_ARCH_TRANSPARENT_HUGEPAGE select HAVE_DYNAMIC_FTRACE --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -137,7 +137,7 @@ * When used, an architecture is expected to provide __tlb_remove_table() * which does the actual freeing of these pages. * - * HAVE_RCU_TABLE_NO_INVALIDATE + * MMU_GATHER_NO_TABLE_INVALIDATE * * This makes MMU_GATHER_RCU_TABLE_FREE avoid calling tlb_flush_mmu_tlbonly() before * freeing the page-table pages. This can be avoided if you use --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -102,7 +102,7 @@ bool __tlb_remove_page_size(struct mmu_g */ static inline void tlb_table_invalidate(struct mmu_gather *tlb) { -#ifndef CONFIG_HAVE_RCU_TABLE_NO_INVALIDATE +#ifndef CONFIG_MMU_GATHER_NO_TABLE_INVALIDATE /* * Invalidate page-table caches used by hardware walkers. Then we still * need to RCU-sched wait while freeing the pages because software From patchwork Wed Dec 11 12:07:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D10A7139A for ; Wed, 11 Dec 2019 12:31:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 948332173E for ; Wed, 11 Dec 2019 12:31:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Mzf9cK+2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 948332173E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E41346B31F9; Wed, 11 Dec 2019 07:31:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DA29E6B31FA; Wed, 11 Dec 2019 07:31:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF9086B31FB; Wed, 11 Dec 2019 07:31:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0045.hostedemail.com [216.40.44.45]) by kanga.kvack.org (Postfix) with ESMTP id 9BA6C6B31F9 for ; Wed, 11 Dec 2019 07:31:32 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 570F040EE for ; Wed, 11 Dec 2019 12:31:32 +0000 (UTC) X-FDA: 76252796424.05.idea48_479cebd859124 X-Spam-Summary: 2,0,0,c5ffb515a515855c,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1978:1981:2194:2196:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:4250:4385:4605:5007:6119:6261:6653:6742:8603:8634:8660:8784:10004:10400:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:13148:13230:14096:14097:14181:14394:14659:14721:21080:21627:30054:30055:30089,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-62.8.55.100 64.201. 201.201, X-HE-Tag: idea48_479cebd859124 X-Filterd-Recvd-Size: 5539 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=krexXFAWafcSDWP7nl53PyLhwjx3LAnf0o/FpsyPrGc=; b=Mzf9cK+2CmiAS4VMfcObJYmTHI AxATk9mhf/VYv6YQbPqClbGHnPHwPiWDg76zUcgOoadRhn31LVxj9DQxfILmwLoIzH1So/ebZA+uL j7pszd4ugX+U5PK3UnJAzcPnAtTLT+v8CPAlzHNSIqrzfG10MMGYBd+jkXUOOiLQIWTxDVaz6nrui uqRgZPI0gzkpmVKsX9/1Zu2L8Ty+qeBnvWTATNzlkw993CiP61OrFyw270zfhFbcbdw+EsAHOX2jJ 310kxZij0cDqU5GDH4JAT224CKhq6jj5itwSamZfEHWRiPqnYVPyw8E6/4TISpLpRAA77V5Vv0k14 RhlPw++w==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0003sk-9J; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 9CB2E306060; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 3287320137CB1; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122955.997638865@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:19 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 06/17] asm-generic/tlb: Rename HAVE_MMU_GATHER_PAGE_SIZE References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Towards a more consistent naming scheme. Signed-off-by: Peter Zijlstra (Intel) --- arch/Kconfig | 2 +- arch/powerpc/Kconfig | 2 +- include/asm-generic/tlb.h | 9 ++++++--- mm/mmu_gather.c | 4 ++-- 4 files changed, 10 insertions(+), 7 deletions(-) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -400,7 +400,7 @@ config MMU_GATHER_NO_TABLE_INVALIDATE bool depends on MMU_GATHER_RCU_TABLE_FREE -config HAVE_MMU_GATHER_PAGE_SIZE +config MMU_GATHER_PAGE_SIZE bool config MMU_GATHER_NO_RANGE --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -224,7 +224,7 @@ config PPC select HAVE_PERF_USER_STACK_DUMP select MMU_GATHER_RCU_TABLE_FREE if SMP select MMU_GATHER_NO_TABLE_INVALIDATE if MMU_GATHER_RCU_TABLE_FREE - select HAVE_MMU_GATHER_PAGE_SIZE + select MMU_GATHER_PAGE_SIZE select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RELIABLE_STACKTRACE if PPC_BOOK3S_64 && CPU_LITTLE_ENDIAN select HAVE_SYSCALL_TRACEPOINTS --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -121,11 +121,14 @@ * * Additionally there are a few opt-in features: * - * HAVE_MMU_GATHER_PAGE_SIZE + * MMU_GATHER_PAGE_SIZE * * This ensures we call tlb_flush() every time tlb_change_page_size() actually * changes the size and provides mmu_gather::page_size to tlb_flush(). * + * This might be useful if your architecture has size specific TLB + * invalidation instructions. + * * MMU_GATHER_RCU_TABLE_FREE * * This provides tlb_remove_table(), to be used instead of tlb_remove_page() @@ -271,7 +274,7 @@ struct mmu_gather { struct mmu_gather_batch local; struct page *__pages[MMU_GATHER_BUNDLE]; -#ifdef CONFIG_HAVE_MMU_GATHER_PAGE_SIZE +#ifdef CONFIG_MMU_GATHER_PAGE_SIZE unsigned int page_size; #endif #endif @@ -422,7 +425,7 @@ static inline void tlb_remove_page(struc static inline void tlb_change_page_size(struct mmu_gather *tlb, unsigned int page_size) { -#ifdef CONFIG_HAVE_MMU_GATHER_PAGE_SIZE +#ifdef CONFIG_MMU_GATHER_PAGE_SIZE if (tlb->page_size && tlb->page_size != page_size) { if (!tlb->fullmm && !tlb->need_flush_all) tlb_flush_mmu(tlb); --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -69,7 +69,7 @@ bool __tlb_remove_page_size(struct mmu_g VM_BUG_ON(!tlb->end); -#ifdef CONFIG_HAVE_MMU_GATHER_PAGE_SIZE +#ifdef CONFIG_MMU_GATHER_PAGE_SIZE VM_WARN_ON(tlb->page_size != page_size); #endif @@ -223,7 +223,7 @@ void tlb_gather_mmu(struct mmu_gather *t #ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE tlb->batch = NULL; #endif -#ifdef CONFIG_HAVE_MMU_GATHER_PAGE_SIZE +#ifdef CONFIG_MMU_GATHER_PAGE_SIZE tlb->page_size = 0; #endif From patchwork Wed Dec 11 12:07:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284901 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 20FE8139A for ; Wed, 11 Dec 2019 12:52:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CA6D12173E for ; Wed, 11 Dec 2019 12:52:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ToapP2ql" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA6D12173E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 270856B3218; Wed, 11 Dec 2019 07:52:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 248C46B3219; Wed, 11 Dec 2019 07:52:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15D606B321A; Wed, 11 Dec 2019 07:52:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0230.hostedemail.com [216.40.44.230]) by kanga.kvack.org (Postfix) with ESMTP id F3E076B3218 for ; Wed, 11 Dec 2019 07:52:37 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id A91062C12 for ; Wed, 11 Dec 2019 12:52:37 +0000 (UTC) X-FDA: 76252849554.29.use38_6e2b261a4891d X-Spam-Summary: 2,0,0,d03c09dcac835f39,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:1978:1981:2194:2196:2199:2200:2393:2559:2562:2693:3138:3139:3140:3141:3142:3354:3865:3866:3867:3870:3871:3872:3874:4117:4250:4385:4605:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:14181:14394:14659:14721:21080:21451:21627:21740:21990:30054:30055:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache :0,MSF:n X-HE-Tag: use38_6e2b261a4891d X-Filterd-Recvd-Size: 6191 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:52:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=kha1DNRsNJX9wg9pqNRv2AfGj3UZUSQsnrv68oCgIj0=; b=ToapP2qlFGSbItIorKP5sCoHSM Yv+euDU9Y7Y7yxk43M7SLRy9A4z9BFDWddAnZJWoizRRhswezDlHNRd0DolNbPDM7U0P3wcydTSBK oyT9bRlG5cK6Y+vkBIFxXCaaEyhxXCmVR5BPsueaEPvpHCLl2AgPxGN718ZQ0sjVaq1BQfLW3Spvu 1TTaJFwhZsUD19UiLLA2T9O4AMWBGx+ANbTUgKVW9Z4NVm7eObFzyPIAbZx+rgC7ikj6/sMybRYJ2 d+pXmIrAesnXvzL6vN3/VsXP5OKYhpG10XnpIwXcEfpe9diDY3ysEFw38AOvQPLYW65Pj1Sg/SNDY /J8spJeg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if18z-0001PN-U7; Wed, 11 Dec 2019 12:31:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id BB13A30606A; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 3478520137CAE; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.055418242@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:20 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 07/17] asm-generic/tlb: Rename HAVE_MMU_GATHER_NO_GATHER References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Towards a more consistent naming scheme. Signed-off-by: Peter Zijlstra (Intel) --- arch/Kconfig | 2 +- arch/s390/Kconfig | 2 +- include/asm-generic/tlb.h | 14 ++++++++++++-- mm/mmu_gather.c | 10 +++++----- 4 files changed, 19 insertions(+), 9 deletions(-) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -406,7 +406,7 @@ config MMU_GATHER_PAGE_SIZE config MMU_GATHER_NO_RANGE bool -config HAVE_MMU_GATHER_NO_GATHER +config MMU_GATHER_NO_GATHER bool config ARCH_HAVE_NMI_SAFE_CMPXCHG --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -162,7 +162,7 @@ config S390 select HAVE_PERF_USER_STACK_DUMP select HAVE_MEMBLOCK_NODE_MAP select HAVE_MEMBLOCK_PHYS_MAP - select HAVE_MMU_GATHER_NO_GATHER + select MMU_GATHER_NO_GATHER select HAVE_MOD_ARCH_SPECIFIC select HAVE_NOP_MCOUNT select HAVE_OPROFILE --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -147,6 +147,16 @@ * MMU_GATHER_NO_RANGE * * Use this if your architecture lacks an efficient flush_tlb_range(). + * + * MMU_GATHER_NO_GATHER + * + * If the option is set the mmu_gather will not track individual pages for + * delayed page free anymore. A platform that enables the option needs to + * provide its own implementation of the __tlb_remove_page_size() function to + * free pages. + * + * This is useful if your architecture already flushes TLB entries in the + * various ptep_get_and_clear() functions. */ #ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE @@ -191,7 +201,7 @@ extern void tlb_remove_table(struct mmu_ #endif -#ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER +#ifndef CONFIG_MMU_GATHER_NO_GATHER /* * If we can't allocate a page to make a big batch of page pointers * to work on, then just handle a few from the on-stack structure. @@ -266,7 +276,7 @@ struct mmu_gather { unsigned int batch_count; -#ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER +#ifndef CONFIG_MMU_GATHER_NO_GATHER struct mmu_gather_batch *active; struct mmu_gather_batch local; struct page *__pages[MMU_GATHER_BUNDLE]; --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -11,7 +11,7 @@ #include #include -#ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER +#ifndef CONFIG_MMU_GATHER_NO_GATHER static bool tlb_next_batch(struct mmu_gather *tlb) { @@ -89,7 +89,7 @@ bool __tlb_remove_page_size(struct mmu_g return false; } -#endif /* HAVE_MMU_GATHER_NO_GATHER */ +#endif /* MMU_GATHER_NO_GATHER */ #ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE @@ -180,7 +180,7 @@ static void tlb_flush_mmu_free(struct mm #ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE tlb_table_flush(tlb); #endif -#ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER +#ifndef CONFIG_MMU_GATHER_NO_GATHER tlb_batch_pages_flush(tlb); #endif } @@ -211,7 +211,7 @@ void tlb_gather_mmu(struct mmu_gather *t /* Is it from 0 to ~0? */ tlb->fullmm = !(start | (end+1)); -#ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER +#ifndef CONFIG_MMU_GATHER_NO_GATHER tlb->need_flush_all = 0; tlb->local.next = NULL; tlb->local.nr = 0; @@ -271,7 +271,7 @@ void tlb_finish_mmu(struct mmu_gather *t tlb_flush_mmu(tlb); -#ifndef CONFIG_HAVE_MMU_GATHER_NO_GATHER +#ifndef CONFIG_MMU_GATHER_NO_GATHER tlb_batch_list_free(tlb); #endif dec_tlb_flush_pending(tlb->mm); From patchwork Wed Dec 11 12:07:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284839 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A8D53139A for ; Wed, 11 Dec 2019 12:31:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 51618214AF for ; Wed, 11 Dec 2019 12:31:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="tLnuOKMO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51618214AF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A99116B31F6; Wed, 11 Dec 2019 07:31:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A21E66B31F8; Wed, 11 Dec 2019 07:31:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84D716B31F7; Wed, 11 Dec 2019 07:31:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0210.hostedemail.com [216.40.44.210]) by kanga.kvack.org (Postfix) with ESMTP id 6662D6B31F4 for ; Wed, 11 Dec 2019 07:31:28 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 19EF6181AEF1E for ; Wed, 11 Dec 2019 12:31:28 +0000 (UTC) X-FDA: 76252796256.17.crate52_46f73a5fcab16 X-Spam-Summary: 10,1,0,c20fbae502b1ac0c,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:1:41:69:152:355:379:404:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1978:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2637:2693:2731:2737:2904:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:4250:4385:4390:4395:4605:5007:6117:6119:6261:6653:6691:6742:7903:8603:8634:9168:11026:11473:11658:11914:12043:12048:12114:12291:12295:12296:12297:12438:12517:12519:12683:14096:14097:14394:14659:21080:21324:21451:21611:21627:21772:30012:30 054:3005 X-HE-Tag: crate52_46f73a5fcab16 X-Filterd-Recvd-Size: 14436 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=U0k4mN9cFCu5+aqx6ymdkEnsXLUi+bB76rWj6pAfRWM=; b=tLnuOKMOP8bBm6ghrrl24UoXqX qQcFf4ijFCoTeOx/9kLNycpIlse9BCYd9HlnauCdB20DcY4xxd9AfG/s28SiKHWCUB8gcpefMATxM jKxrnV+af7VgG1Y3o7BYswmsKlvQc5I/WD760SDEywhBkmUaDafbAk4ZILaNl0DLSDGn2P9HEeCfS +qm6Qg7/IrFMaOkDqHDWfnRBUNyQP3QCL4KT43XE2suSOEFfiqg5FbrbQvxm5T7gypyjkSfP4a6iG DLPy8EfZooKPxxq/EoJVI3rHVylLVGUBxx5G+LAPCa9LoqY6TYdF0FGhMiz3BJHqhufdh6zpdnGuf iPs3H4hw==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if192-0003sz-OK; Wed, 11 Dec 2019 12:31:09 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id C4F32306117; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 391BF20137CB3; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.112607298@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:21 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 08/17] asm-generic/tlb: Provide MMU_GATHER_TABLE_FREE References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As described in the comment, the correct order for freeing pages is: 1) unhook page 2) TLB invalidate page 3) free page This order equally applies to page directories. Currently there are two correct options: - use tlb_remove_page(), when all page directores are full pages and there are no futher contraints placed by things like software walkers (HAVE_FAST_GUP). - use MMU_GATHER_RCU_TABLE_FREE and tlb_remove_table() when the architecture does not do IPI based TLB invalidate and has HAVE_FAST_GUP (or software TLB fill). This however leaves architectures that don't have page based directories but don't need RCU in a bind. For those, provide MMU_GATHER_TABLE_FREE, which provides the independent batching for directories without the additional RCU freeing. Signed-off-by: Peter Zijlstra (Intel) --- arch/Kconfig | 5 + include/asm-generic/tlb.h | 76 ++++++++++++++--------------- mm/mmu_gather.c | 120 +++++++++++++++++++++++++++++++++------------- 3 files changed, 131 insertions(+), 70 deletions(-) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -393,8 +393,12 @@ config HAVE_ARCH_JUMP_LABEL config HAVE_ARCH_JUMP_LABEL_RELATIVE bool +config MMU_GATHER_TABLE_FREE + bool + config MMU_GATHER_RCU_TABLE_FREE bool + select MMU_GATHER_TABLE_FREE config MMU_GATHER_NO_TABLE_INVALIDATE bool @@ -408,6 +412,7 @@ config MMU_GATHER_NO_RANGE config MMU_GATHER_NO_GATHER bool + depends on MMU_GATHER_TABLE_FREE config ARCH_HAVE_NMI_SAFE_CMPXCHG bool --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -56,6 +56,15 @@ * Defaults to flushing at tlb_end_vma() to reset the range; helps when * there's large holes between the VMAs. * + * - tlb_remove_table() + * + * tlb_remove_table() is the basic primitive to free page-table directories + * (__p*_free_tlb()). In it's most primitive form it is an alias for + * tlb_remove_page() below, for when page directories are pages and have no + * additional constraints. + * + * See also MMU_GATHER_TABLE_FREE and MMU_GATHER_RCU_TABLE_FREE. + * * - tlb_remove_page() / __tlb_remove_page() * - tlb_remove_page_size() / __tlb_remove_page_size() * @@ -129,21 +138,28 @@ * This might be useful if your architecture has size specific TLB * invalidation instructions. * - * MMU_GATHER_RCU_TABLE_FREE + * MMU_GATHER_TABLE_FREE * * This provides tlb_remove_table(), to be used instead of tlb_remove_page() - * for page directores (__p*_free_tlb()). This provides separate freeing of - * the page-table pages themselves in a semi-RCU fashion (see comment below). - * Useful if your architecture doesn't use IPIs for remote TLB invalidates - * and therefore doesn't naturally serialize with software page-table walkers. + * for page directores (__p*_free_tlb()). + * + * Useful if your architecture has non-page page directories. * * When used, an architecture is expected to provide __tlb_remove_table() * which does the actual freeing of these pages. * + * MMU_GATHER_RCU_TABLE_FREE + * + * Like MMU_GATHER_TABLE_FREE, and adds semi-RCU semantics to the free (see + * comment below). + * + * Useful if your architecture doesn't use IPIs for remote TLB invalidates + * and therefore doesn't naturally serialize with software page-table walkers. + * * MMU_GATHER_NO_TABLE_INVALIDATE * - * This makes MMU_GATHER_RCU_TABLE_FREE avoid calling tlb_flush_mmu_tlbonly() before - * freeing the page-table pages. This can be avoided if you use + * This makes MMU_GATHER_RCU_TABLE_FREE avoid calling tlb_flush_mmu_tlbonly() + * before freeing the page-table pages. This can be avoided if you use * MMU_GATHER_RCU_TABLE_FREE and your architecture does _NOT_ use the Linux * page-tables natively. * @@ -162,37 +178,12 @@ * various ptep_get_and_clear() functions. */ -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE -/* - * Semi RCU freeing of the page directories. - * - * This is needed by some architectures to implement software pagetable walkers. - * - * gup_fast() and other software pagetable walkers do a lockless page-table - * walk and therefore needs some synchronization with the freeing of the page - * directories. The chosen means to accomplish that is by disabling IRQs over - * the walk. - * - * Architectures that use IPIs to flush TLBs will then automagically DTRT, - * since we unlink the page, flush TLBs, free the page. Since the disabling of - * IRQs delays the completion of the TLB flush we can never observe an already - * freed page. - * - * Architectures that do not have this (PPC) need to delay the freeing by some - * other means, this is that means. - * - * What we do is batch the freed directory pages (tables) and RCU free them. - * We use the sched RCU variant, as that guarantees that IRQ/preempt disabling - * holds off grace periods. - * - * However, in order to batch these pages we need to allocate storage, this - * allocation is deep inside the MM code and can thus easily fail on memory - * pressure. To guarantee progress we fall back to single table freeing, see - * the implementation of tlb_remove_table_one(). - * - */ +#ifdef CONFIG_MMU_GATHER_TABLE_FREE + struct mmu_table_batch { +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE struct rcu_head rcu; +#endif unsigned int nr; void *tables[0]; }; @@ -202,7 +193,16 @@ struct mmu_table_batch { extern void tlb_remove_table(struct mmu_gather *tlb, void *table); -#endif +#else /* !CONFIG_MMU_GATHER_HAVE_TABLE_FREE */ + +/* + * Without either HAVE_TABLE_FREE || CONFIG_HAVE_RCU_TABLE_FREE the + * architecture is assumed to have page based page directories and + * we can use the normal page batching to free them. + */ +#define tlb_remove_table(tlb, page) tlb_remove_page((tlb), (page)) + +#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ #ifndef CONFIG_MMU_GATHER_NO_GATHER /* @@ -240,7 +240,7 @@ extern bool __tlb_remove_page_size(struc struct mmu_gather { struct mm_struct *mm; -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_TABLE_FREE struct mmu_table_batch *batch; #endif --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -91,56 +91,106 @@ bool __tlb_remove_page_size(struct mmu_g #endif /* MMU_GATHER_NO_GATHER */ -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE +#ifdef CONFIG_MMU_GATHER_TABLE_FREE -/* - * See the comment near struct mmu_table_batch. - */ +static void __tlb_remove_table_free(struct mmu_table_batch *batch) +{ + int i; + + for (i = 0; i < batch->nr; i++) + __tlb_remove_table(batch->tables[i]); + + free_page((unsigned long)batch); +} + +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* - * If we want tlb_remove_table() to imply TLB invalidates. + * Semi RCU freeing of the page directories. + * + * This is needed by some architectures to implement software pagetable walkers. + * + * gup_fast() and other software pagetable walkers do a lockless page-table + * walk and therefore needs some synchronization with the freeing of the page + * directories. The chosen means to accomplish that is by disabling IRQs over + * the walk. + * + * Architectures that use IPIs to flush TLBs will then automagically DTRT, + * since we unlink the page, flush TLBs, free the page. Since the disabling of + * IRQs delays the completion of the TLB flush we can never observe an already + * freed page. + * + * Architectures that do not have this (PPC) need to delay the freeing by some + * other means, this is that means. + * + * What we do is batch the freed directory pages (tables) and RCU free them. + * We use the sched RCU variant, as that guarantees that IRQ/preempt disabling + * holds off grace periods. + * + * However, in order to batch these pages we need to allocate storage, this + * allocation is deep inside the MM code and can thus easily fail on memory + * pressure. To guarantee progress we fall back to single table freeing, see + * the implementation of tlb_remove_table_one(). + * */ -static inline void tlb_table_invalidate(struct mmu_gather *tlb) -{ -#ifndef CONFIG_MMU_GATHER_NO_TABLE_INVALIDATE - /* - * Invalidate page-table caches used by hardware walkers. Then we still - * need to RCU-sched wait while freeing the pages because software - * walkers can still be in-flight. - */ - tlb_flush_mmu_tlbonly(tlb); -#endif -} static void tlb_remove_table_smp_sync(void *arg) { /* Simply deliver the interrupt */ } -static void tlb_remove_table_one(void *table) +static void tlb_remove_table_sync_one(void) { /* * This isn't an RCU grace period and hence the page-tables cannot be * assumed to be actually RCU-freed. * * It is however sufficient for software page-table walkers that rely on - * IRQ disabling. See the comment near struct mmu_table_batch. + * IRQ disabling. */ smp_call_function(tlb_remove_table_smp_sync, NULL, 1); - __tlb_remove_table(table); } static void tlb_remove_table_rcu(struct rcu_head *head) { - struct mmu_table_batch *batch; - int i; + __tlb_remove_table_free(container_of(head, struct mmu_table_batch, rcu)); +} - batch = container_of(head, struct mmu_table_batch, rcu); +static void tlb_remove_table_free(struct mmu_table_batch *batch) +{ + call_rcu(&batch->rcu, tlb_remove_table_rcu); +} - for (i = 0; i < batch->nr; i++) - __tlb_remove_table(batch->tables[i]); +#else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */ - free_page((unsigned long)batch); +static void tlb_remove_table_sync_one(void) { } + +static void tlb_remove_table_free(struct mmu_table_batch *batch) +{ + __tlb_remove_table_free(batch); +} + +#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ + +/* + * If we want tlb_remove_table() to imply TLB invalidates. + */ +static inline void tlb_table_invalidate(struct mmu_gather *tlb) +{ +#ifndef CONFIG_MMU_GATHER_NO_TABLE_INVALIDATE + /* + * Invalidate page-table caches used by hardware walkers. Then we still + * need to RCU-sched wait while freeing the pages because software + * walkers can still be in-flight. + */ + tlb_flush_mmu_tlbonly(tlb); +#endif +} + +static void tlb_remove_table_one(void *table) +{ + tlb_remove_table_sync_one(); + __tlb_remove_table(table); } static void tlb_table_flush(struct mmu_gather *tlb) @@ -149,7 +199,7 @@ static void tlb_table_flush(struct mmu_g if (*batch) { tlb_table_invalidate(tlb); - call_rcu(&(*batch)->rcu, tlb_remove_table_rcu); + tlb_remove_table_free(*batch); *batch = NULL; } } @@ -173,13 +223,21 @@ void tlb_remove_table(struct mmu_gather tlb_table_flush(tlb); } -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ +static inline void tlb_table_init(struct mmu_gather *tlb) +{ + tlb->batch = NULL; +} + +#else /* !CONFIG_MMU_GATHER_TABLE_FREE */ + +static inline void tlb_table_flush(struct mmu_gather *tlb) { } +static inline void tlb_table_init(struct mmu_gather *tlb) { } + +#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ static void tlb_flush_mmu_free(struct mmu_gather *tlb) { -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE tlb_table_flush(tlb); -#endif #ifndef CONFIG_MMU_GATHER_NO_GATHER tlb_batch_pages_flush(tlb); #endif @@ -220,9 +278,7 @@ void tlb_gather_mmu(struct mmu_gather *t tlb->batch_count = 0; #endif -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE - tlb->batch = NULL; -#endif + tlb_table_init(tlb); #ifdef CONFIG_MMU_GATHER_PAGE_SIZE tlb->page_size = 0; #endif From patchwork Wed Dec 11 12:07:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C092C138D for ; Wed, 11 Dec 2019 12:32:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E11C2173E for ; Wed, 11 Dec 2019 12:32:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="eyYSAOGb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E11C2173E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CBA1E6B31FA; Wed, 11 Dec 2019 07:31:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C6A7C6B31FB; Wed, 11 Dec 2019 07:31:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B31C46B31FC; Wed, 11 Dec 2019 07:31:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0076.hostedemail.com [216.40.44.76]) by kanga.kvack.org (Postfix) with ESMTP id 99C596B31FA for ; Wed, 11 Dec 2019 07:31:33 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 573E3180AD817 for ; Wed, 11 Dec 2019 12:31:33 +0000 (UTC) X-FDA: 76252796466.22.toys64_47bff1bcb2c62 X-Spam-Summary: 2,0,0,a4c2597226bd9e69,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3872:3874:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:14096:14097:14181:14394:14659:14721:21080:21451:21627:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-6 2.8.55.1 X-HE-Tag: toys64_47bff1bcb2c62 X-Filterd-Recvd-Size: 4889 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=FvghZFBaoqrMAEVkDDRcPbYOLQl3+Nssq3SxvKZJaQI=; b=eyYSAOGbwuKI+/tsaYEU5bznHd rczIvlRuyztabhFl3bCt9VDUhDf4mBhA+lY7x9tqz4PAb0skfrtoPhJTPMeiZ4XeRU53ia0VekyOO uAo2tqtKGg8GYJfmFe9iPytIgsf5uDNGXLsqdXk4pgOgeu+PaWVbZ6XrF9+bIxUA40EqldwR/L/Gb /qnN85DQU/nWxRFmoXeIpZ6vSzYHsewDDJIpLLokkREYuXmaxAYWyPBx5cLIRYgv3yoAJz/+6TlUL YYV/irr9dWhr4QniMt6SZQNJfFTAIx7bAWgrQ6qJF3PGPEbAwbt1uGi0X0Axl/q0HTGQr3Kh7KjY5 mAIfyp/A==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0003sl-9d; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id E6C463066B3; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 3B16F20137CB2; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.169764611@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:22 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 09/17] sh/tlb: Fix __pmd_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: SH violates the freeing order for it's PMD page directories. It's __pmd_free_tlb() does not ensure there is a TLB invalidation between itself and the eventualy freeing of the page. Further complicating the situation is that SH uses non page based allocation for it's PMDs. Use the shiny new HAVE_TABLE_FREE option to enable a custom page table freeer. (SuperH uses IPI based TLB invalidation and therefore doesn't need HAVE_RCU_TABLE_FREE for its HAVE_FAST_GUP). Signed-off-by: Peter Zijlstra (Intel) --- arch/sh/Kconfig | 1 + arch/sh/include/asm/pgalloc.h | 3 ++- arch/sh/mm/pgtable.c | 6 ++++++ 3 files changed, 9 insertions(+), 1 deletion(-) --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -15,6 +15,7 @@ config SUPERH select HAVE_PERF_EVENTS select HAVE_DEBUG_BUGVERBOSE select HAVE_FAST_GUP if MMU + select MMU_GATHER_TABLE_FREE if X2TLB select ARCH_HAVE_CUSTOM_GPIO_H select ARCH_HAVE_NMI_SAFE_CMPXCHG if (GUSA_RB || CPU_SH4A) select ARCH_HAS_GCOV_PROFILE_ALL --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -12,7 +12,8 @@ extern void pgd_free(struct mm_struct *m extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); -#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) +extern void __tlb_remove_table(void *table); +#define __pmd_free_tlb(tlb, pmdp, addr) tlb_remove_table((tlb), (pmdp)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, --- a/arch/sh/mm/pgtable.c +++ b/arch/sh/mm/pgtable.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #define PGALLOC_GFP GFP_KERNEL | __GFP_ZERO @@ -55,4 +56,9 @@ void pmd_free(struct mm_struct *mm, pmd_ { kmem_cache_free(pmd_cachep, pmd); } + +void __tlb_remove_table(void *table) +{ + pmd_free(NULL, table); +} #endif /* PAGETABLE_LEVELS > 2 */ From patchwork Wed Dec 11 12:07:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284817 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1B8B214E3 for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DCEC22173E for ; Wed, 11 Dec 2019 12:31:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="l1m7Gu9a" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DCEC22173E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 05EF66B31EA; Wed, 11 Dec 2019 07:31:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F293C6B31EB; Wed, 11 Dec 2019 07:31:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF14C6B31EC; Wed, 11 Dec 2019 07:31:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0028.hostedemail.com [216.40.44.28]) by kanga.kvack.org (Postfix) with ESMTP id C58D76B31EA for ; Wed, 11 Dec 2019 07:31:18 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 64DB7181AEF1E for ; Wed, 11 Dec 2019 12:31:18 +0000 (UTC) X-FDA: 76252795836.14.stove29_45838c2966144 X-Spam-Summary: 2,0,0,48c8f05109f6a933,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3354:3865:3867:3871:3872:4385:4390:4395:4605:5007:6119:6261:6653:6742:7903:8603:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:14181:14394:14659:14721:21080:21451:21627:30054:30055,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-62.8.0.100 64.201.201. 201,Cach X-HE-Tag: stove29_45838c2966144 X-Filterd-Recvd-Size: 5508 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ms24ZH5KSAe8YqglSclvMlUcqKf59iwRs7IAhQbihiw=; b=l1m7Gu9a4Q2E3VAhtFH0Vp6pEx 7EKYlfQaoYSG5yxKZAShvCgoRjLQLsCVedrCySOZcv6m5VGUW/XMk4JyEIv+2kdPEoFbrZBhEmU8/ sWx5LLMB2Ga23SlgeQXhLty27LpTfHyB27BJYG9fH0adgzixITKepoNnnkINnwB8yaevGHnAQbFL3 Py5E6bYHv7SO0HuoKwzAv1DtGWUwjCu2q1YuH6L4xT82EeEC9n9gMLQBk+3m7oamydC1AagQQynhU n0o5I6r+eRnMLPrTGoDkI2fSRDgs5EURT1qY7MZkiJl+4Zh1PcEwBoNz5PAFyBp/H8FH+rZvIszY3 u223hRVA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0001Pa-Lc; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id E6C323061E6; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 3FD7920137CB6; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.227316370@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:23 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 10/17] sparc32/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Because Sparc32 has non-page based page directories, use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/sparc/Kconfig | 1 + arch/sparc/include/asm/pgalloc_32.h | 7 +++++-- arch/sparc/mm/srmmu.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 2 deletions(-) --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -57,6 +57,7 @@ config SPARC32 select CLZ_TAB select HAVE_UID16 select OLD_SIGACTION + select MMU_GATHER_TABLE_FREE config SPARC64 def_bool 64BIT --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -12,6 +12,9 @@ struct page; +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx); +extern void __tlb_remove_table(void *table); + void *srmmu_get_nocache(int size, int align); void srmmu_free_nocache(void *addr, int size); @@ -48,7 +51,7 @@ static inline void free_pmd_fast(pmd_t * } #define pmd_free(mm, pmd) free_pmd_fast(pmd) -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) pgtable_free_tlb((tlb), (pmd), 1) void pmd_populate(struct mm_struct *mm, pmd_t *pmdp, struct page *ptep); #define pmd_pgtable(pmd) pmd_page(pmd) @@ -72,6 +75,6 @@ static inline void free_pte_fast(pte_t * #define pte_free_kernel(mm, pte) free_pte_fast(pte) void pte_free(struct mm_struct * mm, pgtable_t pte); -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, addr) pgtable_free_tlb((tlb), (pte), 0) #endif /* _SPARC_PGALLOC_H */ --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -38,6 +38,7 @@ #include #include #include +#include #include /* Now the cpu specific definitions. */ @@ -1849,3 +1850,30 @@ void __init load_mmu(void) sun4m_init_smp(); #endif } + +#define TLB_IDX_MASK 1UL + +void __tlb_remove_table(void *table) +{ + void *dir = (void *)((unsigned long)table & ~TLB_IDX_MASK); + int idx = (unsigned long)table & TLB_IDX_MASK; + + switch (idx) { + case 1: /* PMD */ + pmd_free(NULL, dir); + break; + case 0: /* PTE */ + pte_free(NULL, dir); + break; + } +} + +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx) +{ + unsigned long pgf = (unsigned long)table; + BUG_ON(idx > TLB_IDX_MASK); + BUG_ON(pgf & TLB_IDX_MASK); + pgf |= idx; + tlb_remove_table(tlb, (void *)pgf); +} + From patchwork Wed Dec 11 12:07:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284841 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94F1A138D for ; Wed, 11 Dec 2019 12:31:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 587BD214AF for ; Wed, 11 Dec 2019 12:31:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="SBq4aNiE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 587BD214AF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E93E76B31F4; Wed, 11 Dec 2019 07:31:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DF3596B31F7; Wed, 11 Dec 2019 07:31:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BFA696B31F9; Wed, 11 Dec 2019 07:31:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0203.hostedemail.com [216.40.44.203]) by kanga.kvack.org (Postfix) with ESMTP id 960F66B31F4 for ; Wed, 11 Dec 2019 07:31:28 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 4C5F7181AEF1D for ; Wed, 11 Dec 2019 12:31:28 +0000 (UTC) X-FDA: 76252796256.17.cow80_4703d39144b51 X-Spam-Summary: 2,0,0,0f555f5125f5e321,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3354:3865:3867:3868:3871:3872:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:7903:8603:8634:10004:10394:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:14181:14394:14659:14721:21080:21451:21627:21789:21990:30012:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl 8.mailsh X-HE-Tag: cow80_4703d39144b51 X-Filterd-Recvd-Size: 5351 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=jViyJyc27KPeg0UifOYoPRTZbieI85V/fY78N3x1I9Q=; b=SBq4aNiEimiVT923ThYU5T1BWY 1WNnRyNYHvP1B55z03gtpKtRHJrKE3sDnwyC18/WB2fa0c5zoyj7LRl1xxxuH96nlcXzvKUozyFBQ mMNNucnRFKyqNGv6M4ytMrNGLAOfH6CBHfZvSuR6lCMu4KyI9xagC5pKzO8do81f2ZOlvGuEMp5Kn GEdrD4AlyxqDBgDNVVTr9+yIUK+01uWon5q2BFxFAjACCOnhnIqnB7C7ff1U0F8OOeD8Lb9NJqNWp 3WHFrNTCKMuDW6Af4PkXTSSefss8rQ+YyYORfzaadXfM/l9DDICk9KCGu6VUN1Y3iV+fRJ5Ua9GKm RvNrOqMw==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0003sm-AO; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id EB92F306BB9; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 4206520137CB5; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.285169327@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:24 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 11/17] parisc/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Because PARISC has non-page-size PMDs, use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/parisc/Kconfig | 1 + arch/parisc/include/asm/pgalloc.h | 25 ++++++++++++++++++++++--- arch/parisc/include/asm/tlb.h | 7 +++++-- 3 files changed, 28 insertions(+), 5 deletions(-) --- a/arch/parisc/Kconfig +++ b/arch/parisc/Kconfig @@ -62,6 +62,7 @@ config PARISC select HAVE_FTRACE_MCOUNT_RECORD if HAVE_DYNAMIC_FTRACE select HAVE_KPROBES_ON_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS + select MMU_GATHER_TABLE_FREE if PGTABLE_LEVELS >= 3 help The PA-RISC microprocessor is designed by Hewlett-Packard and used --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -73,7 +73,7 @@ static inline pmd_t *pmd_alloc_one(struc return pmd; } -static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +static inline bool __pmd_free(struct mm_struct *mm, pmd_t *pmd) { if (pmd_flag(*pmd) & PxD_FLAG_ATTACHED) { /* @@ -83,11 +83,28 @@ static inline void pmd_free(struct mm_st * done by generic mm code. */ mm_inc_nr_pmds(mm); - return; + return false; } - free_pages((unsigned long)pmd, PMD_ORDER); + return true; +} + +static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +{ + if (__pmd_free(mm, pmd)) + free_pages((unsigned long)pmd, PMD_ORDER); } +static inline void __tlb_remove_table(void *table) +{ + free_pages((unsigned long)table, PMD_ORDER); +} + +#define __pmd_free_tlb(tlb, pmd, addr) \ +do { \ + if (__pmd_free((tlb)->mm, (pmd))) \ + tlb_remove_table((tlb), (pmd)); \ +} while (0) + #else /* Two Level Page Table Support for pmd's */ @@ -101,6 +118,8 @@ static inline void pmd_free(struct mm_st #define pmd_free(mm, x) do { } while (0) #define pgd_populate(mm, pmd, pte) BUG() +#define __pmd_free_tlb(tlb, pmd, addr) do { } while (0) + #endif static inline void --- a/arch/parisc/include/asm/tlb.h +++ b/arch/parisc/include/asm/tlb.h @@ -4,7 +4,10 @@ #include -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb,pte,addr) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_page((tlb), (pte)); \ +} while (0) #endif From patchwork Wed Dec 11 12:07:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284823 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4EF0139A for ; Wed, 11 Dec 2019 12:31:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A89F820836 for ; Wed, 11 Dec 2019 12:31:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="U+50Fffw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A89F820836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 018A36B31ED; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EDA0C6B31EE; Wed, 11 Dec 2019 07:31:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA2766B31EF; Wed, 11 Dec 2019 07:31:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id C46A96B31ED for ; Wed, 11 Dec 2019 07:31:20 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 6B401181AEF1D for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) X-FDA: 76252795920.16.feast06_45d356aa26e01 X-Spam-Summary: 2,0,0,1ab8e2288b051cbb,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:69:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1801:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3353:3865:3867:3871:3872:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:8634:9592:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13161:13229:14181:14394:14659:14721:21080:21451:21627:21990:30003:30054:30055,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-6 2.8.0.10 X-HE-Tag: feast06_45d356aa26e01 X-Filterd-Recvd-Size: 5316 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=6kZNNxSi21G4pXOCtXTpl9gJxQNMiVTuEix3Ulh9cmE=; b=U+50FffwCXOOmCFvs0ul2LtaFV hZXEbR8JwFIHgJxOtnzEuLJhbTgu/iLzJyBkFwz+/L9ig1+9iG+tN/50OiRVoeGNimDy0Cbj4xuY7 bO2Vs8F0SOAtThWosCnNtDOez0aUiE4CX6pieCiH/ItSRf9VEM1GdopsL384J5Q2+a8ZpAjzZ/fQd 5of6hD4fwuTT1mD9s0EvEfKwaFrLhS8jHCzJSRmSjHqz3WIFHPescBDqQ0xwffllkhZgBb71wGqaC YDcVwIo2S+eiZH/q4Md3azal/pDEv9YXSI6Gg9Gnj5Hp8/s7qQxFFodqMxsGDL45vFOVDS8xqRJT8 ipsqcG0w==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0001Pb-LA; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id EC9E8306BF0; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 4770520137CB9; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.342306385@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:25 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 12/17] m68k/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. The Motorola MMU has funny PMD stuff, so use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/m68k/Kconfig | 1 + arch/m68k/include/asm/mcf_pgalloc.h | 11 +++++------ arch/m68k/include/asm/motorola_pgalloc.h | 22 ++++++++++------------ 3 files changed, 16 insertions(+), 18 deletions(-) --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -31,6 +31,7 @@ config M68K select OLD_SIGSUSPEND3 select OLD_SIGACTION select MMU_GATHER_NO_RANGE if MMU + select MMU_GATHER_TABLE_FREE if MMU_MOTOROLA config CPU_BIG_ENDIAN def_bool y --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -38,12 +38,11 @@ extern inline pmd_t *pmd_alloc_kernel(pg #define pmd_pgtable(pmd) pmd_page(pmd) -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t page, - unsigned long address) -{ - pgtable_pte_page_dtor(page); - __free_page(page); -} +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_page((tlb), (pte)); \ +} while (0) #define __pmd_free_tlb(tlb, pmd, address) do { } while (0) --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -57,15 +57,13 @@ static inline void pte_free(struct mm_st __free_page(page); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t page, - unsigned long address) -{ - pgtable_pte_page_dtor(page); - cache_page(kmap(page)); - kunmap(page); - __free_page(page); -} - +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + cache_page(kmap(pte)); \ + kunmap(pte); \ + tlb_remove_page((tlb), (pte)); \ +} while (0) static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) { @@ -77,12 +75,12 @@ static inline int pmd_free(struct mm_str return free_pointer_table(pmd); } -static inline int __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, - unsigned long address) +static inline void __tlb_remove_table(void *table) { - return free_pointer_table(pmd); + free_pointer_table(table); } +#define __pmd_free_tlb(tlb, pmd, address) tlb_remove_table((tlb), (pmd)) static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) { From patchwork Wed Dec 11 12:07:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284831 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F908139A for ; Wed, 11 Dec 2019 12:31:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D54A220836 for ; Wed, 11 Dec 2019 12:31:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="WrnTBGto" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D54A220836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3468D6B31F5; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2815C6B31F3; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D9376B31F4; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id C9CC26B31F1 for ; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 7657A180AD817 for ; Wed, 11 Dec 2019 12:31:21 +0000 (UTC) X-FDA: 76252795962.21.dogs37_45fdc97efea1e X-Spam-Summary: 2,0,0,691b7ea845e0b611,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com:paul.burton@mips.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3355:3865:3867:3870:3871:3872:3874:4117:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:7875:7903:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12683:14181:14394:14659:14721:14877:21080:21451:21627:21772:21990:30003:30054:3005 5,0,RBL: X-HE-Tag: dogs37_45fdc97efea1e X-Filterd-Recvd-Size: 6424 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Kmci9S6AR0FYTskb+05odp4dpXOL68/q4robJqM04t0=; b=WrnTBGtoDFlccGkA5QZK9j0uWw 57DBHjQs0S/lsu1PhmhB9IFpejHbtD9kJq7MeSq8LN4BazpDPoWGfZgdkZid6plViTf+3YXazhJfu VZOZo2nvohUmKhR9bOufnk25W2mgUeXw5Tfo7KFJfoRIwMOfddpM5oPxw6nA9kwv6owTz/y4qRZ8E tnbT0r8MQDL8luI0zqxoflw/kl3Q0PeOEXULKaQURxxxtYHg7wZqt2vrofd2zT0vljXj+RbcexQWz 1nYzsUwh7KcGVspUYsRZVZh2ms+UsnKRtgHlhQtz2bdo4U9wjvzXJrCMPmQtQw1R8ZzLhOaPooP6D nt7poLqQ==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if193-0001Pj-2S; Wed, 11 Dec 2019 12:31:09 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id F0A57306CBC; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 4BA3120137CBA; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.399804770@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:26 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , Paul Burton Subject: [PATCH 13/17] mips/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. MIPS has (thanks to Paul Burton for setting me straight): - HAVE_FAST_GUP, which means we need to consider software walkers (many MIPS also have software TLB-fill, which is a similar software walker). - non-page, page directories, which requires a custom table free. - Mostly IPI-based TLB invalidate, but it wouldn't be MIPS if it didn't also have broadcast TLB invalidate (I6500, GINVT). This means that in generic MIPS needs HAVE_RCU_TABLE_FREE. Cc: Paul Burton Signed-off-by: Peter Zijlstra (Intel) --- arch/mips/Kconfig | 1 + arch/mips/include/asm/pgalloc.h | 13 ++++++------- arch/mips/mm/pgtable.c | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 7 deletions(-) --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -57,6 +57,7 @@ config MIPS select HAVE_DYNAMIC_FTRACE select HAVE_EXIT_THREAD select HAVE_FAST_GUP + select MMU_GATHER_RCU_TABLE_FREE select HAVE_FTRACE_MCOUNT_RECORD select HAVE_FUNCTION_GRAPH_TRACER select HAVE_FUNCTION_TRACER --- a/arch/mips/include/asm/pgalloc.h +++ b/arch/mips/include/asm/pgalloc.h @@ -41,6 +41,9 @@ static inline void pud_populate(struct m } #endif +extern void __tlb_remove_table(void *table); +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx); + /* * Initialize a new pgd / pmd table with invalid pointers. */ @@ -52,11 +55,7 @@ static inline void pgd_free(struct mm_st free_pages((unsigned long)pgd, PGD_ORDER); } -#define __pte_free_tlb(tlb,pte,address) \ -do { \ - pgtable_pte_page_dtor(pte); \ - tlb_remove_page((tlb), pte); \ -} while (0) +#define __pte_free_tlb(tlb,pte,address) pgtable_free_tlb((tlb), (pte), 0) #ifndef __PAGETABLE_PMD_FOLDED @@ -75,7 +74,7 @@ static inline void pmd_free(struct mm_st free_pages((unsigned long)pmd, PMD_ORDER); } -#define __pmd_free_tlb(tlb, x, addr) pmd_free((tlb)->mm, x) +#define __pmd_free_tlb(tlb, x, addr) pgtable_free_tlb((tlb), (x), 1) #endif @@ -101,7 +100,7 @@ static inline void p4d_populate(struct m set_p4d(p4d, __p4d((unsigned long)pud)); } -#define __pud_free_tlb(tlb, x, addr) pud_free((tlb)->mm, x) +#define __pud_free_tlb(tlb, x, addr) pgtable_free_tlb((tlb), (x), 2) #endif /* __PAGETABLE_PUD_FOLDED */ --- a/arch/mips/mm/pgtable.c +++ b/arch/mips/mm/pgtable.c @@ -7,6 +7,7 @@ #include #include #include +#include pgd_t *pgd_alloc(struct mm_struct *mm) { @@ -23,3 +24,36 @@ pgd_t *pgd_alloc(struct mm_struct *mm) return ret; } EXPORT_SYMBOL_GPL(pgd_alloc); + +#define TLB_IDX_MASK 0x3 + +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx) +{ + unsigned long pgf = (unsigned long)table; + BUG_ON(idx > TLB_IDX_MASK); + BUG_ON(pgf & TLB_IDX_MASK); + pgf |= idx; + tlb_remove_table(tlb, (void *)pgf); +} + +void __tlb_remove_table(void *table) +{ + void *dir = (void *)((unsigned long)table & ~TLB_IDX_MASK); + int idx = (unsigned long)table & TLB_IDX_MASK; + + switch (idx) { +#ifndef __PAGETABLE_PUD_FOLDED + case 2: /* PUD */ + pud_free(NULL, dir); + break; +#endif +#ifndef __PAGETABLE_PMD_FOLDED + case 1: /* PMD */ + pmd_free(NULL, dir); + break; +#endif + case 0: /* PTE */ + pte_free(NULL, dir); + break; + } +} From patchwork Wed Dec 11 12:07:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E733E14E3 for ; Wed, 11 Dec 2019 12:31:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B3D532173E for ; Wed, 11 Dec 2019 12:31:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="hcORLxXG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3D532173E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3E3D66B31F8; Wed, 11 Dec 2019 07:31:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 31C7B6B31F9; Wed, 11 Dec 2019 07:31:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20B396B31FA; Wed, 11 Dec 2019 07:31:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id F0CF86B31F8 for ; Wed, 11 Dec 2019 07:31:30 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id AFC008249980 for ; Wed, 11 Dec 2019 12:31:30 +0000 (UTC) X-FDA: 76252796340.20.lamp47_475c346c76829 X-Spam-Summary: 2,0,0,ef72465d660b0c85,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4250:4385:4390:4395:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:21772:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-62.8 .55.100 X-HE-Tag: lamp47_475c346c76829 X-Filterd-Recvd-Size: 4367 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=G2PV3JD1hofFY1qgnR3BY+EQPfEM9HsXume4Dr7DRnk=; b=hcORLxXGPCck7A1YsNcz4BOVw5 VZ9YQWv9XZlSpLEEn9qepixFuPimXOy60XXWt3ripw7Xqv+I/N1ByNWua4F4JQBfkkD4d/7Ap3R9m P82/XSGQ0NUmbO/TKmrB78+YK/T7O7RA5T225bbnXot+ENok2WLlz6Go1QOoCNtLcaabPE8hJMV54 S7C0x9k6/oOcaS+2zOvwfFG7Fz5cVhHbJ99wS4IOFPouTCtCbM4X2QB3Cr5E9IZllo5upKTgx4hNE 3dBwNpWWD3IA+40d1UFETBBQUZ36Wf+/cHGQjHx5yn4Ljg/1SaM6ZTv0+OWVVX4N6yy2unAeTbjNe X94eo1fg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if191-0003sn-Fw; Wed, 11 Dec 2019 12:31:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id F3675307019; Wed, 11 Dec 2019 13:29:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 4F17E20137CB7; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.456987895@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:27 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 14/17] ia64/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since IA64 has page based P[UM]Ds, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/ia64/include/asm/pgalloc.h | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) --- a/arch/ia64/include/asm/pgalloc.h +++ b/arch/ia64/include/asm/pgalloc.h @@ -50,7 +50,9 @@ static inline void pud_free(struct mm_st { free_page((unsigned long)pud); } -#define __pud_free_tlb(tlb, pud, address) pud_free((tlb)->mm, pud) + +#define __pud_free_tlb(tlb, pud, address) \ + tlb_remove_table((tlb), virt_to_page(pud)) #endif /* CONFIG_PGTABLE_LEVELS == 4 */ static inline void @@ -69,7 +71,8 @@ static inline void pmd_free(struct mm_st free_page((unsigned long)pmd); } -#define __pmd_free_tlb(tlb, pmd, address) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, address) \ + tlb_remove_table((tlb), virt_to_page(pmd) static inline void pmd_populate(struct mm_struct *mm, pmd_t * pmd_entry, pgtable_t pte) @@ -84,6 +87,10 @@ pmd_populate_kernel(struct mm_struct *mm pmd_val(*pmd_entry) = __pa(pte); } -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) #endif /* _ASM_IA64_PGALLOC_H */ From patchwork Wed Dec 11 12:07:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6853D139A for ; Wed, 11 Dec 2019 12:31:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3471A214AF for ; Wed, 11 Dec 2019 12:31:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="cODSCc6k" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3471A214AF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3B9FA6B31EE; Wed, 11 Dec 2019 07:31:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 344BB6B31EF; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 196816B31F1; Wed, 11 Dec 2019 07:31:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 00CC46B31EF for ; Wed, 11 Dec 2019 07:31:20 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id AAA754428 for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) X-FDA: 76252795920.25.map17_45e10aed4180e X-Spam-Summary: 2,0,0,982a3d74f0c1a2cb,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4250:4385:4390:4395:5007:6119:6261:6653:6742:8634:10004:10400:11026:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:21772:30003:30054:30055,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net -62.8.0. X-HE-Tag: map17_45e10aed4180e X-Filterd-Recvd-Size: 3903 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=lqUXsp1YoP1pMIfyAAamvA0e1hiYKdF5cDUHBRasd7E=; b=cODSCc6khjn9lP4A4q98C7H8NG 1IjFybet2yMRA1nson2KvHNrOwtf8g9XdOTbDuvTeaGjj36PIV2VccJbEmWGm3kBpy8AbO16NCVK+ nmvK+YZf72EbclqQZsBDlwP/8S6KFxZRf/CnPRVTFWvTDjXe6TDp3Vkj24o+GOBVER+FybJk1OXwy C9Atb2i3FbxYRcrD92DJXEyWG+jxZiQe4xXQjYuhwZXHjYcqJmb2Hyu2LLx9/04Gn4bWQjUGk8M0m wjRi/vKqGOBQc+9xo6s3NUtKkFW501Yo9LbdQjX38chWOZuG0FXE1AXixVugf2r4shoxEPp6UR6xy mrxb31zw==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if192-0001Pe-BZ; Wed, 11 Dec 2019 12:31:08 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 0072530706D; Wed, 11 Dec 2019 13:29:44 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 559F920137CBC; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.514715737@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:28 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 15/17] alpha/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since Alpha has page based PMDs, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/alpha/include/asm/tlb.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) --- a/arch/alpha/include/asm/tlb.h +++ b/arch/alpha/include/asm/tlb.h @@ -4,7 +4,16 @@ #include -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) -#define __pmd_free_tlb(tlb, pmd, address) pmd_free((tlb)->mm, pmd) +extern void ___pte_free_tlb(struct mmu_gather *tlb, pte_t *pte); +extern void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd); + +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) + +#define __pmd_free_tlb(tlb, pmd, address) \ + tlb_remove_table((tlb), virt_to_page(pmd)) #endif From patchwork Wed Dec 11 12:07:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284833 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2E7C139A for ; Wed, 11 Dec 2019 12:31:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8FD5021556 for ; Wed, 11 Dec 2019 12:31:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ddpzUjpP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FD5021556 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7B39E6B31F1; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 73FA26B31F6; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3432A6B31F1; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id 13CDB6B31F2 for ; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id B0B938249980 for ; Wed, 11 Dec 2019 12:31:21 +0000 (UTC) X-FDA: 76252795962.08.kite83_460e5faca8c1c X-Spam-Summary: 2,0,0,c03c00e52ad38eee,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1534:1540:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3870:3871:3872:4250:4385:4390:4395:5007:6119:6261:6653:6742:8634:10004:10400:11026:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-62.8.0.100 64.201.201.201,Ca cheIP:no X-HE-Tag: kite83_460e5faca8c1c X-Filterd-Recvd-Size: 3656 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+ZJ1DOKZOq/0j+v4BO/2WlQxRCr7GEKCNwB8JR0YAik=; b=ddpzUjpPmxkTKtzj7fKv8rcz3A YKfmo8Y8IeDliNTqcxKrsXSnpR4uU9o1TKeMsoeXW9Uox7y7K+MNJUWtU8UgySEi4LSC0nLKV49IP 4pgdnDEVoF75K7Uzjbw9OOYDhN621bhspd5JqaMePUyJw0vpbEyS8zzbodbVWzcXeQYIFBw5RNIjd kIgmXThqHI/r6tdwzY2BuLg4UQpYBayEYi7/YDiFgCNNHrVEbtUggrYw6dCBRs1H8Wh+z3Nu3pV7K i71g0NWdh13A4Smypw5ftx0m1dSDmv+5/tq7/f6ccPg3BFr/PPElpf+QBLRiB7ZPXqh0sHoUpVF1n 8QwOuO0Q==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if193-0001Ph-2y; Wed, 11 Dec 2019 12:31:09 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 0271030719C; Wed, 11 Dec 2019 13:29:44 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 5AF6420137CBE; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.572325873@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:29 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 16/17] nds32/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Even though NDS32 is UP only, we still need to observe this order because mmu_gather is preemptible. NDS32 does not in fact have PMDs. Signed-off-by: Peter Zijlstra (Intel) --- arch/nds32/include/asm/tlb.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) --- a/arch/nds32/include/asm/tlb.h +++ b/arch/nds32/include/asm/tlb.h @@ -6,7 +6,12 @@ #include -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) + #define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tln)->mm, pmd) #endif From patchwork Wed Dec 11 12:07:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11284835 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68742138D for ; Wed, 11 Dec 2019 12:31:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2B4D32173E for ; Wed, 11 Dec 2019 12:31:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="aglqWHi1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B4D32173E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BD3236B31F3; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B58696B31F2; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 965C26B31F7; Wed, 11 Dec 2019 07:31:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0193.hostedemail.com [216.40.44.193]) by kanga.kvack.org (Postfix) with ESMTP id 5FC4C6B31F3 for ; Wed, 11 Dec 2019 07:31:22 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 16214180AD817 for ; Wed, 11 Dec 2019 12:31:22 +0000 (UTC) X-FDA: 76252796004.01.stove36_4618ab338d63a X-Spam-Summary: 2,0,0,15ae2d94c3ab23a6,d41d8cd98f00b204,peterz@infradead.org,:will@kernel.org:aneesh.kumar@linux.ibm.com:akpm@linux-foundation.org:npiggin@gmail.com:peterz@infradead.org:linux-arch@vger.kernel.org::linux-kernel@vger.kernel.org:ysato@users.sourceforge.jp:dalias@libc.org:davem@davemloft.net:deller@gmx.de:geert@linux-m68k.org:paulburton@kernel.org:tony.luck@intel.com:rth@twiddle.net:nickhu@andestech.com:paul.walmsley@sifive.com,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:1801:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:21772:30054:30055,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net -62.8.0. X-HE-Tag: stove36_4618ab338d63a X-Filterd-Recvd-Size: 3961 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 12:31:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-Id:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=3R2DK7HvOJXWBjGxheTnCh1+oyh0A3xwV1bV88LUreM=; b=aglqWHi18wn3lAH9Lx5D/B4kie lJrNLFF2pxctrEMnYBHJIEBPv58Uj2A/vOKGdU8ZDPpr8FRStCEj0aezmWfn31MarEVKYHnc5yWex o4z0fl0klPQ5ega9cW16Tu2P2HCSxYvHoFGZ/aWVIoOt9c2cHZ/FcPuq3kEZvabVEq+Eq+I6OrTPN dtIJQP6gNNjFP/E6MFesMAcqOR3T40Ryem7Xr9dYn2tgznxlX8JgvxiQghvD3NSRLSueGnFntUOfM DcFRhZWamjcLkkYGEgDd2rlXeZIGbLC0EEw5Fjo3t1fgIYGg3VCHsMJ6tJw05K20y5rLuhDQqO5CI wITu+cqA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1if192-0001Pf-Bl; Wed, 11 Dec 2019 12:31:08 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 026FB307093; Wed, 11 Dec 2019 13:29:44 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 605A920137CBF; Wed, 11 Dec 2019 13:31:02 +0100 (CET) Message-Id: <20191211122956.629995180@infradead.org> User-Agent: quilt/0.65 Date: Wed, 11 Dec 2019 13:07:30 +0100 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley Subject: [PATCH 17/17] riscv/tlb: Fix __p*_free_tlb() References: <20191211120713.360281197@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since RISC-V has page based PMDs, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/riscv/include/asm/pgalloc.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -73,14 +73,15 @@ static inline void pmd_free(struct mm_st free_page((unsigned long)pmd); } -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) \ + tlb_remove_table((tlb), virt_to_page(pmd)) #endif /* __PAGETABLE_PMD_FOLDED */ -#define __pte_free_tlb(tlb, pte, buf) \ -do { \ - pgtable_pte_page_dtor(pte); \ - tlb_remove_page((tlb), pte); \ +#define __pte_free_tlb(tlb, pte, buf) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ } while (0) #endif /* CONFIG_MMU */