From patchwork Fri Jul 17 11:10:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669909 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2D4B514E3 for ; Fri, 17 Jul 2020 11:14:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ED2B22076A for ; Fri, 17 Jul 2020 11:14:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="nyyWe7Al" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED2B22076A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2D4C08D0047; Fri, 17 Jul 2020 07:14:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2838C8D0040; Fri, 17 Jul 2020 07:14:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14BDF8D0047; Fri, 17 Jul 2020 07:14:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id EF2FA8D0040 for ; Fri, 17 Jul 2020 07:14:52 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id ADD338248047 for ; Fri, 17 Jul 2020 11:14:52 +0000 (UTC) X-FDA: 77047310424.10.jam89_151270226f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 1C6B15C4C9 for ; Fri, 17 Jul 2020 11:14:26 +0000 (UTC) X-Spam-Summary: 10,1,0,3803341a8631814b,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:69:152:355:379:404:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1978:1981:2194:2196:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3872:3874:4250:4385:4390:4395:5007:6261:6653:6742:8634:9592:10004:10400:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12679:12683:13161:13229:14096:14097:14110:14181:14394:14659:14721:21080:21451:21627:21772:21795:30051:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.55.100;04yr1kcnah9ssb5phm6nyef3k85x8ypxo74kgpkm1aubd1w6q5mm7bwe9wxxbt1.xt6bhkh6con6c1ub9784t147i8jstwjc6ts885stprtwpbxqg3odb5t3j7kpccn.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMA RY:none X-HE-Tag: jam89_151270226f0a X-Filterd-Recvd-Size: 5022 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=FUm1Ir3tUJJdB8qdXiVgRoT6pMrHP/Au+B6bLpG9yBA=; b=nyyWe7AlZtYdYSw49Sonwdxz2r KoQtTILVvXgJriLvasWk1Jh78FRgv6TwvxgW9OfTZnHOfpDdp0XRGhHkROr9M0rS8EE1ZWERqEZCb B4aLODSSNLN9BuP9VeeCsJRd1lCzDfSXB1o9kJN9xYbsbwUPyglN+NaEeeHEdZoR+6xv56usNBrAg E4M1ALbe2VwlUOwGw0TtgdLJDkOPtGYvMUB7f/nq3raSnihsty8lX4VKZGHH7eb7vSNthUAW4d0m3 ClYXcabVNTdqUwUgL5MxtmMU6aZva1lf7kQ0POZDK3bIv1gmVVtN3QMYW68wBEeJlpSJOR+/z2oJT H3PPqaIA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zl-Hw; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 1525F305E21; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id A9E9E203D408E; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.417688532@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:06 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 01/11] asm-generic/tlb: Fix MMU_GATHER_TABLE_FREE References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1C6B15C4C9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The first MMU_GATHER_TABLE_FREE user showed a logic error in the tlb_needs_table_invalidate() definition. Make sure any TABLE_FREE has it defined. Fixes: 0d6e24d430ef ("asm-generic/tlb: provide MMU_GATHER_TABLE_FREE") Signed-off-by: Peter Zijlstra (Intel) --- include/asm-generic/tlb.h | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-) --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -172,6 +172,18 @@ * various ptep_get_and_clear() functions. */ +#ifndef CONFIG_MMU_GATHER_RCU_TABLE_FREE + +/* + * Only RCU table free can override this; otherwise the TLBI is needed to + * provide existence guarantees for software walkers. + */ +#ifdef tlb_needs_table_invalidate +#error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE +#endif + +#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ + #ifdef CONFIG_MMU_GATHER_TABLE_FREE struct mmu_table_batch { @@ -187,17 +199,6 @@ struct mmu_table_batch { extern void tlb_remove_table(struct mmu_gather *tlb, void *table); -#else /* !CONFIG_MMU_GATHER_HAVE_TABLE_FREE */ - -/* - * Without MMU_GATHER_TABLE_FREE the architecture is assumed to have page based - * page directories and we can use the normal page batching to free them. - */ -#define tlb_remove_table(tlb, page) tlb_remove_page((tlb), (page)) - -#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ - -#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE /* * This allows an architecture that does not use the linux page-tables for * hardware to skip the TLBI when freeing page tables. @@ -206,13 +207,15 @@ extern void tlb_remove_table(struct mmu_ #define tlb_needs_table_invalidate() (true) #endif -#else +#else /* !CONFIG_MMU_GATHER_HAVE_TABLE_FREE */ -#ifdef tlb_needs_table_invalidate -#error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE -#endif +/* + * Without MMU_GATHER_TABLE_FREE the architecture is assumed to have page based + * page directories and we can use the normal page batching to free them. + */ +#define tlb_remove_table(tlb, page) tlb_remove_page((tlb), (page)) -#endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ +#endif /* CONFIG_MMU_GATHER_TABLE_FREE */ #ifndef CONFIG_MMU_GATHER_NO_GATHER From patchwork Fri Jul 17 11:10:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669999 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E6EBB1751 for ; Fri, 17 Jul 2020 11:35:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B191B20717 for ; Fri, 17 Jul 2020 11:35:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="GsN9f7zd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B191B20717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3DC4C8D004A; Fri, 17 Jul 2020 07:35:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3B4018D0040; Fri, 17 Jul 2020 07:35:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CA7F8D004A; Fri, 17 Jul 2020 07:35:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 191108D0040 for ; Fri, 17 Jul 2020 07:35:15 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id CFA831827058B for ; Fri, 17 Jul 2020 11:35:14 +0000 (UTC) X-FDA: 77047361748.29.shape51_050555e26f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id A3E5A1846CCD4 for ; Fri, 17 Jul 2020 11:35:14 +0000 (UTC) X-Spam-Summary: 1,0,0,0347ffa33ed17122,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3874:4250:4385:5007:6261:6653:6742:8634:9592:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:12679:13069:13221:13229:13311:13357:14181:14394:14659:14721:21080:21433:21451:21627:30003:30054:30055,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfta3c7osmku7wpugine9mpetiwop9qtndm36xaewkgfu195sir5fiko5kgxa.kdx4jn449gmdt8rdjarmtr17y79ropnkgsnky1py1bf16zrkkbhep6x4yp57ynj.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: shape51_050555e26f0a X-Filterd-Recvd-Size: 4178 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:35:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=JpsKOsZ9DAnC7hF8A6UR44iGc8j6T+HCUrwgJT3qna0=; b=GsN9f7zd3xGQm9E7Rs/9GO6Onv tQFOciWVNRa0dSkuBi2M3LCrvRWcBpajEzmYqEbvlBaXEJj5TVu8rwQy1Q7CNM7FZR48rVSTeBEGI rxqAK1jJtMJgtXRWRByIMPriLYyhgef4TVlkdIWBk/HEuGJcrF5ZqBi4hDqMeVize23cLJ0dbQ2Rr ip6Sj9RwHTkJWgq4VEYAlzp2yq6rkCH7ht7KXWI/JDmBlDwMD1hfEqR0twYDRHj0D9qMYMWXXgvJy 1co1QpT2PKFq+vwxgBdv2Cf4ZMf4aetp6NLBpyaXqUnapfRc5si+FOdHmr45RHoiibPH/LNSuP/jA R5Ykw5SA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJa-00051V-96; Fri, 17 Jul 2020 11:14:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 0D7ED300446; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id AC1F7203D4090; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.475459591@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:07 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig , Geert Uytterhoeven Subject: [PATCH v2 02/11] sh/tlb: Fix PGTABLE_LEVELS > 2 References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: A3E5A1846CCD4 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Geert reported that his SH7722-based Migo-R board failed to boot after commit: c5b27a889da9 ("sh/tlb: Convert SH to generic mmu_gather") That commit fell victim to copying the wrong pattern -- __pmd_free_tlb() used to be implemented with pmd_free(). Fixes: c5b27a889da9 ("sh/tlb: Convert SH to generic mmu_gather") Reported-by: Geert Uytterhoeven Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Geert Uytterhoeven Tested-by: Geert Uytterhoeven --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -12,6 +12,7 @@ extern void pgd_free(struct mm_struct *m extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); +#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, @@ -33,13 +34,4 @@ do { \ tlb_remove_page((tlb), (pte)); \ } while (0) -#if CONFIG_PGTABLE_LEVELS > 2 -#define __pmd_free_tlb(tlb, pmdp, addr) \ -do { \ - struct page *page = virt_to_page(pmdp); \ - pgtable_pmd_page_dtor(page); \ - tlb_remove_page((tlb), page); \ -} while (0); -#endif - #endif /* __ASM_SH_PGALLOC_H */ From patchwork Fri Jul 17 11:10:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669889 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0FC1C1392 for ; Fri, 17 Jul 2020 11:14:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D08802070E for ; Fri, 17 Jul 2020 11:14:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mW9BkUrd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D08802070E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B5F6C8D0030; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B12658D0036; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98BEC8D0030; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 6741E8D0037 for ; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2CC1C5C098 for ; Fri, 17 Jul 2020 11:14:26 +0000 (UTC) X-FDA: 77047309332.13.snow74_2a0e5d326f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id F2473184B4AF9 for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) X-Spam-Summary: 1,0,0,b5972979155d0ab3,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3872:3874:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:14096:14097:14181:14394:14659:14721:21080:21451:21627:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-62.8.55.100 64.201.201.201;04yfpgj1sdx1mu6ff5c9h5je9wdmgoc73u6mdu1fjj533oe45epuxyksgf61moi.c674fstjuw3uka3psdfyk1a615k5eond6xrb1eoeudhyddkx8kic9fo91tqxjdo.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: snow74_2a0e5d326f0a X-Filterd-Recvd-Size: 4850 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=LJ3sw53oU99bJUNlz+ZPKQESK/v2SDXFGx5kri1RxHA=; b=mW9BkUrdORbTIfPggDbuXHG04P vCB4+S08tm1Ocf2AwK2fgRL5Z3xoCetQtur/ZtwHqsPjZyWzsEWXg/Ip+JJ01RKTNXQtvbRs+p4wQ Rtlh3rxfbsLXEdDC4EA+JGYP4+cth5V1pYwo/CUI+uhfMF/FCH+FPxsGHa0Qxv98nUlMB2O5LfeGn 76uku3heKukpjwJsmr+QWhayyffHIapPntJuifnOtS8eftI1cLoiD8fwy79B12vBBVdWVK6nwNu9V c3dXF9QSaUXQEzzdrBQNbf7xX4kUygNN/Os21nFZoo+1Bzj9/2pKz1V+KcRM2zJ+FzRpvfRyHdXc7 T+DnKayA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zm-Jw; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 144D5305DD1; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id AF019203D4091; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.533637732@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:08 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 03/11] sh/tlb: Fix __pmd_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: F2473184B4AF9 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: SH violates the freeing order for it's PMD page directories. It's __pmd_free_tlb() does not ensure there is a TLB invalidation between itself and the eventualy freeing of the page. Further complicating the situation is that SH uses non page based allocation for it's PMDs. Use the shiny new HAVE_TABLE_FREE option to enable a custom page table freeer. (SuperH uses IPI based TLB invalidation and therefore doesn't need HAVE_RCU_TABLE_FREE for its HAVE_FAST_GUP). Signed-off-by: Peter Zijlstra (Intel) --- arch/sh/Kconfig | 1 + arch/sh/include/asm/pgalloc.h | 3 ++- arch/sh/mm/pgtable.c | 6 ++++++ 3 files changed, 9 insertions(+), 1 deletion(-) --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -14,6 +14,7 @@ config SUPERH select HAVE_PERF_EVENTS select HAVE_DEBUG_BUGVERBOSE select HAVE_FAST_GUP if MMU + select MMU_GATHER_TABLE_FREE if X2TLB select ARCH_HAVE_CUSTOM_GPIO_H select ARCH_HAVE_NMI_SAFE_CMPXCHG if (GUSA_RB || CPU_SH4A) select ARCH_HAS_GCOV_PROFILE_ALL --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -12,7 +12,8 @@ extern void pgd_free(struct mm_struct *m extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); -#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) +extern void __tlb_remove_table(void *table); +#define __pmd_free_tlb(tlb, pmdp, addr) tlb_remove_table((tlb), (pmdp)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, --- a/arch/sh/mm/pgtable.c +++ b/arch/sh/mm/pgtable.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #define PGALLOC_GFP GFP_KERNEL | __GFP_ZERO @@ -55,4 +56,9 @@ void pmd_free(struct mm_struct *mm, pmd_ { kmem_cache_free(pmd_cachep, pmd); } + +void __tlb_remove_table(void *table) +{ + pmd_free(NULL, table); +} #endif /* PAGETABLE_LEVELS > 2 */ From patchwork Fri Jul 17 11:10:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669913 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 140FD138C for ; Fri, 17 Jul 2020 11:15:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D4311207F5 for ; Fri, 17 Jul 2020 11:15:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="i1mn0vBQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4311207F5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 14C998D0048; Fri, 17 Jul 2020 07:15:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0FCEA8D0040; Fri, 17 Jul 2020 07:15:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2DA88D0048; Fri, 17 Jul 2020 07:15:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id DAD458D0040 for ; Fri, 17 Jul 2020 07:15:02 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9AD2D2CFFC for ; Fri, 17 Jul 2020 11:15:02 +0000 (UTC) X-FDA: 77047310844.05.veil06_370ff3626f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 680E218490854 for ; Fri, 17 Jul 2020 11:14:24 +0000 (UTC) X-Spam-Summary: 1,0,0,25d808685709c09c,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3354:3865:3867:3871:3872:4385:4390:4395:4605:5007:6119:6261:6653:6742:7903:8603:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:14181:14394:14659:14721:21080:21451:21627:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-62.8.55.100 64.201.201.201;04y89c1ac3quoo6txo73c4cuwx3buopicnejaedazn6pecrx9kb51yb46577qqd.mx8h685fjj51eyhaze6up184ajytg6pa9gqayk5tpdg35f1mxrz5oaea5omcd16.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: veil06_370ff3626f0a X-Filterd-Recvd-Size: 5454 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=fCoot8j0cWA9ciC5yvURdWvuq4otc5g+z9FwRrkMqPI=; b=i1mn0vBQKBaNacLXDfsoK+7QG6 g0aa0ADr+4bhdnmt9IAIw4EPzJLymoj2ob/Pya5sBqVBjtTrLs5vxWkeWzIq7rGnfwzBQO0abZ1Uj BuR8IUQ0Ml0ePeFrjIt+l0gsm+yIU4UoeY5S/I68dX4EeXb+RtLM/VFN4c/WB11lJSyBJ/1GN29F3 yGhK/yyH5IZPFpfHrT6WRUl9WY0rbD0MfUTCczycRrvJeXpzl0OT0EXN7kuPWC6KR61BCLR5PDDTJ 8OmSei+N/7sTQS1KghadHxTFf1CdjVubg7GAgPWtYzeluEq1AZ7yh7NnFu/4QSNEOEyQqJLXEsnwH dBXDZN6w==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJb-0001eW-7I; Fri, 17 Jul 2020 11:14:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 0E5B3304B90; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id B515C203D4093; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.591934380@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:09 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 04/11] sparc32/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 680E218490854 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Because Sparc32 has non-page based page directories, use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/sparc/Kconfig | 1 + arch/sparc/include/asm/pgalloc_32.h | 7 +++++-- arch/sparc/mm/srmmu.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 2 deletions(-) --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -57,6 +57,7 @@ config SPARC32 select CLZ_TAB select HAVE_UID16 select OLD_SIGACTION + select MMU_GATHER_TABLE_FREE config SPARC64 def_bool 64BIT --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -12,6 +12,9 @@ struct page; +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx); +extern void __tlb_remove_table(void *table); + void *srmmu_get_nocache(int size, int align); void srmmu_free_nocache(void *addr, int size); @@ -48,7 +51,7 @@ static inline void free_pmd_fast(pmd_t * } #define pmd_free(mm, pmd) free_pmd_fast(pmd) -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) pgtable_free_tlb((tlb), (pmd), 1) #define pmd_populate(mm, pmd, pte) pmd_set(pmd, pte) #define pmd_pgtable(pmd) (pgtable_t)__pmd_page(pmd) @@ -73,6 +76,6 @@ static inline void free_pte_fast(pte_t * #define pte_free_kernel(mm, pte) free_pte_fast(pte) void pte_free(struct mm_struct * mm, pgtable_t pte); -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, addr) pgtable_free_tlb((tlb), (pte), 0) #endif /* _SPARC_PGALLOC_H */ --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -38,6 +38,7 @@ #include #include #include +#include #include /* Now the cpu specific definitions. */ @@ -1831,3 +1832,30 @@ void __init load_mmu(void) sun4m_init_smp(); #endif } + +#define TLB_IDX_MASK 1UL + +void __tlb_remove_table(void *table) +{ + void *dir = (void *)((unsigned long)table & ~TLB_IDX_MASK); + int idx = (unsigned long)table & TLB_IDX_MASK; + + switch (idx) { + case 1: /* PMD */ + pmd_free(NULL, dir); + break; + case 0: /* PTE */ + pte_free(NULL, dir); + break; + } +} + +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx) +{ + unsigned long pgf = (unsigned long)table; + BUG_ON(idx > TLB_IDX_MASK); + BUG_ON(pgf & TLB_IDX_MASK); + pgf |= idx; + tlb_remove_table(tlb, (void *)pgf); +} + From patchwork Fri Jul 17 11:10:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669895 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C83D1392 for ; Fri, 17 Jul 2020 11:14:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3860820775 for ; Fri, 17 Jul 2020 11:14:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="a6jjRyXa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3860820775 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 808038D0038; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7655F8D0037; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 656568D0038; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id 393A88D0037 for ; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id F31EA3E540 for ; Fri, 17 Jul 2020 11:14:30 +0000 (UTC) X-FDA: 77047309542.13.ants49_180476626f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id BB33D184C0522 for ; Fri, 17 Jul 2020 11:14:30 +0000 (UTC) X-Spam-Summary: 1,0,0,4ba3d8a3db6e3cff,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3353:3865:3867:3868:3871:3872:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:7903:8603:8634:10004:10394:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:14181:14394:14659:14721:14877:21080:21451:21627:21789:21990:30012:30054:30055,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y84zejt66k3ci3wfxkr8x69fz5bocusdwx63arjeqaeo68aknba8wptc3by3e.imm8imconsb7ot5i6n7qqiqzjd799j1zr4pfns9bu8b7krjjumnoz7ymtryqr9y.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: ants49_180476626f0a X-Filterd-Recvd-Size: 5052 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=S8UknOxkHjh+fFZ6g8rERXF5PzCEDpVIPOf+v6XVy+E=; b=a6jjRyXa8+VsSAkTLZRRukYqZ6 sF/rHVHvFgmcHZ2kQMPhnZVqjNxs1t3wZduvAlgEeZGgwwjdyIUHL1XjV8vRf25apeRy8T0u0LBDL WBzFUF7pYu0sRdQU48yeiTjaXuGdmTVZ+CPQW/fszXLSP1O1C4kwFvK6FW6X+83ZOVVAumaU/qsE2 Z6fLJzH0bMeObAvJKwRGuKb1rn4oZeuKUjav1U2XCxVVdcYjh4wSShDvK7dQlYkNx2IOcY+q30M+P lir3rDJY5+14N+MhQ4r2ZdXZ8OdOFXwxGx/7flyn0th+WbXNKxKO7MjGmpiVAHad8XlBsgfFatTfB PUxtEjHg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-00051I-1P; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 6BB553060C5; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id BC664203D4094; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.650029395@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:10 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 05/11] parisc/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: BB33D184C0522 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Because PARISC has non-page-size PMDs, use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/parisc/Kconfig | 1 + arch/parisc/include/asm/pgalloc.h | 23 ++++++++++++++++++++--- arch/parisc/include/asm/tlb.h | 9 +++++---- 3 files changed, 26 insertions(+), 7 deletions(-) --- a/arch/parisc/Kconfig +++ b/arch/parisc/Kconfig @@ -63,6 +63,7 @@ config PARISC select HAVE_KPROBES_ON_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_COPY_THREAD_TLS + select MMU_GATHER_TABLE_FREE if PGTABLE_LEVELS >= 3 help The PA-RISC microprocessor is designed by Hewlett-Packard and used --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -73,7 +73,7 @@ static inline pmd_t *pmd_alloc_one(struc return pmd; } -static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +static inline bool __pmd_free(struct mm_struct *mm, pmd_t *pmd) { if (pmd_flag(*pmd) & PxD_FLAG_ATTACHED) { /* @@ -83,11 +83,28 @@ static inline void pmd_free(struct mm_st * done by generic mm code. */ mm_inc_nr_pmds(mm); - return; + return false; } - free_pages((unsigned long)pmd, PMD_ORDER); + return true; +} + +static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +{ + if (__pmd_free(mm, pmd)) + free_pages((unsigned long)pmd, PMD_ORDER); } +static inline void __tlb_remove_table(void *table) +{ + free_pages((unsigned long)table, PMD_ORDER); +} + +#define __pmd_free_tlb(tlb, pmd, addr) \ +do { \ + if (__pmd_free((tlb)->mm, (pmd))) \ + tlb_remove_table((tlb), (pmd)); \ +} while (0) + #endif static inline void --- a/arch/parisc/include/asm/tlb.h +++ b/arch/parisc/include/asm/tlb.h @@ -4,9 +4,10 @@ #include -#if CONFIG_PGTABLE_LEVELS == 3 -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) -#endif -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb,pte,addr) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_page((tlb), (pte)); \ +} while (0) #endif From patchwork Fri Jul 17 11:10:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669885 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CE121392 for ; Fri, 17 Jul 2020 11:14:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 088F8207E8 for ; Fri, 17 Jul 2020 11:14:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Y/MXDI0t" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 088F8207E8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1250D8D0034; Fri, 17 Jul 2020 07:14:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0AEFA8D0030; Fri, 17 Jul 2020 07:14:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB7898D0034; Fri, 17 Jul 2020 07:14:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0233.hostedemail.com [216.40.44.233]) by kanga.kvack.org (Postfix) with ESMTP id D09848D0030 for ; Fri, 17 Jul 2020 07:14:24 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 861E1180EAE3F for ; Fri, 17 Jul 2020 11:14:24 +0000 (UTC) X-FDA: 77047309248.30.wish22_09162f926f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 5D6DE181B7E63 for ; Fri, 17 Jul 2020 11:14:24 +0000 (UTC) X-Spam-Summary: 1,0,0,cd9ca5d2b1d3cec8,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3355:3865:3867:3870:3871:3872:3874:4117:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:7875:7903:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12683:14181:14394:14659:14721:14877:21080:21450:21451:21627:21772:21990:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.55.100;04y8ih5s9rcqa8m3xciqkebroo9swycj85m53wqr653tnwoae5twyaxi3f4jwga.45yatbotwptbohxc5hu8o41t9we569orwpgqojxyp1sboohbdgtua6ywuy6ozdz.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23, LUA_SUMM X-HE-Tag: wish22_09162f926f0a X-Filterd-Recvd-Size: 6341 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=gOQYLPqlkXaE5f1BcTQenQfqzoyHPAVuIDnigWhSoRE=; b=Y/MXDI0tbjGV1H6k+/3UUA8Lu5 zcG4hZO3jyz9bLcRi2dIJysN79dO3dnN/J6fbvIK4gxxQiyBlOKQuuCn0E4ddYCbGpPOTo3VVtlki b8indkgQ/FVRbEsFBMBPCZcy3KunV3UanWjKfz3yzA2GvqjF7q6418pyW6NM0aW8XTHtuYN9wNG74 sxLzg+HiCTzpnr/zc2fq3XNvfm31YDJdhEXKopYgPN6tBPKbu/nVWCyNI6khi3A9VUpNMTP2aF7eI 3elFTvXbPIuOkzZ1J7mS4tlF7nVaUtIA8IAJ8rLPZMHlzNolSp/mIGsc5xtO6SOhkwzpM6jWmaDOb UHPdiS5w==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zn-LT; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 6F914306CCF; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C0DDF203D4095; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.708289813@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:11 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 06/11] mips/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5D6DE181B7E63 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. MIPS has (thanks to Paul Burton for setting me straight): - HAVE_FAST_GUP, which means we need to consider software walkers (many MIPS also have software TLB-fill, which is a similar software walker). - non-page, page directories, which requires a custom table free. - Mostly IPI-based TLB invalidate, but it wouldn't be MIPS if it didn't also have broadcast TLB invalidate (I6500, GINVT). This means that in generic MIPS needs HAVE_RCU_TABLE_FREE. Signed-off-by: Peter Zijlstra (Intel) --- arch/mips/Kconfig | 1 + arch/mips/include/asm/pgalloc.h | 13 ++++++------- arch/mips/mm/pgtable.c | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 7 deletions(-) --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -84,6 +84,7 @@ config MIPS select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP select IRQ_FORCED_THREADING select ISA if EISA + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_REL if MODULES select MODULES_USE_ELF_RELA if MODULES && 64BIT select PERF_USE_VMALLOC --- a/arch/mips/include/asm/pgalloc.h +++ b/arch/mips/include/asm/pgalloc.h @@ -41,6 +41,9 @@ static inline void pud_populate(struct m } #endif +extern void __tlb_remove_table(void *table); +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx); + /* * Initialize a new pgd / pmd table with invalid pointers. */ @@ -52,11 +55,7 @@ static inline void pgd_free(struct mm_st free_pages((unsigned long)pgd, PGD_ORDER); } -#define __pte_free_tlb(tlb,pte,address) \ -do { \ - pgtable_pte_page_dtor(pte); \ - tlb_remove_page((tlb), pte); \ -} while (0) +#define __pte_free_tlb(tlb,pte,address) pgtable_free_tlb((tlb), (pte), 0) #ifndef __PAGETABLE_PMD_FOLDED @@ -75,7 +74,7 @@ static inline void pmd_free(struct mm_st free_pages((unsigned long)pmd, PMD_ORDER); } -#define __pmd_free_tlb(tlb, x, addr) pmd_free((tlb)->mm, x) +#define __pmd_free_tlb(tlb, x, addr) pgtable_free_tlb((tlb), (x), 1) #endif @@ -101,7 +100,7 @@ static inline void p4d_populate(struct m set_p4d(p4d, __p4d((unsigned long)pud)); } -#define __pud_free_tlb(tlb, x, addr) pud_free((tlb)->mm, x) +#define __pud_free_tlb(tlb, x, addr) pgtable_free_tlb((tlb), (x), 2) #endif /* __PAGETABLE_PUD_FOLDED */ --- a/arch/mips/mm/pgtable.c +++ b/arch/mips/mm/pgtable.c @@ -7,6 +7,7 @@ #include #include #include +#include pgd_t *pgd_alloc(struct mm_struct *mm) { @@ -23,3 +24,36 @@ pgd_t *pgd_alloc(struct mm_struct *mm) return ret; } EXPORT_SYMBOL_GPL(pgd_alloc); + +#define TLB_IDX_MASK 0x3 + +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int idx) +{ + unsigned long pgf = (unsigned long)table; + BUG_ON(idx > TLB_IDX_MASK); + BUG_ON(pgf & TLB_IDX_MASK); + pgf |= idx; + tlb_remove_table(tlb, (void *)pgf); +} + +void __tlb_remove_table(void *table) +{ + void *dir = (void *)((unsigned long)table & ~TLB_IDX_MASK); + int idx = (unsigned long)table & TLB_IDX_MASK; + + switch (idx) { +#ifndef __PAGETABLE_PUD_FOLDED + case 2: /* PUD */ + pud_free(NULL, dir); + break; +#endif +#ifndef __PAGETABLE_PMD_FOLDED + case 1: /* PMD */ + pmd_free(NULL, dir); + break; +#endif + case 0: /* PTE */ + pte_free(NULL, dir); + break; + } +} From patchwork Fri Jul 17 11:10:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669883 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF84A14E3 for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9171A2070E for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ka83rRYo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9171A2070E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A25BE8D0033; Fri, 17 Jul 2020 07:14:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9D5968D0030; Fri, 17 Jul 2020 07:14:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 89CDC8D0033; Fri, 17 Jul 2020 07:14:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id 71D2C8D0030 for ; Fri, 17 Jul 2020 07:14:24 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 2C8D4184799E5 for ; Fri, 17 Jul 2020 11:14:24 +0000 (UTC) X-FDA: 77047309248.27.humor87_32135e726f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id CD0452BCB7 for ; Fri, 17 Jul 2020 11:14:23 +0000 (UTC) X-Spam-Summary: 1,0,0,ef72465d660b0c85,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4250:4385:4390:4395:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:21772:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.55.100;04yfna5i3ip5m88ujr3gfm9nxx7nfyptc5tarafbi95gdhmq3cq5hqb8q43kzbd.b6zkqkspxc7jfy5qzeo91kg5phhqhk7rgbj7t1b3o1h8rdfx378f8fya9cyaqey.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: humor87_32135e726f0a X-Filterd-Recvd-Size: 4328 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=G2PV3JD1hofFY1qgnR3BY+EQPfEM9HsXume4Dr7DRnk=; b=Ka83rRYoEtlhZlelqXZ2reoVNG c3psqE8LlKgY5/dxe8pHVAsn5xsC/K10d0dms9/MK3//sK20Xjwt/qTRwMPKptZyaoM81YRhrOOyP aPK2g/aIBYbfai8KbUpc7keOXMTTHznTBmSyA3RzslV20ge7d4zbgDncUMgNPU1/0KeXMsTiUzwFP GM4AAkK6f4JzqywkFCYyQ3YtScmPM34DM2DMpmTJPuElir8eUzZt6X5kKc2kReIUxv5+Rx+nDvvuU DhZTZbgf87SagTOmVsHLiZ5AGaCSE9R5k6ITPOprV8aaSl4WiMdbTHimgTsh/lAoEISmsPFyKW9Cz 4ck418bQ==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJb-0001da-1P; Fri, 17 Jul 2020 11:14:07 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 7A3E2306E07; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C4D5B203D4096; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.766472738@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:12 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 07/11] ia64/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: CD0452BCB7 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since IA64 has page based P[UM]Ds, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/ia64/include/asm/pgalloc.h | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) --- a/arch/ia64/include/asm/pgalloc.h +++ b/arch/ia64/include/asm/pgalloc.h @@ -50,7 +50,9 @@ static inline void pud_free(struct mm_st { free_page((unsigned long)pud); } -#define __pud_free_tlb(tlb, pud, address) pud_free((tlb)->mm, pud) + +#define __pud_free_tlb(tlb, pud, address) \ + tlb_remove_table((tlb), virt_to_page(pud)) #endif /* CONFIG_PGTABLE_LEVELS == 4 */ static inline void @@ -69,7 +71,8 @@ static inline void pmd_free(struct mm_st free_page((unsigned long)pmd); } -#define __pmd_free_tlb(tlb, pmd, address) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, address) \ + tlb_remove_table((tlb), virt_to_page(pmd) static inline void pmd_populate(struct mm_struct *mm, pmd_t * pmd_entry, pgtable_t pte) @@ -84,6 +87,10 @@ pmd_populate_kernel(struct mm_struct *mm pmd_val(*pmd_entry) = __pa(pte); } -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) #endif /* _ASM_IA64_PGALLOC_H */ From patchwork Fri Jul 17 11:10:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669907 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43F0A1392 for ; Fri, 17 Jul 2020 11:14:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 067CB2076A for ; Fri, 17 Jul 2020 11:14:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DJYHEvab" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 067CB2076A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F31558D0041; Fri, 17 Jul 2020 07:14:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EDF0B8D0040; Fri, 17 Jul 2020 07:14:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD0238D0041; Fri, 17 Jul 2020 07:14:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id C47A08D0040 for ; Fri, 17 Jul 2020 07:14:42 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 851AA1848B0BE for ; Fri, 17 Jul 2020 11:14:42 +0000 (UTC) X-FDA: 77047310004.28.coast48_430229126f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id ECB7C56778 for ; Fri, 17 Jul 2020 11:14:23 +0000 (UTC) X-Spam-Summary: 1,0,0,982a3d74f0c1a2cb,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4250:4385:4390:4395:5007:6119:6261:6653:6742:8634:10004:10400:11026:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:21772:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.55.100;04y8g11kdfh5p6tref8cudq6qez57op7qc59zypma6dsgpdyxizkti9njsxz7i8.rfmnpuork3yxnj5ixsrw8jbyz5qbqt643i1esgenay3tyohjcbu94s3oaa7eydn.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: coast48_430229126f0a X-Filterd-Recvd-Size: 3855 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=lqUXsp1YoP1pMIfyAAamvA0e1hiYKdF5cDUHBRasd7E=; b=DJYHEvab2ej+if7sHNrHhPkAHQ dtX/b3zsSgVYJU7V5Dn3E9vuGr7oZGGGUJyrkLESLeIdfken09z/+Xmeoodj8j7J9Uk2bFnWw7QqD ED2Gj3xnfSJLH0AuDO2Dwk+yAsN1WzjUR+wGs7/dI39ZwLfCcluJFWHLyvtNIu04nk9BpMHOUh4qn Jmhq0K3zEMjJPwDa8SJxeMCF2hez8A5+YgdfCwXMhG+Dzv3FYYDbde19J4LCbtVi2XDdYJJi5rnr0 0QwwAPp770wRZZV9tISe/AAnO/4Sa2F6Q7Lm1yCiPKJzjQAPXpuMXMa+hL4mcj2yPlg+0sc+kKuKj 88FCPyug==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJa-0001b0-H7; Fri, 17 Jul 2020 11:14:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 82F35307488; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C89A9203D4092; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.825172312@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:13 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 08/11] alpha/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: ECB7C56778 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since Alpha has page based PMDs, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/alpha/include/asm/tlb.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) --- a/arch/alpha/include/asm/tlb.h +++ b/arch/alpha/include/asm/tlb.h @@ -4,7 +4,16 @@ #include -#define __pte_free_tlb(tlb, pte, address) pte_free((tlb)->mm, pte) -#define __pmd_free_tlb(tlb, pmd, address) pmd_free((tlb)->mm, pmd) +extern void ___pte_free_tlb(struct mmu_gather *tlb, pte_t *pte); +extern void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd); + +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) + +#define __pmd_free_tlb(tlb, pmd, address) \ + tlb_remove_table((tlb), virt_to_page(pmd)) #endif From patchwork Fri Jul 17 11:10:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669891 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 410B414E3 for ; Fri, 17 Jul 2020 11:14:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0306D2070E for ; Fri, 17 Jul 2020 11:14:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="yjSqa+5f" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0306D2070E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E99A08D0036; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DAFCB8D0038; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A72AE8D0037; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 61B038D0036 for ; Fri, 17 Jul 2020 07:14:26 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1624A1826F617 for ; Fri, 17 Jul 2020 11:14:26 +0000 (UTC) X-FDA: 77047309332.20.mom25_3d078c126f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id DCF0E181BE98B for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) X-Spam-Summary: 1,0,0,0a1635b3331cd4de,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1431:1437:1516:1518:1534:1540:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3870:3871:3872:4250:4385:4390:4395:5007:6119:6261:6653:6742:8634:10004:10400:11026:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:30003:30054:30055,0,RBL:205.233.59.134:@infradead.org:.lbl8.mailshell.net-62.8.55.100 64.201.201.201;04y8t6nzc4ogpsbdwrd4e69twkmjzopyr7tpfxmp4ew7yxep9ygf6x1anupmdip.csby15uzisrq7y8tx9jq1htmhm6epgkfby68puhp93tzcrmfooe8o6n7m99wfy8.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: mom25_3d078c126f0a X-Filterd-Recvd-Size: 3542 Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=mXdH0EdEU6Cy2G+amp6Zc6DclrxoI0Xq6oO6KLBpmqE=; b=yjSqa+5fwh6Wy8j+pNJsjC/qU1 B2W55tXrcly0UqYSpaENQEh4QFIfSAwqFfE+vAJgyC6OLtPX9la8lpviT1MifASoGfPTlDnAlsFvJ 1c7lO2spg8rO5c925FNqxh1Z9SaTqtPcBMJ2TFskFB3abEcjcuuaM3ZvTLwzF6SNJJP0jWPMprbG/ wDNzdmdXu8XExJClvzvqC7btCKV0ixVINJH7odcI9gFbt8hDwfQyEzNMiQmv+yNKd/h1tTAO/7mmI DdVsystUlYxe9pki8bthn8jpHCIaw8jgSwxbdgw4wr0T2AMqTmkLQ82Lfkon8Q/Fl65+OKpC0AK1a mraN58Yg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by merlin.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-0001Zo-Nd; Fri, 17 Jul 2020 11:14:05 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 82068306FFE; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id CBE5F203D4097; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.883694582@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:14 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 09/11] nds32/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: DCF0E181BE98B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Even though NDS32 is UP only, we still need to observe this order because mmu_gather is preemptible. NDS32 does not in fact have PMDs. Signed-off-by: Peter Zijlstra (Intel) --- arch/nds32/include/asm/tlb.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) --- a/arch/nds32/include/asm/tlb.h +++ b/arch/nds32/include/asm/tlb.h @@ -6,6 +6,11 @@ #include -#define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb(tlb, pte, address) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ +} while (0) + #endif From patchwork Fri Jul 17 11:10:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669897 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47BBF14E3 for ; Fri, 17 Jul 2020 11:14:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1429C20768 for ; Fri, 17 Jul 2020 11:14:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="FAUumSd9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1429C20768 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EBBE48D0037; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E6AA68D0040; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE9AC8D0037; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0035.hostedemail.com [216.40.44.35]) by kanga.kvack.org (Postfix) with ESMTP id 931E88D003B for ; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 53C7D184C0757 for ; Fri, 17 Jul 2020 11:14:31 +0000 (UTC) X-FDA: 77047309542.21.crook68_191310d26f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 25738184C0750 for ; Fri, 17 Jul 2020 11:14:31 +0000 (UTC) X-Spam-Summary: 1,0,0,15ae2d94c3ab23a6,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1042:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:1801:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:8634:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13069:13311:13357:14181:14394:14659:14721:21080:21451:21627:21772:30054:30055,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y89k61fr141ubmyjnsuu98bkk3xyp3oxw471nftekgzogk4g9791qjs5mdj17.czkff6a85f87tubs3wta7rystu7pqx5gcyzk4wh89hgeok8jhm7te4weurbyfd3.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: crook68_191310d26f0a X-Filterd-Recvd-Size: 3911 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=3R2DK7HvOJXWBjGxheTnCh1+oyh0A3xwV1bV88LUreM=; b=FAUumSd9I3/VNHUgNXx4k0r+oK XPsBvhXBs2S9D8KdP1DdzCn+DjvPQ/N53rubfaZdx/Zq28lfMnE4FWFrXxchpt7Fbf1H73vWibr4j kfeW2q7hWNzjn/xQ6xAH8dq5lajcLheqYNu/je7J2zCrVzuEsYRRAFG8On6JhRJVcrbLv0gwFgUug bT0FH7mxmojgXV6jLLYfJmaqGbBGG8nQCUBThUvi3fkTD9RIO3Clg7X9ZI3AaI+BL+SV+CRdZCc+o BsKMmuBNmewZ0MHaEgdFKq/FNkMMPDtazHdq2QEWx7CFn5z/oZZa+NB8HoSkX62V2Q62o6751m0i0 7KMjrc4A==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJa-00051T-8Y; Fri, 17 Jul 2020 11:14:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 81233306E3A; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id D05A1203D4099; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111349.942447764@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:15 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 10/11] riscv/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 25738184C0750 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. Since RISC-V has page based PMDs, no software walkers and IPI based TLB invalidation, it can use the simple tlb_remove_page() based freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/riscv/include/asm/pgalloc.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -73,14 +73,15 @@ static inline void pmd_free(struct mm_st free_page((unsigned long)pmd); } -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) \ + tlb_remove_table((tlb), virt_to_page(pmd)) #endif /* __PAGETABLE_PMD_FOLDED */ -#define __pte_free_tlb(tlb, pte, buf) \ -do { \ - pgtable_pte_page_dtor(pte); \ - tlb_remove_page((tlb), pte); \ +#define __pte_free_tlb(tlb, pte, buf) \ +do { \ + pgtable_pte_page_dtor(pte); \ + tlb_remove_table((tlb), (pte)); \ } while (0) #endif /* CONFIG_MMU */ From patchwork Fri Jul 17 11:10:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 11669901 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EEF51392 for ; Fri, 17 Jul 2020 11:14:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D5BE92076A for ; Fri, 17 Jul 2020 11:14:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="tp/BH55Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D5BE92076A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 31E988D003B; Fri, 17 Jul 2020 07:14:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0DF4B8D0041; Fri, 17 Jul 2020 07:14:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF8AB8D003B; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id 95C438D0040 for ; Fri, 17 Jul 2020 07:14:31 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 494BA4E7F9 for ; Fri, 17 Jul 2020 11:14:31 +0000 (UTC) X-FDA: 77047309542.29.page12_250635e26f0a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 1BA0E184B56C4 for ; Fri, 17 Jul 2020 11:14:31 +0000 (UTC) X-Spam-Summary: 1,0,0,b1b90c0e845381e3,d41d8cd98f00b204,peterz@infradead.org,,RULES_HIT:41:152:355:379:800:960:965:966:968:973:988:989:1183:1260:1261:1277:1313:1314:1345:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:1801:2194:2196:2198:2199:2200:2201:2393:2559:2562:2731:2903:3138:3139:3140:3141:3142:3354:3865:3866:3867:3870:3871:3872:4117:4250:4385:4390:4395:4605:5007:6119:6261:6653:6742:7903:8603:8634:9036:9592:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:13161:13229:14181:14394:14659:14721:21080:21451:21627:21990:30003:30054:30055,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yg5nokxc8xk78afxubq4xbo6hijopcm58yqssd39d8xi9f53o46d6dzxyatkn.89y7xdo66afy9tzzrxoi73hcsjia4u94ubtq7jqsp9c1jbnur5ok1a47jj3kojc.4-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: page12_250635e26f0a X-Filterd-Recvd-Size: 6079 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Jul 2020 11:14:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=nUUfhERBps7kvabI82RVV43naTK71jo6AAVltGUbW/s=; b=tp/BH55Z/QaPl4II/4kpXNcQdv XDl6UXJmW/vuilqUj195qptQ8vI7Xjlkz+6fvZRtbA3aih5li6rf3HtArbX12fyCj3oHtV5B5n4x9 pcc8tGMTIGSttnHzrRfWW0GvzPzKr+OYzu71OrB+gClwoAs8q2snBYoiufgmW5vPb/O0n7oS/VgLp 1NhQGSuISW5KYKcEhffZD+fJUGtlP066i0gESEXqxUnAkm4l8RznaZm0krzFJi3a2czVK2nIuw55x OboZwga/+8b43D0SyvxLOGCPFwZjTAjdlw36TvLSKt/cw886FL9tjZ2hH7WNnF1cGZeJlxfAKWDZY wt8k6hUA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jwOJZ-00051O-Og; Fri, 17 Jul 2020 11:14:06 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 86B813074D4; Fri, 17 Jul 2020 13:14:04 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id D4590203D409A; Fri, 17 Jul 2020 13:14:02 +0200 (CEST) Message-ID: <20200717111350.001250805@infradead.org> User-Agent: quilt/0.66 Date: Fri, 17 Jul 2020 13:10:16 +0200 From: Peter Zijlstra To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-arch@vger.kernel.org, linux-sh@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yoshinori Sato , Rich Felker , "David S. Miller" , Helge Deller , Geert Uytterhoeven , Paul Burton , Tony Luck , Richard Henderson , Nick Hu , Paul Walmsley , John Paul Adrian Glaubitz , Christoph Hellwig Subject: [PATCH v2 11/11] m68k/tlb: Fix __p*_free_tlb() References: <20200717111005.024867618@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1BA0E184B56C4 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Just like regular pages, page directories need to observe the following order: 1) unhook 2) TLB invalidate 3) free to ensure it is safe against concurrent accesses. The Motorola MMU has funny PMD stuff, so use a custom table freeer. Signed-off-by: Peter Zijlstra (Intel) --- arch/m68k/Kconfig | 1 + arch/m68k/include/asm/mcf_pgalloc.h | 14 ++++++-------- arch/m68k/include/asm/motorola_pgalloc.h | 10 ++++++---- arch/m68k/mm/motorola.c | 17 +++++++++++++++++ 4 files changed, 30 insertions(+), 12 deletions(-) --- a/arch/m68k/Kconfig +++ b/arch/m68k/Kconfig @@ -32,6 +32,7 @@ config M68K select OLD_SIGSUSPEND3 select OLD_SIGACTION select MMU_GATHER_NO_RANGE if MMU + select MMU_GATHER_TABLE_FREE if MMU_MOTOROLA config CPU_BIG_ENDIAN def_bool y --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -34,14 +34,12 @@ extern inline pmd_t *pmd_alloc_kernel(pg #define pmd_pgtable(pmd) pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT) -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, - unsigned long address) -{ - struct page *page = virt_to_page(pgtable); - - pgtable_pte_page_dtor(page); - __free_page(page); -} +#define __pte_free_tlb(tlb, pgtable, addr) \ +do { \ + struct page *page = virt_to_page(pgtable); \ + pgtable_pte_page_dtor(page); \ + tlb_remove_page((tlb), page); \ +} while (0) static inline pgtable_t pte_alloc_one(struct mm_struct *mm) { --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -14,6 +14,8 @@ enum m68k_table_types { TABLE_PTE = 1, }; +extern void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int type); +extern void __tlb_remove_table(void *table); extern void init_pointer_table(void *table, int type); extern void *get_pointer_table(int type); extern int free_pointer_table(void *table, int type); @@ -47,7 +49,7 @@ static inline void pte_free(struct mm_st static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, unsigned long address) { - free_pointer_table(pgtable, TABLE_PTE); + pgtable_free_tlb(tlb, pgtable, TABLE_PTE); } @@ -61,10 +63,10 @@ static inline int pmd_free(struct mm_str return free_pointer_table(pmd, TABLE_PMD); } -static inline int __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, - unsigned long address) +static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, + unsigned long address) { - return free_pointer_table(pmd, TABLE_PMD); + pgtable_free_tlb(tlb, pmd, TABLE_PMD); } --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -215,6 +215,23 @@ int free_pointer_table(void *table, int return 0; } +void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int type) +{ + unsigned long ptr = (unsigned long)table; + BUG_ON(type > 1); + BUG_ON(ptr & 1); + ptr |= type; + tlb_remove_table(tlb, (void *)ptr); +} + +void __tlb_remove_table(void *table) +{ + int type = (unsigned long)table & 1; + table = (void *)((unsigned long)table & ~1); + + free_pointer_table(table, type); +} + /* size of memory already mapped in head.S */ extern __initdata unsigned long m68k_init_mapped_size;