From patchwork Fri May 17 19:00:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13667321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B4B2C04FFE for ; Fri, 17 May 2024 19:01:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DA216B00A9; Fri, 17 May 2024 15:01:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9872C6B00AA; Fri, 17 May 2024 15:01:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DC536B00AB; Fri, 17 May 2024 15:01:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 566DF6B00A9 for ; Fri, 17 May 2024 15:01:03 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0D2D74035F for ; Fri, 17 May 2024 19:01:03 +0000 (UTC) X-FDA: 82128805206.22.A8C8374 Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by imf12.hostedemail.com (Postfix) with ESMTP id B300140024 for ; Fri, 17 May 2024 19:01:00 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=csgroup.eu; spf=pass (imf12.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715972461; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ipgkHbh4Zl6A+a7EP2bIIjgdG0nAhq0HZzTx8jhu+7w=; b=bC1hRVK1lixsm37MgvlxkWWm/RptQLYAc7x4tgjze/cr+bmIS4gGU1Zz31C1zD2fH/oAJZ 6lWH1hYqR3/qFqMzbKCtPvKLbhQytljV9KoCw99jpo3vcvrTrmZiO42tulqtjOEto7GH8Q FeIMOjrOOSEu3Dmh2lfjUBlfOyrMuUo= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=csgroup.eu; spf=pass (imf12.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715972461; a=rsa-sha256; cv=none; b=qauHpasd8HkU4rPOVCCSwQ7UUEeO4A9iH5ZS7PfoVE/MO5b0i/1XurWdDdVHEAkeVw1Dw/ v8aFiayPuVkM9iGCLNHhvzPYWZ3f9WiYDvxjC82GgnTSgr25wlmrJZxkVg/0UdBEC5ENhE D4/P1SvYTmy+JLYDRtIHiM0LIGZxHfc= Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4Vgx9Y1CsWz9t0K; Fri, 17 May 2024 21:00:17 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id McQAJbbFVMk3; Fri, 17 May 2024 21:00:17 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4Vgx9Q4FYTz9sv8; Fri, 17 May 2024 21:00:10 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 8D7548B783; Fri, 17 May 2024 21:00:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id iDr09WJAIIlY; Fri, 17 May 2024 21:00:10 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [192.168.232.121]) by messagerie.si.c-s.fr (Postfix) with ESMTP id E90988B766; Fri, 17 May 2024 21:00:09 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Jason Gunthorpe , Peter Xu , Oscar Salvador , Michael Ellerman , Nicholas Piggin Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH v2 13/20] powerpc/64e: Clean up impossible setups Date: Fri, 17 May 2024 21:00:07 +0200 Message-ID: <7faaea1bf493a12279e8d1afe298001fe3e83a71.1715971869.git.christophe.leroy@csgroup.eu> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1715972397; l=15296; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=JrGONtEYOGeyYF+t+7i2bB1sKhE1BMv6L8QUuELyR1s=; b=c/KyaHTdIpAEaEJCWKFMKA97Qv63wQj4qWxfEBXUJ85Y3lLPA59XZ/hGpSvT3b1xB4msou6R+ 5pXs+/lu1r3B6GgkhD9IAUtazL8lj3smkEs16JigGJUGnDDui5yJOqV X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Rspamd-Queue-Id: B300140024 X-Stat-Signature: b9m3fs3ooxwrdfkeg93f9zeu4n116pf9 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1715972460-922464 X-HE-Meta: U2FsdGVkX1/1XHs4/cl8QIZ9bdyhVsqxnqQDZEbHPwIuJTwHmC9e1w7pd1D6sSmxIzuoHZPxZcreKeEM8k66yGx41rF1ySeBiebefjIPGeS43Oh45Ynv+9NEDukmhGYGVHZZaCXsFegon5KT+qHijV6d5didF3UKw1sKmmDGn6tvUqVvK7LWjITnw5snjiN4BQ3y4wXPLeNlJK/O3ToOApB3CIXPaX9wgNBmthHSELhwZA+HaEyvuYeYHipigr42Cx3N3KD9LzUgRCRg6WgzRpJ2/v7Fmb7h0mBkUIah4z6O0CYMdnxAQJasBRV2ApSCU0NafQI6t3zsOUrEDJevsWMws0jMEyMNbfl8sU/xi0d/K+vhlhxK0I1Yh31Puc3ke/fEHsVkpUq0OkmIyVG+TPOlWrj6uHNPY6KEErbJPT2fYXljui/FvIlQ7EvMiusKrlS2WC+J8J5gqialWBgkqyaXzfNr3Tkul/X6MkbQFXRAhwDDGDZYyvxbggh5TSe3ZoWARzT5sP+wSLKgmvPe25OD0zarbWR9XAU1k73mioiTKAYiERfPzYmaISws0kZEdP55XTFVfz7IYKQW+SdZD1+80Vt2tkDgwQUByYViSsmVUTZoPhseeRfWM/K6QZsw/BI7dmqXsTGpjECy6cRS5R6Mlll0LPcKmiwL0N5tbnUW6548p8YUia3JS+fXfZaeDYXpZCRh4WppOqODuRUlmxOb2CS+dwlKszyrEQBz6mLEDsZ9T+ChRXPGlZg4eneG0ZS71bPUynFdtn50bsQOF4j61NqsCWG8buGPgDCkhFXFul7ggwTiTPqPzPuu7rDwWu4D8iMwpmlloe4hQi1Cv8LG7TGbwfl3l8bjfWrGsrLsHXXHa+IMNkr+afDnYWR3pnqLcOlAZ0D/3NVQDTnSlFeUyUFsLzp98EH7VYTMWjKM1kvY0b54uYE/jAWSGeiy3aVca5NyWTPF++UL/+t qAzBnN8A kqNyq+Jj8pTghPrWQVHa5khENlW1qumHjd9B7wQ3wCIU+N5DsHu5UdMr73WyS3oIKfyUk7LH/vDKOo2uFB0f5ZlryL492VIqB/vM1QgqZLurKmWEuUjfoxcaRj3hYBY95M5bK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: All E500 have MMU_FTR_TYPE_FSL_E. So remove all impossible combinations. This leads to removing PPC_HTW_IBM and related exceptions. Signed-off-by: Christophe Leroy --- arch/powerpc/include/asm/nohash/mmu-e500.h | 1 - arch/powerpc/mm/nohash/tlb.c | 148 ++++------------ arch/powerpc/mm/nohash/tlb_low_64e.S | 194 --------------------- 3 files changed, 36 insertions(+), 307 deletions(-) diff --git a/arch/powerpc/include/asm/nohash/mmu-e500.h b/arch/powerpc/include/asm/nohash/mmu-e500.h index 6ddced0415cb..792bfaafd70b 100644 --- a/arch/powerpc/include/asm/nohash/mmu-e500.h +++ b/arch/powerpc/include/asm/nohash/mmu-e500.h @@ -303,7 +303,6 @@ extern unsigned long linear_map_top; extern int book3e_htw_mode; #define PPC_HTW_NONE 0 -#define PPC_HTW_IBM 1 #define PPC_HTW_E6500 2 /* diff --git a/arch/powerpc/mm/nohash/tlb.c b/arch/powerpc/mm/nohash/tlb.c index d16f1ef7516c..1caccbf4c138 100644 --- a/arch/powerpc/mm/nohash/tlb.c +++ b/arch/powerpc/mm/nohash/tlb.c @@ -400,13 +400,11 @@ void tlb_flush_pgtable(struct mmu_gather *tlb, unsigned long address) static void __init setup_page_sizes(void) { unsigned int tlb0cfg; - unsigned int tlb0ps; unsigned int eptcfg; - int i, psize; + int psize; unsigned int mmucfg = mfspr(SPRN_MMUCFG); - int fsl_mmu = mmu_has_feature(MMU_FTR_TYPE_FSL_E); - if (fsl_mmu && (mmucfg & MMUCFG_MAVN) == MMUCFG_MAVN_V1) { + if ((mmucfg & MMUCFG_MAVN) == MMUCFG_MAVN_V1) { unsigned int tlb1cfg = mfspr(SPRN_TLB1CFG); unsigned int min_pg, max_pg; @@ -429,11 +427,7 @@ static void __init setup_page_sizes(void) if ((shift >= min_pg) && (shift <= max_pg)) def->flags |= MMU_PAGE_SIZE_DIRECT; } - - goto out; - } - - if (fsl_mmu && (mmucfg & MMUCFG_MAVN) == MMUCFG_MAVN_V2) { + } else if ((mmucfg & MMUCFG_MAVN) == MMUCFG_MAVN_V2) { u32 tlb1cfg, tlb1ps; tlb0cfg = mfspr(SPRN_TLB0CFG); @@ -465,54 +459,8 @@ static void __init setup_page_sizes(void) def->flags |= MMU_PAGE_SIZE_INDIRECT; } } - - goto out; - } - - tlb0cfg = mfspr(SPRN_TLB0CFG); - tlb0ps = mfspr(SPRN_TLB0PS); - eptcfg = mfspr(SPRN_EPTCFG); - - /* Look for supported direct sizes */ - for (psize = 0; psize < MMU_PAGE_COUNT; ++psize) { - struct mmu_psize_def *def = &mmu_psize_defs[psize]; - - if (tlb0ps & (1U << (def->shift - 10))) - def->flags |= MMU_PAGE_SIZE_DIRECT; - } - - /* Indirect page sizes supported ? */ - if ((tlb0cfg & TLBnCFG_IND) == 0 || - (tlb0cfg & TLBnCFG_PT) == 0) - goto out; - - book3e_htw_mode = PPC_HTW_IBM; - - /* Now, we only deal with one IND page size for each - * direct size. Hopefully all implementations today are - * unambiguous, but we might want to be careful in the - * future. - */ - for (i = 0; i < 3; i++) { - unsigned int ps, sps; - - sps = eptcfg & 0x1f; - eptcfg >>= 5; - ps = eptcfg & 0x1f; - eptcfg >>= 5; - if (!ps || !sps) - continue; - for (psize = 0; psize < MMU_PAGE_COUNT; psize++) { - struct mmu_psize_def *def = &mmu_psize_defs[psize]; - - if (ps == (def->shift - 10)) - def->flags |= MMU_PAGE_SIZE_INDIRECT; - if (sps == (def->shift - 10)) - def->ind = ps + 10; - } } -out: /* Cleanup array and print summary */ pr_info("MMU: Supported page sizes\n"); for (psize = 0; psize < MMU_PAGE_COUNT; ++psize) { @@ -540,10 +488,6 @@ static void __init setup_mmu_htw(void) */ switch (book3e_htw_mode) { - case PPC_HTW_IBM: - patch_exception(0x1c0, exc_data_tlb_miss_htw_book3e); - patch_exception(0x1e0, exc_instruction_tlb_miss_htw_book3e); - break; case PPC_HTW_E6500: extlb_level_exc = EX_TLB_SIZE; patch_exception(0x1c0, exc_data_tlb_miss_e6500_book3e); @@ -560,6 +504,8 @@ static void __init setup_mmu_htw(void) static void early_init_this_mmu(void) { unsigned int mas4; + unsigned int num_cams; + bool map = true; /* Set MAS4 based on page table setting */ @@ -572,12 +518,6 @@ static void early_init_this_mmu(void) mmu_pte_psize = MMU_PAGE_2M; break; - case PPC_HTW_IBM: - mas4 |= MAS4_INDD; - mas4 |= BOOK3E_PAGESZ_1M << MAS4_TSIZED_SHIFT; - mmu_pte_psize = MMU_PAGE_1M; - break; - case PPC_HTW_NONE: mas4 |= BOOK3E_PAGESZ_4K << MAS4_TSIZED_SHIFT; mmu_pte_psize = mmu_virtual_psize; @@ -585,26 +525,21 @@ static void early_init_this_mmu(void) } mtspr(SPRN_MAS4, mas4); - if (mmu_has_feature(MMU_FTR_TYPE_FSL_E)) { - unsigned int num_cams; - bool map = true; - - /* use a quarter of the TLBCAM for bolted linear map */ - num_cams = (mfspr(SPRN_TLB1CFG) & TLBnCFG_N_ENTRY) / 4; + /* use a quarter of the TLBCAM for bolted linear map */ + num_cams = (mfspr(SPRN_TLB1CFG) & TLBnCFG_N_ENTRY) / 4; - /* - * Only do the mapping once per core, or else the - * transient mapping would cause problems. - */ + /* + * Only do the mapping once per core, or else the + * transient mapping would cause problems. + */ #ifdef CONFIG_SMP - if (hweight32(get_tensr()) > 1) - map = false; + if (hweight32(get_tensr()) > 1) + map = false; #endif - if (map) - linear_map_top = map_mem_in_cams(linear_map_top, - num_cams, false, true); - } + if (map) + linear_map_top = map_mem_in_cams(linear_map_top, + num_cams, false, true); /* A sync won't hurt us after mucking around with * the MMU configuration @@ -620,10 +555,7 @@ static void __init early_init_mmu_global(void) * * Freescale booke only supports 4K pages in TLB0, so use that. */ - if (mmu_has_feature(MMU_FTR_TYPE_FSL_E)) - mmu_vmemmap_psize = MMU_PAGE_4K; - else - mmu_vmemmap_psize = MMU_PAGE_16M; + mmu_vmemmap_psize = MMU_PAGE_4K; /* XXX This code only checks for TLB 0 capabilities and doesn't * check what page size combos are supported by the HW. It @@ -636,13 +568,10 @@ static void __init early_init_mmu_global(void) /* Look for HW tablewalk support */ setup_mmu_htw(); - if (mmu_has_feature(MMU_FTR_TYPE_FSL_E)) { - if (book3e_htw_mode == PPC_HTW_NONE) { - extlb_level_exc = EX_TLB_SIZE; - patch_exception(0x1c0, exc_data_tlb_miss_bolted_book3e); - patch_exception(0x1e0, - exc_instruction_tlb_miss_bolted_book3e); - } + if (book3e_htw_mode == PPC_HTW_NONE) { + extlb_level_exc = EX_TLB_SIZE; + patch_exception(0x1c0, exc_data_tlb_miss_bolted_book3e); + patch_exception(0x1e0, exc_instruction_tlb_miss_bolted_book3e); } /* Set the global containing the top of the linear mapping @@ -655,16 +584,14 @@ static void __init early_init_mmu_global(void) static void __init early_mmu_set_memory_limit(void) { - if (mmu_has_feature(MMU_FTR_TYPE_FSL_E)) { - /* - * Limit memory so we dont have linear faults. - * Unlike memblock_set_current_limit, which limits - * memory available during early boot, this permanently - * reduces the memory available to Linux. We need to - * do this because highmem is not supported on 64-bit. - */ - memblock_enforce_memory_limit(linear_map_top); - } + /* + * Limit memory so we dont have linear faults. + * Unlike memblock_set_current_limit, which limits + * memory available during early boot, this permanently + * reduces the memory available to Linux. We need to + * do this because highmem is not supported on 64-bit. + */ + memblock_enforce_memory_limit(linear_map_top); memblock_set_current_limit(linear_map_top); } @@ -702,19 +629,16 @@ void setup_initial_memory_limit(phys_addr_t first_memblock_base, * We crop it to the size of the first MEMBLOCK to * avoid going over total available memory just in case... */ - if (early_mmu_has_feature(MMU_FTR_TYPE_FSL_E)) { - unsigned long linear_sz; - unsigned int num_cams; + unsigned long linear_sz; + unsigned int num_cams; - /* use a quarter of the TLBCAM for bolted linear map */ - num_cams = (mfspr(SPRN_TLB1CFG) & TLBnCFG_N_ENTRY) / 4; + /* use a quarter of the TLBCAM for bolted linear map */ + num_cams = (mfspr(SPRN_TLB1CFG) & TLBnCFG_N_ENTRY) / 4; - linear_sz = map_mem_in_cams(first_memblock_size, num_cams, - true, true); + linear_sz = map_mem_in_cams(first_memblock_size, num_cams, + true, true); - ppc64_rma_size = min_t(u64, linear_sz, 0x40000000); - } else - ppc64_rma_size = min_t(u64, first_memblock_size, 0x40000000); + ppc64_rma_size = min_t(u64, linear_sz, 0x40000000); /* Finally limit subsequent allocations */ memblock_set_current_limit(first_memblock_base + ppc64_rma_size); diff --git a/arch/powerpc/mm/nohash/tlb_low_64e.S b/arch/powerpc/mm/nohash/tlb_low_64e.S index 7e0b8fe1c279..93ecb8ec82b0 100644 --- a/arch/powerpc/mm/nohash/tlb_low_64e.S +++ b/arch/powerpc/mm/nohash/tlb_low_64e.S @@ -894,200 +894,6 @@ virt_page_table_tlb_miss_whacko_fault: b exc_data_storage_book3e -/************************************************************** - * * - * TLB miss handling for Book3E with hw page table support * - * * - **************************************************************/ - - -/* Data TLB miss */ - START_EXCEPTION(data_tlb_miss_htw) - TLB_MISS_PROLOG - - /* Now we handle the fault proper. We only save DEAR in normal - * fault case since that's the only interesting values here. - * We could probably also optimize by not saving SRR0/1 in the - * linear mapping case but I'll leave that for later - */ - mfspr r14,SPRN_ESR - mfspr r16,SPRN_DEAR /* get faulting address */ - srdi r11,r16,44 /* get region */ - xoris r11,r11,0xc - cmpldi cr0,r11,0 /* linear mapping ? */ - beq tlb_load_linear /* yes -> go to linear map load */ - cmpldi cr1,r11,1 /* vmalloc mapping ? */ - - /* We do the user/kernel test for the PID here along with the RW test - */ - srdi. r11,r16,60 /* Check for user region */ - ld r15,PACAPGD(r13) /* Load user pgdir */ - beq htw_tlb_miss - - /* XXX replace the RMW cycles with immediate loads + writes */ -1: mfspr r10,SPRN_MAS1 - rlwinm r10,r10,0,16,1 /* Clear TID */ - mtspr SPRN_MAS1,r10 - ld r15,PACA_KERNELPGD(r13) /* Load kernel pgdir */ - beq+ cr1,htw_tlb_miss - - /* We got a crappy address, just fault with whatever DEAR and ESR - * are here - */ - TLB_MISS_EPILOG_ERROR - b exc_data_storage_book3e - -/* Instruction TLB miss */ - START_EXCEPTION(instruction_tlb_miss_htw) - TLB_MISS_PROLOG - - /* If we take a recursive fault, the second level handler may need - * to know whether we are handling a data or instruction fault in - * order to get to the right store fault handler. We provide that - * info by keeping a crazy value for ESR in r14 - */ - li r14,-1 /* store to exception frame is done later */ - - /* Now we handle the fault proper. We only save DEAR in the non - * linear mapping case since we know the linear mapping case will - * not re-enter. We could indeed optimize and also not save SRR0/1 - * in the linear mapping case but I'll leave that for later - * - * Faulting address is SRR0 which is already in r16 - */ - srdi r11,r16,44 /* get region */ - xoris r11,r11,0xc - cmpldi cr0,r11,0 /* linear mapping ? */ - beq tlb_load_linear /* yes -> go to linear map load */ - cmpldi cr1,r11,1 /* vmalloc mapping ? */ - - /* We do the user/kernel test for the PID here along with the RW test - */ - srdi. r11,r16,60 /* Check for user region */ - ld r15,PACAPGD(r13) /* Load user pgdir */ - beq htw_tlb_miss - - /* XXX replace the RMW cycles with immediate loads + writes */ -1: mfspr r10,SPRN_MAS1 - rlwinm r10,r10,0,16,1 /* Clear TID */ - mtspr SPRN_MAS1,r10 - ld r15,PACA_KERNELPGD(r13) /* Load kernel pgdir */ - beq+ htw_tlb_miss - - /* We got a crappy address, just fault */ - TLB_MISS_EPILOG_ERROR - b exc_instruction_storage_book3e - - -/* - * This is the guts of the second-level TLB miss handler for direct - * misses. We are entered with: - * - * r16 = virtual page table faulting address - * r15 = PGD pointer - * r14 = ESR - * r13 = PACA - * r12 = TLB exception frame in PACA - * r11 = crap (free to use) - * r10 = crap (free to use) - * - * It can be re-entered by the linear mapping miss handler. However, to - * avoid too much complication, it will save/restore things for us - */ -htw_tlb_miss: -#ifdef CONFIG_PPC_KUAP - mfspr r10,SPRN_MAS1 - rlwinm. r10,r10,0,0x3fff0000 - beq- htw_tlb_miss_fault /* KUAP fault */ -#endif - /* Search if we already have a TLB entry for that virtual address, and - * if we do, bail out. - * - * MAS1:IND should be already set based on MAS4 - */ - PPC_TLBSRX_DOT(0,R16) - beq htw_tlb_miss_done - - /* Now, we need to walk the page tables. First check if we are in - * range. - */ - rldicl. r10,r16,64-PGTABLE_EADDR_SIZE,PGTABLE_EADDR_SIZE+4 - bne- htw_tlb_miss_fault - - /* Get the PGD pointer */ - cmpldi cr0,r15,0 - beq- htw_tlb_miss_fault - - /* Get to PGD entry */ - rldicl r11,r16,64-(PGDIR_SHIFT-3),64-PGD_INDEX_SIZE-3 - clrrdi r10,r11,3 - ldx r15,r10,r15 - cmpdi cr0,r15,0 - bge htw_tlb_miss_fault - - /* Get to PUD entry */ - rldicl r11,r16,64-(PUD_SHIFT-3),64-PUD_INDEX_SIZE-3 - clrrdi r10,r11,3 - ldx r15,r10,r15 - cmpdi cr0,r15,0 - bge htw_tlb_miss_fault - - /* Get to PMD entry */ - rldicl r11,r16,64-(PMD_SHIFT-3),64-PMD_INDEX_SIZE-3 - clrrdi r10,r11,3 - ldx r15,r10,r15 - cmpdi cr0,r15,0 - bge htw_tlb_miss_fault - - /* Ok, we're all right, we can now create an indirect entry for - * a 1M or 256M page. - * - * The last trick is now that because we use "half" pages for - * the HTW (1M IND is 2K and 256M IND is 32K) we need to account - * for an added LSB bit to the RPN. For 64K pages, there is no - * problem as we already use 32K arrays (half PTE pages), but for - * 4K page we need to extract a bit from the virtual address and - * insert it into the "PA52" bit of the RPN. - */ - rlwimi r15,r16,32-9,20,20 - /* Now we build the MAS: - * - * MAS 0 : Fully setup with defaults in MAS4 and TLBnCFG - * MAS 1 : Almost fully setup - * - PID already updated by caller if necessary - * - TSIZE for now is base ind page size always - * MAS 2 : Use defaults - * MAS 3+7 : Needs to be done - */ - ori r10,r15,(BOOK3E_PAGESZ_4K << MAS3_SPSIZE_SHIFT) - - srdi r16,r10,32 - mtspr SPRN_MAS3,r10 - mtspr SPRN_MAS7,r16 - - tlbwe - -htw_tlb_miss_done: - /* We don't bother with restoring DEAR or ESR since we know we are - * level 0 and just going back to userland. They are only needed - * if you are going to take an access fault - */ - TLB_MISS_EPILOG_SUCCESS - rfi - -htw_tlb_miss_fault: - /* We need to check if it was an instruction miss. We know this - * though because r14 would contain -1 - */ - cmpdi cr0,r14,-1 - beq 1f - mtspr SPRN_DEAR,r16 - mtspr SPRN_ESR,r14 - TLB_MISS_EPILOG_ERROR - b exc_data_storage_book3e -1: TLB_MISS_EPILOG_ERROR - b exc_instruction_storage_book3e - /* * This is the guts of "any" level TLB miss handler for kernel linear * mapping misses. We are entered with: