From patchwork Fri Jun 22 14:43:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 10482295 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 293C160388 for ; Fri, 22 Jun 2018 14:43:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3EB928849 for ; Fri, 22 Jun 2018 14:43:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E83B628C95; Fri, 22 Jun 2018 14:43:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8881D28849 for ; Fri, 22 Jun 2018 14:43:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8cqEnp4Rz4NPdfQflUD77J3H0gHUpD/RwD6vQ0wJT2Y=; b=qleCoTqlULm1yL uHm0JIdt1fgupPiwDZO5FSMvK2/KLyIAuaY0PRSuCqExK6MdaWAsxDq0Ow024Tm8FnpGI5O5xZ8dn qMfRiQofQs46ZZbIeD/phnefQciAQdHScug9tkNHRa/l2ABZAfgZi1bkpFhrMI52qy0qSytAkFZtO L3HPVCWZE5Q3S8oykD9Qu5UFFfrtRX7z7LhOFLLZqc+0+6yN4U7ifmEYM7cVVl9YQLTmpfIjPJODX /llqbVCO0kTNHvnwv/zESU3INijnISdH1reGJnSD4msgVkOiWP4usRP7tPHvg7mVuOv0EJ6RbzxqB be7invgGZyvO4dBlytXw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fWNHa-0002O0-8z; Fri, 22 Jun 2018 14:43:26 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fWNHW-0001kb-5l for linux-arm-kernel@lists.infradead.org; Fri, 22 Jun 2018 14:43:24 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C61C680D; Fri, 22 Jun 2018 07:43:10 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9291F3F2EA; Fri, 22 Jun 2018 07:43:10 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 0C7331AE311D; Fri, 22 Jun 2018 15:43:46 +0100 (BST) Date: Fri, 22 Jun 2018 15:43:46 +0100 From: Will Deacon To: Wei Xu Subject: Re: KVM guest sometimes failed to boot because of kernel stack overflow if KPTI is enabled on a hisilicon ARM64 platform. Message-ID: <20180622144346.GB1802@arm.com> References: <20180621091850.GA22505@arm.com> <5B2B7A84.8090309@hisilicon.com> <20180621105404.GB22505@arm.com> <5B2CB440.8040705@hisilicon.com> <20180622092330.GD7601@arm.com> <5B2CD33B.9020702@hisilicon.com> <20180622111614.GA1150@arm.com> <5B2CF723.7010600@hisilicon.com> <20180622133133.GA1802@arm.com> <5B2CFDCD.6040207@hisilicon.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5B2CFDCD.6040207@hisilicon.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180622_074322_256199_B30149EF X-CRM114-Status: GOOD ( 17.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, "Chenxin \(Charles\)" , Linuxarm , Hanjun Guo , xiexiuqi@huawei.com, "kongxinwei \(A\)" , huangdaode , catalin.marinas@arm.com, "Liyuan \(Larry, Turing Solution\)" , "Zhuangyuzeng \(Yisen\)" , Zhangyi ac , suzuki.poulose@arm.com, marc.zyngier@arm.com, John Garry , "Xiongfanggou \(James\)" , jonathan.cameron@huawei.com, linux-arm-kernel@lists.infradead.org, Salil Mehta , linux-kernel@vger.kernel.org, Shameerali Kolothum Thodi , dave.martin@arm.com, zhangbin011@hisilicon.com, "Wangzhou \(B\)" , James Morse , libeijian@hisilicon.com, "Liguozhu \(Kenneth\)" , Shiju Jose Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, Jun 22, 2018 at 09:46:53PM +0800, Wei Xu wrote: > On 2018/6/22 21:31, Will Deacon wrote: > >On Fri, Jun 22, 2018 at 09:18:27PM +0800, Wei Xu wrote: > >>On 2018/6/22 19:16, Will Deacon wrote: > >>>On Fri, Jun 22, 2018 at 06:45:15PM +0800, Wei Xu wrote: > >>>>On 2018/6/22 17:23, Will Deacon wrote: > >>>>>Perhaps just writing back the table entries is enough to cause the issue, > >>>>>although I really can't understand why that would be the case. Can you try > >>>>>the diff below (without my previous change), please? > >>>>Thanks! > >>>>But it does not resolve the issue(only apply this patch based on 4.17.0). > >>>Thanks, that's a useful data point. It means that it still crashes even if > >>>we write back the same table entries, so it's the fact that we're writing > >>>them at all which causes the problem, not the value that we write. > >>> > >>>Whilst looking at the code, we noticed a missing DMB. On the off-chance > >>>that it helps, can you try this instead please? > >>Thanks! > >>Only apply below patch based on 4.17.0, we still got the crash. > >Oh well, it was worth a shot (and that's still a fix worth having). Please > >can you provide the complete disassembly for kpti_install_ng_mappings() > >(I'm referring to the C function in cpufeature.c) along with a corresponding > >crash log so that we can correlate the instruction stream with the crash? > Just let me know if you need more information. Thanks; the disassembly and log are really helpful. I have another patch for you to try below. Please can you let me know how you get on, and sorry for the back-and-forth on this. Will --->8 diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 5f9a73a4452c..26c5c3fabca8 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -216,9 +216,14 @@ ENDPROC(idmap_cpu_replace_ttbr1) .endm .macro __idmap_kpti_put_pgtable_ent_ng, type - orr \type, \type, #PTE_NG // Same bit for blocks and pages + eor \type, \type, #PTE_NG // Same bit for blocks and pages str \type, [cur_\()\type\()p] // Update the entry and ensure it + tbz \type, #11, 1234f dc civac, cur_\()\type\()p // is visible to all CPUs. + b 1235f + 1234: + dc cvac, cur_\()\type\()p + 1235: .endm /* @@ -298,6 +303,7 @@ skip_pgd: /* PUD */ walk_puds: .if CONFIG_PGTABLE_LEVELS > 3 + eor pgd, pgd, #PTE_NG pte_to_phys cur_pudp, pgd add end_pudp, cur_pudp, #(PTRS_PER_PUD * 8) do_pud: __idmap_kpti_get_pgtable_ent pud @@ -319,6 +325,7 @@ next_pud: /* PMD */ walk_pmds: .if CONFIG_PGTABLE_LEVELS > 2 + eor pud, pud, #PTE_NG pte_to_phys cur_pmdp, pud add end_pmdp, cur_pmdp, #(PTRS_PER_PMD * 8) do_pmd: __idmap_kpti_get_pgtable_ent pmd @@ -339,6 +346,7 @@ next_pmd: /* PTE */ walk_ptes: + eor pmd, pmd, #PTE_NG pte_to_phys cur_ptep, pmd add end_ptep, cur_ptep, #(PTRS_PER_PTE * 8) do_pte: __idmap_kpti_get_pgtable_ent pte