From patchwork Wed Jan 31 09:34:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10193685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 84243603EE for ; Wed, 31 Jan 2018 09:40:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64729201BC for ; Wed, 31 Jan 2018 09:40:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59390285FC; Wed, 31 Jan 2018 09:40:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C0F5E201BC for ; Wed, 31 Jan 2018 09:40:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=3T+sKaPtBbImfCtUGSM9LfFhDku7RaXXqnP1l4+Loww=; b=Wb99ovtG/YLMv8W43OAiHaRpMi FWPAaKKpjuWZYDF7eAP00JswK8ysWX4Cbt0kF+D8OHM9C1BsqQqvtIT1QTOYR5kvlqqV1vCQ1k8vo gDaN4Dv6dkO9AbPrpJo8PoSo3EcR83D8eJqqxzX+hKMDM04DwdF/8KGECd6ZWOnoRemWzQclfVfNH kJjz9jT27vOobZ963sNN5mk4N90+v6kJkC1uYahBFK+eJnijsiypjTR90Goh9dzTfwv1Mv40IL2QV f+ML2YDAiQGuh25XxsbTg61PXlXPsXIFN/TjOIxXF9EcaWNGUdu6infHTJUeZSS7NqhsgKKrcuBlX RIKhhx/Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1egosP-0002to-5o; Wed, 31 Jan 2018 09:40:21 +0000 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1egony-0005fR-B6 for linux-arm-kernel@lists.infradead.org; Wed, 31 Jan 2018 09:35:51 +0000 Received: by mail-wm0-x244.google.com with SMTP id j21so7526757wmh.1 for ; Wed, 31 Jan 2018 01:35:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RNbS4pnS/F924Bve05xPqewNo7KYyI30ap4oTO49g1E=; b=hY41xq3gZweqsBwMxybL8vvYdELJjC8z31hZkyxE8YXV+NsbXu1TX5dnhVrSbPYUo5 i9abyMaqMki05oc4S5AU73HB54bFJDRPIG1Ax+jSqFVK2Z+u/Y4ofKxx99yOwVmxXYly T2l43ewpp32NFPRZmv8Xijnfz54Mnbfcc5JJ4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RNbS4pnS/F924Bve05xPqewNo7KYyI30ap4oTO49g1E=; b=RUYCaqg4mgIU608ckyZB2w+Vi50DOqXVdVggZ/BybWTiBiDvECymDECbCyeR7mHdKa 8BJOp6q8ms7xnF1wwN63xuW9bjqhxJhQJ68MAyl0PDG6Y7ftfO4twvmHUDhbKbyQW7yT cUZC/2CtQFI+66nmJ9skqcqn+Est7UidQLhNf/3FQ+rostniE5NVHWKcgtZGoSYlIgCB Mol803TfMouJvnVVMag+Dnk1g3o6gmtK2oMmN3IP1NoHWCNDflQO1oFEx7096v9Iejxp WwPARfhy3vW07PQZO2zA9QqOXVlcNR0xQs+GhMGq7ynzj5ERvMPXzZHEMfvBBIxMvLmH Osng== X-Gm-Message-State: AKwxytcENa9dCoK7AwioBjbaaOxWXvtpGj5IEWDQ1WShQTEFTSBx8bcw SAtnimp460uSRJnhY3vTmTYmn72wm0c= X-Google-Smtp-Source: AH8x226Y6dgE3AhMtRG1VQdHnwqEGH4a9npaA+svXjVx9ooVqwGbSheFRc4eEpXUF6hog6APvUKTHw== X-Received: by 10.80.180.184 with SMTP id w53mr30062988edd.149.1517391334791; Wed, 31 Jan 2018 01:35:34 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id z2sm10247837edc.50.2018.01.31.01.35.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 31 Jan 2018 01:35:34 -0800 (PST) From: Christoffer Dall To: Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Subject: [PULL 15/28] arm64: KVM: Add invalidate_icache_range helper Date: Wed, 31 Jan 2018 10:34:54 +0100 Message-Id: <20180131093507.22219-16-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180131093507.22219-1-christoffer.dall@linaro.org> References: <20180131093507.22219-1-christoffer.dall@linaro.org> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Christoffer Dall , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Marc Zyngier We currently tightly couple dcache clean with icache invalidation, but KVM could do without the initial flush to PoU, as we've already flushed things to PoC. Let's introduce invalidate_icache_range which is limited to invalidating the icache from the linear mapping (and thus has none of the userspace fault handling complexity), and wire it in KVM instead of flush_icache_range. Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Christoffer Dall --- arch/arm64/include/asm/assembler.h | 21 +++++++++++++++++++++ arch/arm64/include/asm/cacheflush.h | 7 +++++++ arch/arm64/include/asm/kvm_mmu.h | 4 ++-- arch/arm64/mm/cache.S | 32 ++++++++++++++++++++++---------- 4 files changed, 52 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index aef72d886677..0884e1fdfd30 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -387,6 +387,27 @@ alternative_endif dsb \domain .endm +/* + * Macro to perform an instruction cache maintenance for the interval + * [start, end) + * + * start, end: virtual addresses describing the region + * label: A label to branch to on user fault. + * Corrupts: tmp1, tmp2 + */ + .macro invalidate_icache_by_line start, end, tmp1, tmp2, label + icache_line_size \tmp1, \tmp2 + sub \tmp2, \tmp1, #1 + bic \tmp2, \start, \tmp2 +9997: +USER(\label, ic ivau, \tmp2) // invalidate I line PoU + add \tmp2, \tmp2, \tmp1 + cmp \tmp2, \end + b.lo 9997b + dsb ish + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 955130762a3c..bef9f418f089 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -52,6 +52,12 @@ * - start - virtual start address * - end - virtual end address * + * invalidate_icache_range(start, end) + * + * Invalidate the I-cache in the region described by start, end. + * - start - virtual start address + * - end - virtual end address + * * __flush_cache_user_range(start, end) * * Ensure coherency between the I-cache and the D-cache in the @@ -66,6 +72,7 @@ * - size - region size */ extern void flush_icache_range(unsigned long start, unsigned long end); +extern int invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); extern void __inval_dcache_area(void *addr, size_t len); extern void __clean_dcache_area_poc(void *addr, size_t len); diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 8034b96fb3a4..56b3e03c85e7 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -250,8 +250,8 @@ static inline void __invalidate_icache_guest_page(struct kvm_vcpu *vcpu, /* PIPT or VPIPT at EL2 (see comment in __kvm_tlb_flush_vmid_ipa) */ void *va = page_address(pfn_to_page(pfn)); - flush_icache_range((unsigned long)va, - (unsigned long)va + size); + invalidate_icache_range((unsigned long)va, + (unsigned long)va + size); } } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 7f1dbe962cf5..bedd23da83f4 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -60,16 +60,7 @@ user_alt 9f, "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE b.lo 1b dsb ish - icache_line_size x2, x3 - sub x3, x2, #1 - bic x4, x0, x3 -1: -USER(9f, ic ivau, x4 ) // invalidate I line PoU - add x4, x4, x2 - cmp x4, x1 - b.lo 1b - dsb ish - isb + invalidate_icache_by_line x0, x1, x2, x3, 9f mov x0, #0 1: uaccess_ttbr0_disable x1 @@ -80,6 +71,27 @@ USER(9f, ic ivau, x4 ) // invalidate I line PoU ENDPROC(flush_icache_range) ENDPROC(__flush_cache_user_range) +/* + * invalidate_icache_range(start,end) + * + * Ensure that the I cache is invalid within specified region. + * + * - start - virtual start address of region + * - end - virtual end address of region + */ +ENTRY(invalidate_icache_range) + uaccess_ttbr0_enable x2, x3 + + invalidate_icache_by_line x0, x1, x2, x3, 2f + mov x0, xzr +1: + uaccess_ttbr0_disable x1 + ret +2: + mov x0, #-EFAULT + b 1b +ENDPROC(invalidate_icache_range) + /* * __flush_dcache_area(kaddr, size) *