From patchwork Tue Jun 15 03:07:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liu Shixin X-Patchwork-Id: 12322453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B1D2C48BDF for ; Tue, 15 Jun 2021 16:51:46 +0000 (UTC) Received: from bombadil.infradead.org (unknown [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 247A760E0B for ; Tue, 15 Jun 2021 16:51:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 247A760E0B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:CC :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=91PcFwsQvde5HiP+cbc0JyJUTFiPNqNgYU/gL5KI6CY=; b=tnI/DVVSxoZzj8 lfMHhN7n0ZRFjIHW7EJJ8f01rZDjFig+R3WozQauVhsotyKK1ktSBUFenpuGTnE9/4a6DT/ODA+Pd 1/w6lQA9yumZhrKxxoY5FR6I8NnK2WFtaB9R9ds66McOYnJkeGr3snUi2a2rQC5738epJOsOoeddh KMrRd7FJdrDenuQrF/J+B83GxA1mHdx1G8oW2YMDciUpMBChq3HiBLbo5Ey7IlzdTPpepVOb1RZN/ 5tHDMZH4nZ2crL2x23W/YDV4dfH4dEm4ZNrhZiLTZHUf0/VKPmiudoOsc3tOh5VdWzfN/tvU/JOTA qsg2uSp8nRsJPYF68lvg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ltCGp-001Ur2-3V; Tue, 15 Jun 2021 16:50:35 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lt9Px-000DDm-L9 for linux-riscv@bombadil.infradead.org; Tue, 15 Jun 2021 13:47:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:Message-ID: Date:Subject:CC:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:In-Reply-To:References; bh=qH70eWL4H3WPCS8Rv8+ia8iR10xokWICkMm7XA6PAsE=; b=bHmntLxiirsDz3t8LmsOMJMXjE KtXjXMOEssqs31spuwOdTw+9iFvv/m3Aird7JecWZaMkxtFMgWlB26hzK9asLPUFgYG1vzJtJQpGY lse7ykq2ZxFqjAGifevh8xILC0yV5Gz4uMKKseENpETEQ5y0yyuniYlxRHyNwzwvJFnHRxZyb/F2J 0VOC0EmshR2ARnL0CEjFIb9broeql2pyikRojr8pg4Br890BXcQ1PyB71L4ZYGIUqQYjv10AWzvG6 y1H5U0Ksbj+olwqxBM97mOdVRXmHLvMhti03dS1mqdcQXq91G7HNxuwxXV48ZKsSiVEAcdnp94ni6 r0touoMA==; Received: from szxga08-in.huawei.com ([45.249.212.255]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lsyva-007GeA-Qf for linux-riscv@lists.infradead.org; Tue, 15 Jun 2021 02:35:56 +0000 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4G3skB1kYZz1BMSs; Tue, 15 Jun 2021 10:30:42 +0800 (CST) Received: from dggpemm500009.china.huawei.com (7.185.36.225) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Tue, 15 Jun 2021 10:35:41 +0800 Received: from huawei.com (10.175.113.32) by dggpemm500009.china.huawei.com (7.185.36.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Tue, 15 Jun 2021 10:35:41 +0800 From: Liu Shixin To: Paul Walmsley , Palmer Dabbelt , Palmer Dabbelt , Albert Ou , Alexander Potapenko , Marco Elver , Dmitry Vyukov CC: , , , Liu Shixin Subject: [PATCH -next v2] riscv: Enable KFENCE for riscv64 Date: Tue, 15 Jun 2021 11:07:34 +0800 Message-ID: <20210615030734.2465923-1-liushixin2@huawei.com> X-Mailer: git-send-email 2.18.0.huawei.25 MIME-Version: 1.0 X-Originating-IP: [10.175.113.32] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500009.china.huawei.com (7.185.36.225) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210615_033554_264299_21DEB90C X-CRM114-Status: GOOD ( 16.16 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add architecture specific implementation details for KFENCE and enable KFENCE for the riscv64 architecture. In particular, this implements the required interface in . KFENCE requires that attributes for pages from its memory pool can individually be set. Therefore, force the kfence pool to be mapped at page granularity. Testing this patch using the testcases in kfence_test.c and all passed. Signed-off-by: Liu Shixin Acked-by: Marco Elver Reviewed-by: Kefeng Wang --- v1->v2: Add the acked-by and Reviewed-by. arch/riscv/Kconfig | 1 + arch/riscv/include/asm/kfence.h | 63 +++++++++++++++++++++++++++++++++ arch/riscv/mm/fault.c | 11 +++++- 3 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/kfence.h diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 4c0bfb2569e9..1421da5ef1a2 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -65,6 +65,7 @@ config RISCV select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if MMU && 64BIT select HAVE_ARCH_KASAN_VMALLOC if MMU && 64BIT + select HAVE_ARCH_KFENCE if MMU && 64BIT select HAVE_ARCH_KGDB select HAVE_ARCH_KGDB_QXFER_PKT select HAVE_ARCH_MMAP_RND_BITS if MMU diff --git a/arch/riscv/include/asm/kfence.h b/arch/riscv/include/asm/kfence.h new file mode 100644 index 000000000000..d887a54042aa --- /dev/null +++ b/arch/riscv/include/asm/kfence.h @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_RISCV_KFENCE_H +#define _ASM_RISCV_KFENCE_H + +#include +#include +#include +#include + +static inline int split_pmd_page(unsigned long addr) +{ + int i; + unsigned long pfn = PFN_DOWN(__pa((addr & PMD_MASK))); + pmd_t *pmd = pmd_off_k(addr); + pte_t *pte = pte_alloc_one_kernel(&init_mm); + + if (!pte) + return -ENOMEM; + + for (i = 0; i < PTRS_PER_PTE; i++) + set_pte(pte + i, pfn_pte(pfn + i, PAGE_KERNEL)); + set_pmd(pmd, pfn_pmd(PFN_DOWN(__pa(pte)), PAGE_TABLE)); + + flush_tlb_kernel_range(addr, addr + PMD_SIZE); + return 0; +} + +static inline bool arch_kfence_init_pool(void) +{ + int ret; + unsigned long addr; + pmd_t *pmd; + + for (addr = (unsigned long)__kfence_pool; is_kfence_address((void *)addr); + addr += PAGE_SIZE) { + pmd = pmd_off_k(addr); + + if (pmd_leaf(*pmd)) { + ret = split_pmd_page(addr); + if (ret) + return false; + } + } + + return true; +} + +static inline bool kfence_protect_page(unsigned long addr, bool protect) +{ + pte_t *pte = virt_to_kpte(addr); + + if (protect) + set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_PRESENT)); + else + set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT)); + + flush_tlb_kernel_range(addr, addr + PAGE_SIZE); + + return true; +} + +#endif /* _ASM_RISCV_KFENCE_H */ diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 096463cc6fff..aa08dd2f8fae 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -45,7 +46,15 @@ static inline void no_context(struct pt_regs *regs, unsigned long addr) * Oops. The kernel tried to access some bad page. We'll have to * terminate things with extreme prejudice. */ - msg = (addr < PAGE_SIZE) ? "NULL pointer dereference" : "paging request"; + if (addr < PAGE_SIZE) + msg = "NULL pointer dereference"; + else { + if (kfence_handle_page_fault(addr, regs->cause == EXC_STORE_PAGE_FAULT, regs)) + return; + + msg = "paging request"; + } + die_kernel_fault(msg, addr, regs); }