From patchwork Thu Aug 29 11:33:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 11121135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C2A614D5 for ; Thu, 29 Aug 2019 11:36:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 273DB2166E for ; Thu, 29 Aug 2019 11:36:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WqlLJMFn"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="E0eqydah" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 273DB2166E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+/mGGfdkDx+eUo6I+9GNQYWdGX34vWnR6YBiz5YiaYs=; b=WqlLJMFnly9fRt yoV3JoX+ZT5YbB0P9vlPuFjN/fM01knmkobnvHvsPYSPlX+gG6SDDammiyq0mZAJG/IHH5L5k1Du7 W9r0kbM/BV3hdZSzWHQS3J7+20MUGalcMVio5FHbZsA3chX4Q5yThQh5bf2ofe7O7xPyOGPP0lcQF WZqF4bS0c0upotkKBX17ba3T4S7yn60WoTzDY/yS1kn1TIIPM0jWbIwQSy8C5Cty1YDLJ3ITbE60X 7Zki3FPg+z3wanQjJf0ADv2UXEvRWbUJbpK449LjtrBEA4TvC0tUiHl2IPPZdmK4XDRDuOgds27db UTlzbiXFF80NsRn3N5+g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1i3Iiu-00074D-Bt; Thu, 29 Aug 2019 11:36:16 +0000 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1i3Ihk-0004sL-Nk for linux-arm-kernel@lists.infradead.org; Thu, 29 Aug 2019 11:35:07 +0000 Received: by mail-pf1-x444.google.com with SMTP id i30so1853816pfk.9 for ; Thu, 29 Aug 2019 04:35:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kHvdeN9oObJ1DWOknB6wJwOgumkOXHrRy2jfFMpABJ8=; b=E0eqydahPr0xeqAZBqi7e7Ni2Cc7nIKSMFIuNCPVKFztaIU87as0wu7lDsChydOqGE NbY+aHH+m1XpuQBiGurAvDyfVB0nHBqDRtTAGQ1awghUZNaq2B48O0LmzC8wpRaq66tY szMNV1zVUisdCHI08NTOlzn4PakgRnebE/apB/GcumiTa4iqV0HWl7ZTIlHBxpNRwhe3 Nv7IOKHIXLlb5fIi32MTa9SZgj5k5ZmrS34KaFtZznALoqeZC7U9wV8aordLxF5kwnnV E0x0OmA/bhCg4SA9x9i6QWFR0otYSFu8tjYUiZp1qjp6JqY4hwfhC3lDHeLTzIVXLlqS TIkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kHvdeN9oObJ1DWOknB6wJwOgumkOXHrRy2jfFMpABJ8=; b=QfVB7qNsq7Du7lVHxWIS+yHJ2J8xD4OMx/IgUlCrrzw7CfxB/80CufGA9hSOtNrsVI vSX28Vxe2GNj1IMAuiVnhhiEF9qq+JmKUZZeKHaJtCAiReMU9jNoAG7haD8ItwP/ORxS 8ethH08v3g/PoNmOzEHq7Ne+yynts0xUYIZllEUEcUoyFWwahtK92hmNm5lOq1GSPI21 Vyx+b5JkpYgW7vSMvtTPj7g7aC8py+UT+XcFaWRGoUkFZJEBw2M7tXbzsYA5NiYThnQt 8r0orncCF+mcOy+avwzjOZQaWgSsH+rdO/dC2kbpbYYSjwtzRiKd8Ndl19DK7VPdKaos PnQw== X-Gm-Message-State: APjAAAVTKUinSD8m8wuuPdyh/4+763McZRBZOV6vHddbQzuqasrO5DsR okhT4CQf15tCdwuD+4wtP0y0bA== X-Google-Smtp-Source: APXvYqzTzXFgTx2DQjccjudShBBaBw2VdBRUo+2BxSAYllEg/QCnesWjQlEkAXwCUSydVWvG86ed9w== X-Received: by 2002:a63:1020:: with SMTP id f32mr8172264pgl.203.1567078502823; Thu, 29 Aug 2019 04:35:02 -0700 (PDT) Received: from localhost ([122.167.132.221]) by smtp.gmail.com with ESMTPSA id e66sm6387216pfe.142.2019.08.29.04.35.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 29 Aug 2019 04:35:02 -0700 (PDT) From: Viresh Kumar To: stable@vger.kernel.org, Julien Thierry , Mark Rutland Subject: [PATCH ARM64 v4.4 V3 04/44] arm64: Make USER_DS an inclusive limit Date: Thu, 29 Aug 2019 17:03:49 +0530 Message-Id: <55f8561f1ee207237fdcfbc8c5dc043be06d3de6.1567077734.git.viresh.kumar@linaro.org> X-Mailer: git-send-email 2.21.0.rc0.269.g1a574e7a288b In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190829_043505_510674_72DFBA21 X-CRM114-Status: GOOD ( 16.11 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:444 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Viresh Kumar , Will Deacon , mark.brown@arm.com, Catalin Marinas , Russell King , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Robin Murphy commit 51369e398d0d33e8f524314e672b07e8cf870e79 upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas [ 4.4: Dropped changes from fault.c and fixed minor rebase conflict ] Signed-off-by: Viresh Kumar Reviewed-by: Mark Rutland [v4.4 backport] --- arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/include/asm/uaccess.h | 45 +++++++++++++++++------------- arch/arm64/kernel/entry.S | 4 +-- 3 files changed, 31 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 75d9ef6c457c..ff1449c25bf4 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 829fa6d3e561..c625cc5531fc 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -56,10 +56,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -87,22 +84,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -111,7 +118,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs /* diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index c849be9231bb..4c5013b09dcb 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -96,10 +96,10 @@ .else add x21, sp, #S_FRAME_SIZE get_thread_info tsk - /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ + /* Save the task's original addr_limit and set USER_DS */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] .endif /* \el == 0 */ mrs x22, elr_el1