From patchwork Wed Dec 5 16:41:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10714599 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0DE6109C for ; Wed, 5 Dec 2018 16:42:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B03CC2DB97 for ; Wed, 5 Dec 2018 16:42:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AE3812DE32; Wed, 5 Dec 2018 16:42:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19D5A2DE43 for ; Wed, 5 Dec 2018 16:42:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A34A6B754D; Wed, 5 Dec 2018 11:42:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 953026B754E; Wed, 5 Dec 2018 11:42:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81AB36B754F; Wed, 5 Dec 2018 11:42:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot1-f71.google.com (mail-ot1-f71.google.com [209.85.210.71]) by kanga.kvack.org (Postfix) with ESMTP id 5B4736B754D for ; Wed, 5 Dec 2018 11:42:07 -0500 (EST) Received: by mail-ot1-f71.google.com with SMTP id s3so9706227otb.0 for ; Wed, 05 Dec 2018 08:42:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ssm9tJWHFrtRksDx3U9R+7F2PdGwOmZQj+FU4qI3efA=; b=c93mZEuq8RprUqVGDvyCpmoafaemQs2WDOB5LWs2Z2DjPCj7gr+hFoArtTNgBbTz7b RM/Br1TD+HWRC42EzxEVg2uXR/3Sw2lq2pw2eIyCD961EF1VCbJnFed5Lj0hViAeDfmI wJ/V5f8wM9792zv8V62p9v0PB8ZxjVFhpPmdORxJMZ0lmnNUKVa6gnPY/ijQGA9nyvE5 hIV1Kaj6mIoyi3MEV96mw6VxBiCWPxPXna5kcbJhZwgmEr7xWqWIpDIJChLrQD+j/TcU UP6L63Ft0qSxJczqIVxF28ecLBRwsEtPuYz6yTSbRxW0hcXJMTUDSqAw4THQinBIUKGX LoeA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: AA+aEWZHeoxAc1F32wHSX+xTtq9g4n5KG+L5AA7p3iAU+DABYCQMpFsC LcOG+tluygqP2OytF3sdy884K7Itcq4qu641jAIwkxYl1eW3tGvWpnt03tHqz2b2FlvdSAeIOIK 3I1NKJwMFRyi1Z38OC9b/Kssr97dyz2SMOa8Mhbxo53ntdeUEoStmfCEauqS35tefcg== X-Received: by 2002:a9d:aa9:: with SMTP id 38mr13885515otq.255.1544028127057; Wed, 05 Dec 2018 08:42:07 -0800 (PST) X-Google-Smtp-Source: AFSGD/VZ826fJh8fjrPdg21ughKJccS0Zbz+cahRPl1/fNy6ygQCWjXpoPUB3xGcNUKFZcoJUk9h X-Received: by 2002:a9d:aa9:: with SMTP id 38mr13885460otq.255.1544028125709; Wed, 05 Dec 2018 08:42:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544028125; cv=none; d=google.com; s=arc-20160816; b=BRBW92uaykdLDO5ZncfTTHt8mRbdJR3M4liFoqt7SXNr9XZcBcuR4xdM7AN/hb2KRk bydgEwt/08nNLNq69BEb2T6JNXE1432xTnmyEWqv60aG2C1uw+D8YNpJ+oo0lQd7CUwA PpVD8+Vs7/dlrgbeNiqB4kvAtiFmdUsauLs29yWVTldNXlaXzAdN6IVLrHrm3aYtixWO zG5AeP/gqGK+HMfIOcJ6zOAEaNqhP3mXFE6Eilk/YBsnRB3zu1fU2nmoXjiVLzOW2xlt vHqNgT+IvCYL5YiUQbTCb1OYg50J/Ky0oNZiRKzkmutI82PXgUWrZrt9dkJHVafMHtJl vCUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Ssm9tJWHFrtRksDx3U9R+7F2PdGwOmZQj+FU4qI3efA=; b=G5zTeqkd8h+iwtf/h5671qm6xE5mreebm9J9J216E5824oESdIC/i0an8DXeiFJdEv 92QdA7wnUl3VATol/8dH8+AbAiTeGx2WcU0ByUxMHEtw3znwbF8J5NMYVZ2jcjFYBXcF edz4QpAegTbFfp4a7zSl25o3nMOclLltGAOpR6gqD5GKFSbVp12VDkmuqc/TNqWwM3hx CO/Ric//xhEvQMESHmHgh3JUtN1SMXsQd5jAl+lTq71hUBKQC0tISuSoQiU+/+XnLjCm x2SjLIjXtle6NP1Yd4GCDplVH0sw/+KxBnLzDtVoMHjxCS+e9ukrU/45ICbAK7YTeD8v PZcg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id j62si9055126oia.268.2018.12.05.08.42.05 for ; Wed, 05 Dec 2018 08:42:05 -0800 (PST) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 19F5315AD; Wed, 5 Dec 2018 08:42:00 -0800 (PST) Received: from capper-debian.emea.arm.com (C02R32KKFVH8.manchester.arm.com [10.32.102.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A2EE03F5AF; Wed, 5 Dec 2018 08:41:58 -0800 (PST) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper , Andrew Morton Subject: [PATCH V4 1/6] mm: mmap: Allow for "high" userspace addresses Date: Wed, 5 Dec 2018 16:41:40 +0000 Message-Id: <20181205164145.24568-2-steve.capper@arm.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181205164145.24568-1-steve.capper@arm.com> References: <20181205164145.24568-1-steve.capper@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds support for "high" userspace addresses that are optionally supported on the system and have to be requested via a hint mechanism ("high" addr parameter to mmap). Architectures such as powerpc and x86 achieve this by making changes to their architectural versions of arch_get_unmapped_* functions. However, on arm64 we use the generic versions of these functions. Rather than duplicate the generic arch_get_unmapped_* implementations for arm64, this patch instead introduces two architectural helper macros and applies them to arch_get_unmapped_*: arch_get_mmap_end(addr) - get mmap upper limit depending on addr hint arch_get_mmap_base(addr, base) - get mmap_base depending on addr hint If these macros are not defined in architectural code then they default to (TASK_SIZE) and (base) so should not introduce any behavioural changes to architectures that do not define them. Signed-off-by: Steve Capper Reviewed-by: Catalin Marinas Cc: Andrew Morton --- Changed in V4, added Catalin's reviewed by Changed in V3, commit log cleared up, explanation given for why core code change over just architectural change --- mm/mmap.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 6c04292e16a7..7bb64381e77c 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2066,6 +2066,15 @@ unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info) return gap_end; } + +#ifndef arch_get_mmap_end +#define arch_get_mmap_end(addr) (TASK_SIZE) +#endif + +#ifndef arch_get_mmap_base +#define arch_get_mmap_base(addr, base) (base) +#endif + /* Get an address range which is currently unmapped. * For shmat() with addr=0. * @@ -2085,8 +2094,9 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, struct mm_struct *mm = current->mm; struct vm_area_struct *vma, *prev; struct vm_unmapped_area_info info; + const unsigned long mmap_end = arch_get_mmap_end(addr); - if (len > TASK_SIZE - mmap_min_addr) + if (len > mmap_end - mmap_min_addr) return -ENOMEM; if (flags & MAP_FIXED) @@ -2095,7 +2105,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, if (addr) { addr = PAGE_ALIGN(addr); vma = find_vma_prev(mm, addr, &prev); - if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && + if (mmap_end - len >= addr && addr >= mmap_min_addr && (!vma || addr + len <= vm_start_gap(vma)) && (!prev || addr >= vm_end_gap(prev))) return addr; @@ -2104,7 +2114,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, info.flags = 0; info.length = len; info.low_limit = mm->mmap_base; - info.high_limit = TASK_SIZE; + info.high_limit = mmap_end; info.align_mask = 0; return vm_unmapped_area(&info); } @@ -2124,9 +2134,10 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, struct mm_struct *mm = current->mm; unsigned long addr = addr0; struct vm_unmapped_area_info info; + const unsigned long mmap_end = arch_get_mmap_end(addr); /* requested length too big for entire address space */ - if (len > TASK_SIZE - mmap_min_addr) + if (len > mmap_end - mmap_min_addr) return -ENOMEM; if (flags & MAP_FIXED) @@ -2136,7 +2147,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, if (addr) { addr = PAGE_ALIGN(addr); vma = find_vma_prev(mm, addr, &prev); - if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && + if (mmap_end - len >= addr && addr >= mmap_min_addr && (!vma || addr + len <= vm_start_gap(vma)) && (!prev || addr >= vm_end_gap(prev))) return addr; @@ -2145,7 +2156,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, info.flags = VM_UNMAPPED_AREA_TOPDOWN; info.length = len; info.low_limit = max(PAGE_SIZE, mmap_min_addr); - info.high_limit = mm->mmap_base; + info.high_limit = arch_get_mmap_base(addr, mm->mmap_base); info.align_mask = 0; addr = vm_unmapped_area(&info); @@ -2159,7 +2170,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, VM_BUG_ON(addr != -ENOMEM); info.flags = 0; info.low_limit = TASK_UNMAPPED_BASE; - info.high_limit = TASK_SIZE; + info.high_limit = mmap_end; addr = vm_unmapped_area(&info); } From patchwork Wed Dec 5 16:41:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10714595 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FA2E13BB for ; Wed, 5 Dec 2018 16:42:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4EFFB2DE00 for ; Wed, 5 Dec 2018 16:42:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4D57C2DD16; Wed, 5 Dec 2018 16:42:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29CC42DD16 for ; Wed, 5 Dec 2018 16:42:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2ED1F6B7548; Wed, 5 Dec 2018 11:42:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 29AA26B754C; Wed, 5 Dec 2018 11:42:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18B286B754D; Wed, 5 Dec 2018 11:42:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by kanga.kvack.org (Postfix) with ESMTP id E24416B7548 for ; Wed, 5 Dec 2018 11:42:04 -0500 (EST) Received: by mail-oi1-f199.google.com with SMTP id t184so12456484oih.22 for ; Wed, 05 Dec 2018 08:42:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=daHBFJzzY00gC37Wi5yuwHKdc1YnCz4b0ak17N5SJgs=; b=aqElE17/Acg43+46FpkPLYkaWTC7O9xRctjgm0XuoUy7oQXKS09QR6bvFSlV9X3VXu 43K3BS8GXgfdovl3HjDr6yHhw9KmuB9M1jcmxn0DovbHoORh/b5lz3fKx8VZo+CKknJe 90Cy5C+iZNrWOQoB+rK6Bhvme2rWOHd598KGU+DHADKG2PwUZCcxZz5btpAiqR7GUHoo JWyCQC8mnw9jBwTDV1IjSNR38lLZI3Z29zXERCi0PPnCDXnTEQspNOLDywhU/lg9jTIC uD+ldAmbDykOVQYH2m49UfdePbSJU3K2OLyk+VIS/3LIcgp2U4EzwmYQof483hFKP+Bs BjKg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: AA+aEWZGToblh3eEgv7oHbnfz15LBv2XfCBpcvG2bfzvusEBChYYfHf7 Nek1/q34kkHeZ22PwoJYMbqsKcLvPF6kDBe5TS2lILKzWxzesfhLurmxE4/6PtL6GvBqQ0nVeuu sDkmRGSP2Mr/ihOt4otHkeDqXMamH5pezl6d3aJZkrl7Wz1YlmhoyE45L8knyxH4+jg== X-Received: by 2002:a9d:22ea:: with SMTP id y97mr14470090ota.311.1544028124603; Wed, 05 Dec 2018 08:42:04 -0800 (PST) X-Google-Smtp-Source: AFSGD/UXiihPu9wISyLEOfF3AKlr0458snneRIZHP8FBjaN/4ZcpruTBLxVku+9ggfN9Ft6r+uDf X-Received: by 2002:a9d:22ea:: with SMTP id y97mr14470032ota.311.1544028123085; Wed, 05 Dec 2018 08:42:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544028123; cv=none; d=google.com; s=arc-20160816; b=C1HeIO7n3qxsjRhlYIsB1cpE1dQdbRusI2zKwvebylRlHXSz1aTpsV3OuwzcQ3xyr6 wtGwhjARA1b9HHFU2kU9dmfYUlU8N4y1ifZNF+lIS1EF/l0PDiIufyCycU1ALhVGqu+4 OUS9wxi5uhX1YQxMk/w5q3m4gYf1aeu9Kny2jxtOc5/hnhZHkRKNWm2SNOEFHnVq+63e HJ02Rng/fHzOWmcGPDFiKZd5XXWwC9rdluO3aOaAJ6EVtErn02UdX5XNW7M0CyPrhWfP OCpWdnDlP42sLlh/ST2CeyARJbUKAJ3Zh6s+qHqhsHgDd/5Ekxs7Svc6REjGALAstPwA GIEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=daHBFJzzY00gC37Wi5yuwHKdc1YnCz4b0ak17N5SJgs=; b=Ns22H4NSw9v4C7c4Xi4b8pZFVAGXhoZhVYQGHrvYUf6L76r98IhT6JhgRruSe73AwH xzWrw7PQ3g/sospsgVI3WNuKSZNy2pw5/Mytp9kI9s62U2tZ17ehEgyGcm+bBwwFPuOw K8ssRIzPX072A9kVUQ0xUBpyzzWfsEtTe8Nc2ciwwUyw6uJa2LOH8XAZCC83KPsehy4l EcjY025Elbvh1DJcddK6r+AKopjq9+0MUjkTD2aZ8R+lXhToASCVgCmgMQXGvKAtl84i 7WSrjpBI4KoPQhHyBbzBf2DikiekXxzW+x0jXCTtxLDXRs6hKZ+4B+7OndKj+1cJcsiG J2MA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id d64si9016062oia.186.2018.12.05.08.42.02 for ; Wed, 05 Dec 2018 08:42:03 -0800 (PST) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 607CA1650; Wed, 5 Dec 2018 08:42:02 -0800 (PST) Received: from capper-debian.emea.arm.com (C02R32KKFVH8.manchester.arm.com [10.32.102.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1A8A13F5AF; Wed, 5 Dec 2018 08:42:00 -0800 (PST) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V4 2/6] arm64: mm: Introduce DEFAULT_MAP_WINDOW Date: Wed, 5 Dec 2018 16:41:41 +0000 Message-Id: <20181205164145.24568-3-steve.capper@arm.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181205164145.24568-1-steve.capper@arm.com> References: <20181205164145.24568-1-steve.capper@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We wish to introduce a 52-bit virtual address space for userspace but maintain compatibility with software that assumes the maximum VA space size is 48 bit. In order to achieve this, on 52-bit VA systems, we make mmap behave as if it were running on a 48-bit VA system (unless userspace explicitly requests a VA where addr[51:48] != 0). On a system running a 52-bit userspace we need TASK_SIZE to represent the 52-bit limit as it is used in various places to distinguish between kernelspace and userspace addresses. Thus we need a new limit for mmap, stack, ELF loader and EFI (which uses TTBR0) to represent the non-extended VA space. This patch introduces DEFAULT_MAP_WINDOW and DEFAULT_MAP_WINDOW_64 and switches the appropriate logic to use that instead of TASK_SIZE. Signed-off-by: Steve Capper Reviewed-by: Catalin Marinas --- Changed in V3: corrections to allow COMPAT 32-bit EL0 mode to work --- arch/arm64/include/asm/elf.h | 2 +- arch/arm64/include/asm/processor.h | 10 ++++++++-- arch/arm64/mm/init.c | 2 +- drivers/firmware/efi/arm-runtime.c | 2 +- drivers/firmware/efi/libstub/arm-stub.c | 2 +- 5 files changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h index 433b9554c6a1..bc9bd9e77d9d 100644 --- a/arch/arm64/include/asm/elf.h +++ b/arch/arm64/include/asm/elf.h @@ -117,7 +117,7 @@ * 64-bit, this is above 4GB to leave the entire 32-bit address * space open for things that want to use the area for 32-bit pointers. */ -#define ELF_ET_DYN_BASE (2 * TASK_SIZE_64 / 3) +#define ELF_ET_DYN_BASE (2 * DEFAULT_MAP_WINDOW_64 / 3) #ifndef __ASSEMBLY__ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 3e2091708b8e..50586ca6bacb 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -45,19 +45,25 @@ * TASK_SIZE - the maximum size of a user space task. * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area. */ + +#define DEFAULT_MAP_WINDOW_64 (UL(1) << VA_BITS) + #ifdef CONFIG_COMPAT #define TASK_SIZE_32 UL(0x100000000) #define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \ TASK_SIZE_32 : TASK_SIZE_64) #define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \ TASK_SIZE_32 : TASK_SIZE_64) +#define DEFAULT_MAP_WINDOW (test_thread_flag(TIF_32BIT) ? \ + TASK_SIZE_32 : DEFAULT_MAP_WINDOW_64) #else #define TASK_SIZE TASK_SIZE_64 +#define DEFAULT_MAP_WINDOW DEFAULT_MAP_WINDOW_64 #endif /* CONFIG_COMPAT */ -#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4)) +#define TASK_UNMAPPED_BASE (PAGE_ALIGN(DEFAULT_MAP_WINDOW / 4)) +#define STACK_TOP_MAX DEFAULT_MAP_WINDOW_64 -#define STACK_TOP_MAX TASK_SIZE_64 #ifdef CONFIG_COMPAT #define AARCH32_VECTORS_BASE 0xffff0000 #define STACK_TOP (test_thread_flag(TIF_32BIT) ? \ diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 9d9582cac6c4..7239c103be06 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -609,7 +609,7 @@ void __init mem_init(void) * detected at build time already. */ #ifdef CONFIG_COMPAT - BUILD_BUG_ON(TASK_SIZE_32 > TASK_SIZE_64); + BUILD_BUG_ON(TASK_SIZE_32 > DEFAULT_MAP_WINDOW_64); #endif #ifdef CONFIG_SPARSEMEM_VMEMMAP diff --git a/drivers/firmware/efi/arm-runtime.c b/drivers/firmware/efi/arm-runtime.c index 922cfb813109..952cec5b611a 100644 --- a/drivers/firmware/efi/arm-runtime.c +++ b/drivers/firmware/efi/arm-runtime.c @@ -38,7 +38,7 @@ static struct ptdump_info efi_ptdump_info = { .mm = &efi_mm, .markers = (struct addr_marker[]){ { 0, "UEFI runtime start" }, - { TASK_SIZE_64, "UEFI runtime end" } + { DEFAULT_MAP_WINDOW_64, "UEFI runtime end" } }, .base_addr = 0, }; diff --git a/drivers/firmware/efi/libstub/arm-stub.c b/drivers/firmware/efi/libstub/arm-stub.c index 30ac0c975f8a..d1ec7136e3e1 100644 --- a/drivers/firmware/efi/libstub/arm-stub.c +++ b/drivers/firmware/efi/libstub/arm-stub.c @@ -33,7 +33,7 @@ #define EFI_RT_VIRTUAL_SIZE SZ_512M #ifdef CONFIG_ARM64 -# define EFI_RT_VIRTUAL_LIMIT TASK_SIZE_64 +# define EFI_RT_VIRTUAL_LIMIT DEFAULT_MAP_WINDOW_64 #else # define EFI_RT_VIRTUAL_LIMIT TASK_SIZE #endif From patchwork Wed Dec 5 16:41:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10714597 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1347413BB for ; Wed, 5 Dec 2018 16:42:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 026A82DDB9 for ; Wed, 5 Dec 2018 16:42:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 008572DE0E; Wed, 5 Dec 2018 16:42:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FC6B2DE00 for ; Wed, 5 Dec 2018 16:42:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A41976B754C; Wed, 5 Dec 2018 11:42:06 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9A53C6B754D; Wed, 5 Dec 2018 11:42:06 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A6C46B754E; Wed, 5 Dec 2018 11:42:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f200.google.com (mail-oi1-f200.google.com [209.85.167.200]) by kanga.kvack.org (Postfix) with ESMTP id 596B96B754C for ; Wed, 5 Dec 2018 11:42:06 -0500 (EST) Received: by mail-oi1-f200.google.com with SMTP id r82so12783430oie.14 for ; Wed, 05 Dec 2018 08:42:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ym97/tP6J2zubdqpy4pD5UshmSnQ3v6bkb87/PGhIaA=; b=IGwPzMRN0LRAB5HxA+80r7HSXeRiCNnd6LImzVLy8zR+d0CppQMCjta2Pj6md/uRsH tM6d0THyN51uSLIfXL+lZKLeicD64+MKQdcuzKqQZnCq5QDqtb3l9FI665NRGAk6UDS+ Et1LUTv8R4mwd0ZVfT7ZMaphDJRst5beYKDfINof9txXaWy4YkM+lcI6OlkXDl5rsb/v A7IWEEayPf1VSjtp0us2w/2JGMlZZbvyOjyOEBeYiHOAYRoDDuTfE1hjkNDo48Dd24Hw N06ynPs54j9UBeFJdjQG2UwK9ga0nRnG4FOQGBTXcRxUJ7HEgglcDsfVA16uFZiCjDN5 w+1A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: AA+aEWYimEh1D5bONXCVr1SMO9zkPJqitzZbeSxxkpp/sprLRFnzkdUA IlOmmEdhDYN9U1v88rr++ykx5vnKHJjd4sK2DD6C4ZAt2frsTiThX6brLhNoM0dANRZ5VikEgru j5MWdJT+oBO4Sy04Oye/EONoV3b6W2qKtDzfsOnMhmMRPM6ZDxfWSal4Yz6n92EnrtQ== X-Received: by 2002:a9d:4e06:: with SMTP id p6mr16997483otf.73.1544028126101; Wed, 05 Dec 2018 08:42:06 -0800 (PST) X-Google-Smtp-Source: AFSGD/XIsTBPIrcI71Q8YehGKf2ZXChfraCzSblhygzO/T4U8xorkrpAdLYmd7QzI+D/XCEE8rtB X-Received: by 2002:a9d:4e06:: with SMTP id p6mr16997450otf.73.1544028125282; Wed, 05 Dec 2018 08:42:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544028125; cv=none; d=google.com; s=arc-20160816; b=VpvcaCVpQV0NfSusL0z5qYAakLm5+Z8qYGr+fIFOu9ptMa2SN3qyqG0Njem18luJDb /au1VYPMWMaGfHwiNAgtJ97gKfmreca3NjsLnNzhhn3g42fu2vFjKFrZfV9rWC6bgO5M RxnUurI8EJ3myd7vZlyofMU9Sh6XCzrYvmZq1rlyAs9xApvoXKSiagU2A2nKiCwBHE4m 1dvqCP0o1DVXwsE8wAPpZH78wmfejdBMNQHbR2usg2S6iVDZpGq1Fncr3nLBqX5eGHuJ etJh1y7GaDe5kKzivdZh5thAxpC8vxyCDMV2C8EKtE6KiTIf8FouBlJyCbjN8UKMIE3b K+0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ym97/tP6J2zubdqpy4pD5UshmSnQ3v6bkb87/PGhIaA=; b=fA/teMdJsHfnuFHf43SrnlH0C6dnMP6TWBRQH4oB851efpJ/poTJXIN7jN/doJ1rmX 7k8wUXHuaD7ptUTmEaRju9G938SQwqlN0muYM7JwYlq1w7bzMyWgIbb+TVEkfSgId0pk ZwbfYDI7I5uC9z6kuWt4Mt1uzh5hOgRxEB4rjyZEufN2Q2EBqB9Cc3Nc+lha/9O8Cdhy qzKVa9+X2AXIRs99UVssG76mGWYcbp3UsQjnTIxWcI+dNPElaTJ1YTBpbzAMJe2jXR32 3xOc+Lmr+40Gj4aGuIdPFbQlIc5iry8JyDGw/A3TOjXA1vLLAd+UC0TnQ5ShDj2xIYTE Vq0g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id y8si11921497otb.143.2018.12.05.08.42.05 for ; Wed, 05 Dec 2018 08:42:05 -0800 (PST) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A898C1682; Wed, 5 Dec 2018 08:42:04 -0800 (PST) Received: from capper-debian.emea.arm.com (C02R32KKFVH8.manchester.arm.com [10.32.102.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 63BC33F5AF; Wed, 5 Dec 2018 08:42:03 -0800 (PST) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V4 3/6] arm64: mm: Define arch_get_mmap_end, arch_get_mmap_base Date: Wed, 5 Dec 2018 16:41:42 +0000 Message-Id: <20181205164145.24568-4-steve.capper@arm.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181205164145.24568-1-steve.capper@arm.com> References: <20181205164145.24568-1-steve.capper@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now that we have DEFAULT_MAP_WINDOW defined, we can arch_get_mmap_end and arch_get_mmap_base helpers to allow for high addresses in mmap. Signed-off-by: Steve Capper Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/processor.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 50586ca6bacb..fe95fd8b065e 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -72,6 +72,13 @@ #define STACK_TOP STACK_TOP_MAX #endif /* CONFIG_COMPAT */ +#define arch_get_mmap_end(addr) ((addr > DEFAULT_MAP_WINDOW) ? TASK_SIZE :\ + DEFAULT_MAP_WINDOW) + +#define arch_get_mmap_base(addr, base) ((addr > DEFAULT_MAP_WINDOW) ? \ + base + TASK_SIZE - DEFAULT_MAP_WINDOW :\ + base) + extern phys_addr_t arm64_dma_phys_limit; #define ARCH_LOW_ADDRESS_LIMIT (arm64_dma_phys_limit - 1) From patchwork Wed Dec 5 16:41:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10714603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7A1613BB for ; Wed, 5 Dec 2018 16:42:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A74192DDC6 for ; Wed, 5 Dec 2018 16:42:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 915982DE49; Wed, 5 Dec 2018 16:42:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD6252DDC6 for ; Wed, 5 Dec 2018 16:42:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 251C96B754E; Wed, 5 Dec 2018 11:42:09 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1DBBF6B754F; Wed, 5 Dec 2018 11:42:09 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0033C6B7550; Wed, 5 Dec 2018 11:42:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f200.google.com (mail-oi1-f200.google.com [209.85.167.200]) by kanga.kvack.org (Postfix) with ESMTP id CE1216B754E for ; Wed, 5 Dec 2018 11:42:08 -0500 (EST) Received: by mail-oi1-f200.google.com with SMTP id 73so12499504oii.12 for ; Wed, 05 Dec 2018 08:42:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lFR32fSr1ly6v5yn7Uw54cPiu8J3gkqBA0EbNA51000=; b=UY8Iqpth+Fm9j3p03sb6Oz43SkruxwtQIASHjx2kytnQvjwDqBFg7FDhF2NfVYmMFn UHxpdO4GuKsRV2G/mG6NL0Ymz0YqGXIb50pDcgqog1n3GYaFQfEZQPgXUOPshjrX3Dp2 VLIziJToyesBJ9WFfABLZ701GxlH5ArQFSJN+7sbhp6N8DHUoPkc7XoVbfPAGp7nInJi IlbfmuPhlj9bk79Ngw49XUSAxAubJSYTtylnmpMInC9anSbwubT8OyPTA9RjWAif5/Fv tQywI/nnJHytCPoE7T+WneD5av5e/AOaGuQ063bdhDGIJiv1w+fEMv4ngvYqzNzEKcDh hpXA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: AA+aEWa7gmhwcjyCoLpX3sWBJYxgqoRPns1etX6BWdZNmaJLDSzkf7SC 82Y5WhFwgHc6UDCEBjRX3UuxHfME2zv75cUrp1jUFLZ4YtOPHST33znyIeR21FJe2MFP7l3IMbn bNun3bmr6d86c/nlg7GytR+bnzriMzfaCur8BEW3oZv4/rpEGew6yOt/TGz3vqetXOg== X-Received: by 2002:aca:ce86:: with SMTP id e128mr16092550oig.247.1544028128551; Wed, 05 Dec 2018 08:42:08 -0800 (PST) X-Google-Smtp-Source: AFSGD/UeOxFgedcT3fEBy4mh+glAvHWWGUkSwx4CKoxw0k+P+HyWE8GYcL/axBxX6ZEwej6+pbi8 X-Received: by 2002:aca:ce86:: with SMTP id e128mr16092505oig.247.1544028127584; Wed, 05 Dec 2018 08:42:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544028127; cv=none; d=google.com; s=arc-20160816; b=U82iW8pdEav4Q/fZ8Q0euQGQBv/hpPv958ibY8wCPUXT/6jPZHxkwm+smvOwNv8Hu/ mVE6UYFvI3MxqIpT081P7AE+NmdH+u7Maz0IjuIj7dzcr8M8f2G8/xE1N8sguepQNF4V O25pAl18Udzxhqq+s3JPGDYZXUmvoWpxUduTLO14A0p00yHBbLVGTaJxfftqKppDrUm6 6qIIht/1U+aIRKggG9W/3WGP2MPKE1lBZaFMEd1CPqORoa1YDNpHNnovZptrKR2ol4f6 2Wssgh59anG2/iuIMlZD3ivvAIR/KQg/lP3p1GbuEqb0f1MVuzWj6QiCBQoukJM9JK9k GfdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=lFR32fSr1ly6v5yn7Uw54cPiu8J3gkqBA0EbNA51000=; b=fecuoGkPPDXLOWUGZKhuLhaLD5e7HEKic47almHtxlbbS4gcuzWFkVvECjj6l/EcGz CdgKELkfFjrMso1RSjBj2MDuEr6p0TG5p1tPtkSMKdLQrNGM/9BISOPjKDowshHgV/J1 kYeGjKFfruS/ZhOY36r7te9bLKqqWSWeT6H9XADLjeORNuSPeVPXwWk7jrFTeIVWWFlF ZR3uEANlCs7KKpHFY579rTxP69aA5LNySdD6YghTfhugwQg7rdqZl6kZmgGesX8Rl49n 7elHUxL8BkEZVwak5lHYAecJ4LtQ9/t9md5Y38VNgsD7Uma4hcfyYOoVSzQ4EbPAl3sl 90lA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id x188si9127266oix.138.2018.12.05.08.42.07 for ; Wed, 05 Dec 2018 08:42:07 -0800 (PST) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DECAC168F; Wed, 5 Dec 2018 08:42:06 -0800 (PST) Received: from capper-debian.emea.arm.com (C02R32KKFVH8.manchester.arm.com [10.32.102.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 95B283F5AF; Wed, 5 Dec 2018 08:42:05 -0800 (PST) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V4 4/6] arm64: mm: Offset TTBR1 to allow 52-bit PTRS_PER_PGD Date: Wed, 5 Dec 2018 16:41:43 +0000 Message-Id: <20181205164145.24568-5-steve.capper@arm.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181205164145.24568-1-steve.capper@arm.com> References: <20181205164145.24568-1-steve.capper@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Enabling 52-bit VAs on arm64 requires that the PGD table expands from 64 entries (for the 48-bit case) to 1024 entries. This quantity, PTRS_PER_PGD is used as follows to compute which PGD entry corresponds to a given virtual address, addr: pgd_index(addr) -> (addr >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1) Userspace addresses are prefixed by 0's, so for a 48-bit userspace address, uva, the following is true: (uva >> PGDIR_SHIFT) & (1024 - 1) == (uva >> PGDIR_SHIFT) & (64 - 1) In other words, a 48-bit userspace address will have the same pgd_index when using PTRS_PER_PGD = 64 and 1024. Kernel addresses are prefixed by 1's so, given a 48-bit kernel address, kva, we have the following inequality: (kva >> PGDIR_SHIFT) & (1024 - 1) != (kva >> PGDIR_SHIFT) & (64 - 1) In other words a 48-bit kernel virtual address will have a different pgd_index when using PTRS_PER_PGD = 64 and 1024. If, however, we note that: kva = 0xFFFF << 48 + lower (where lower[63:48] == 0b) and, PGDIR_SHIFT = 42 (as we are dealing with 64KB PAGE_SIZE) We can consider: (kva >> PGDIR_SHIFT) & (1024 - 1) - (kva >> PGDIR_SHIFT) & (64 - 1) = (0xFFFF << 6) & 0x3FF - (0xFFFF << 6) & 0x3F // "lower" cancels out = 0x3C0 In other words, one can switch PTRS_PER_PGD to the 52-bit value globally provided that they increment ttbr1_el1 by 0x3C0 * 8 = 0x1E00 bytes when running with 48-bit kernel VAs (TCR_EL1.T1SZ = 16). For kernel configuration where 52-bit userspace VAs are possible, this patch offsets ttbr1_el1 and sets PTRS_PER_PGD corresponding to the 52-bit value. Suggested-by: Catalin Marinas Signed-off-by: Steve Capper --- This patch is new in V4 of the series --- arch/arm64/include/asm/asm-uaccess.h | 4 ++++ arch/arm64/include/asm/assembler.h | 23 +++++++++++++++++++++++ arch/arm64/include/asm/pgtable-hwdef.h | 9 +++++++++ arch/arm64/include/asm/uaccess.h | 4 ++++ arch/arm64/kernel/head.S | 1 + arch/arm64/kernel/hibernate-asm.S | 1 + arch/arm64/mm/proc.S | 4 ++++ 7 files changed, 46 insertions(+) diff --git a/arch/arm64/include/asm/asm-uaccess.h b/arch/arm64/include/asm/asm-uaccess.h index 4128bec033f6..cd361dd16b12 100644 --- a/arch/arm64/include/asm/asm-uaccess.h +++ b/arch/arm64/include/asm/asm-uaccess.h @@ -14,11 +14,13 @@ #ifdef CONFIG_ARM64_SW_TTBR0_PAN .macro __uaccess_ttbr0_disable, tmp1 mrs \tmp1, ttbr1_el1 // swapper_pg_dir + restore_ttbr1 \tmp1 bic \tmp1, \tmp1, #TTBR_ASID_MASK sub \tmp1, \tmp1, #RESERVED_TTBR0_SIZE // reserved_ttbr0 just before swapper_pg_dir msr ttbr0_el1, \tmp1 // set reserved TTBR0_EL1 isb add \tmp1, \tmp1, #RESERVED_TTBR0_SIZE + offset_ttbr1 \tmp1 msr ttbr1_el1, \tmp1 // set reserved ASID isb .endm @@ -27,8 +29,10 @@ get_thread_info \tmp1 ldr \tmp1, [\tmp1, #TSK_TI_TTBR0] // load saved TTBR0_EL1 mrs \tmp2, ttbr1_el1 + restore_ttbr1 \tmp2 extr \tmp2, \tmp2, \tmp1, #48 ror \tmp2, \tmp2, #16 + offset_ttbr1 \tmp2 msr ttbr1_el1, \tmp2 // set the active ASID isb msr ttbr0_el1, \tmp1 // set the non-PAN TTBR0_EL1 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6142402c2eb4..e2fe378d2a63 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -515,6 +515,29 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU mrs \rd, sp_el0 .endm +/* + * Offset ttbr1 to allow for 48-bit kernel VAs set with 52-bit PTRS_PER_PGD. + * orr is used as it can cover the immediate value (and is idempotent). + * In future this may be nop'ed out when dealing with 52-bit kernel VAs. + * ttbr: Value of ttbr to set, modified. + */ + .macro offset_ttbr1, ttbr +#ifdef CONFIG_ARM64_52BIT_VA + orr \ttbr, \ttbr, #TTBR1_BADDR_4852_OFFSET +#endif + .endm + +/* + * Perform the reverse of offset_ttbr1. + * bic is used as it can cover the immediate value and, in future, won't need + * to be nop'ed out when dealing with 52-bit kernel VAs. + */ + .macro restore_ttbr1, ttbr +#ifdef CONFIG_ARM64_52BIT_VA + bic \ttbr, \ttbr, #TTBR1_BADDR_4852_OFFSET +#endif + .endm + /* * Arrange a physical address in a TTBR register, taking care of 52-bit * addresses. diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 1d7d8da2ef9b..4a29c7e03ae4 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -80,7 +80,11 @@ #define PGDIR_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - CONFIG_PGTABLE_LEVELS) #define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) #define PGDIR_MASK (~(PGDIR_SIZE-1)) +#ifdef CONFIG_ARM64_52BIT_VA +#define PTRS_PER_PGD (1 << (52 - PGDIR_SHIFT)) +#else #define PTRS_PER_PGD (1 << (VA_BITS - PGDIR_SHIFT)) +#endif /* * Section address mask and size definitions. @@ -306,4 +310,9 @@ #define TTBR_BADDR_MASK_52 (((UL(1) << 46) - 1) << 2) #endif +#ifdef CONFIG_ARM64_52BIT_VA +#define TTBR1_BADDR_4852_OFFSET (((UL(1) << (52 - PGDIR_SHIFT)) - \ + (UL(1) << (48 - PGDIR_SHIFT))) * 8) +#endif + #endif diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 07c34087bd5e..df60b3978568 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -124,7 +124,11 @@ static inline void __uaccess_ttbr0_disable(void) ttbr = read_sysreg(ttbr1_el1); ttbr &= ~TTBR_ASID_MASK; /* reserved_ttbr0 placed before swapper_pg_dir */ +#ifdef CONFIG_ARM64_52BIT_VA + write_sysreg((ttbr & ~TTBR1_BADDR_4852_OFFSET) - RESERVED_TTBR0_SIZE, ttbr0_el1); +#else write_sysreg(ttbr - RESERVED_TTBR0_SIZE, ttbr0_el1); +#endif isb(); /* Set reserved ASID */ write_sysreg(ttbr, ttbr1_el1); diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 4471f570a295..f60081be9a1b 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -769,6 +769,7 @@ ENTRY(__enable_mmu) phys_to_ttbr x1, x1 phys_to_ttbr x2, x2 msr ttbr0_el1, x2 // load TTBR0 + offset_ttbr1 x1 msr ttbr1_el1, x1 // load TTBR1 isb msr sctlr_el1, x0 diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index dd14ab8c9f72..fe36d85c60bd 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -40,6 +40,7 @@ tlbi vmalle1 dsb nsh phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp msr ttbr1_el1, \tmp isb .endm diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 2c75b0b903ae..2db1c491d45d 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -182,6 +182,7 @@ ENDPROC(cpu_do_switch_mm) .macro __idmap_cpu_set_reserved_ttbr1, tmp1, tmp2 adrp \tmp1, empty_zero_page phys_to_ttbr \tmp2, \tmp1 + offset_ttbr1 \tmp2 msr ttbr1_el1, \tmp2 isb tlbi vmalle1 @@ -200,6 +201,7 @@ ENTRY(idmap_cpu_replace_ttbr1) __idmap_cpu_set_reserved_ttbr1 x1, x3 + offset_ttbr1 x0 msr ttbr1_el1, x0 isb @@ -254,6 +256,7 @@ ENTRY(idmap_kpti_install_ng_mappings) pte .req x16 mrs swapper_ttb, ttbr1_el1 + restore_ttbr1 swapper_ttb adr flag_ptr, __idmap_kpti_flag cbnz cpu, __idmap_kpti_secondary @@ -373,6 +376,7 @@ __idmap_kpti_secondary: cbnz w18, 1b /* All done, act like nothing happened */ + offset_ttbr1 swapper_ttb msr ttbr1_el1, swapper_ttb isb ret From patchwork Wed Dec 5 16:41:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10714605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 936A1109C for ; Wed, 5 Dec 2018 16:42:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 837ED2DE4F for ; Wed, 5 Dec 2018 16:42:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 771272DE51; Wed, 5 Dec 2018 16:42:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9264E2DE4A for ; Wed, 5 Dec 2018 16:42:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EABE86B754F; Wed, 5 Dec 2018 11:42:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E5CC66B7550; Wed, 5 Dec 2018 11:42:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D27916B7551; Wed, 5 Dec 2018 11:42:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot1-f69.google.com (mail-ot1-f69.google.com [209.85.210.69]) by kanga.kvack.org (Postfix) with ESMTP id A6EC76B754F for ; Wed, 5 Dec 2018 11:42:11 -0500 (EST) Received: by mail-ot1-f69.google.com with SMTP id q23so9616353otn.3 for ; Wed, 05 Dec 2018 08:42:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ogv68o3rEOKKpDT+i3n0/thFEhNapc6azfHAOXdiFNA=; b=SKT411V7MnyYk7DWge9JtnG4/GxM2bQ90rsqVGIT8vLxmua0GcIkor4lJD0+9GMJFj 46xMCk4fCgyhw+c+kA2JYRyYrQYR9IUfsoeVjsnKIpk2RarhttJtXu3mSpLK3n0EWW3/ IMAm7r/OxCLeWOIJIMEFp59m22UVSdiaC/94LI/qeL8mc04Xf4Z6ehRtB6mSdd6GfwnH IoEicGv9QitpisntOGCdddOybsRrXxS0EC84y0piB245pNrPgCG5uHhZ7UbMo8A3hoVL ioZuqv2C0jKU5Jgc0VfoUDY5v821/yPOXbIRRquas8S9mzU3frxoJS7OKejjLtUpGTzm o8qg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: AA+aEWaS4Den0xzlZivsh7YG0KLORJCjIOxkl6HRmKeby2lhhDrlvwmB Vi9Djp8AA/XHPOBkgD5fNjelDsW75F0X9oQ9NQkUBqpNaDS/gMoCQpkjeVJptM6h+SakhW7kEFd mIWD2qcK3h2vZ6qKvvJt5KIRyUdjmbYRahlywYIEdUjzQBrm7eJoWq5qrFtBEc/XdAw== X-Received: by 2002:aca:5c4:: with SMTP id 187mr12685465oif.20.1544028131367; Wed, 05 Dec 2018 08:42:11 -0800 (PST) X-Google-Smtp-Source: AFSGD/U+TTzfbcpVfAfKTalRETmT+D+BJIrv4jTDTdax9FB19UQmArHXQK20kz8ew02P55TtcEEk X-Received: by 2002:aca:5c4:: with SMTP id 187mr12685408oif.20.1544028129759; Wed, 05 Dec 2018 08:42:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544028129; cv=none; d=google.com; s=arc-20160816; b=ahJKNw3PJr9aPqvHvhvN732FipjftVO6sCtCcz8tt4r7H/+L9ciLm4Au6iH3VbfnUd Iyc0Aw7DYPlsTxAMSf8tXMdJg9PqxtjaHQweWzAyhyTPRq5lZrMbGctYqguXM42AzhUy BQnZFwsJEgWfB42IWcYj9gw4QsWUlkrc7lI9s7j2loL7mTh/4b8M9ZQIIklpMrZgaWtW TdzR6pWXNrVI7cxI0EO5NhE0xLwbeJWqDN6xKalxgWytLvB6h9eqM8Dviiq506Q4LQfn lfGN8IyUDt5r6Hke/L60kczHYEV0P/Vc+8TgslR40m7zbaseX5kIWOmg0pvYCUso9N9K 2fEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ogv68o3rEOKKpDT+i3n0/thFEhNapc6azfHAOXdiFNA=; b=ROHCVMUpAY+Lick176mmjiq6eruOUB6kKt1vKTrwoT3EP7PFAHqcbUQTgEigk9Mb4C fiJDVr+GKIuLuSX3LMbnFfTCWXd6B01Yj9YfrE2Kk+Oz6AgmcuZfv7hdRNCNOiZPiTxZ 2AU/egBuaCJAo6kwfbINzGCLpQpVonMvcy9hmroFrwpkLTFloDgB7VFsoyyVPfoE4R03 mnpOi1V6qn2TW0MeZxxkWfxa3veKzLNsNvVw0jMtrl4teErr4cN8udXLTOHvbVT+SBE1 ZYYtcEs6HX7tNEAqlo+INJ/5LXmNNvGvqoomL3lEf05+WpSJLLgxuikkZMdJOozQElTx gIvA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id e24si9611951otl.194.2018.12.05.08.42.09 for ; Wed, 05 Dec 2018 08:42:09 -0800 (PST) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 183C2169E; Wed, 5 Dec 2018 08:42:09 -0800 (PST) Received: from capper-debian.emea.arm.com (C02R32KKFVH8.manchester.arm.com [10.32.102.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C64253F5AF; Wed, 5 Dec 2018 08:42:07 -0800 (PST) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V4 5/6] arm64: mm: introduce 52-bit userspace support Date: Wed, 5 Dec 2018 16:41:44 +0000 Message-Id: <20181205164145.24568-6-steve.capper@arm.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181205164145.24568-1-steve.capper@arm.com> References: <20181205164145.24568-1-steve.capper@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On arm64 there is optional support for a 52-bit virtual address space. To exploit this one has to be running with a 64KB page size and be running on hardware that supports this. For an arm64 kernel supporting a 48 bit VA with a 64KB page size, some changes are needed to support a 52-bit userspace: * TCR_EL1.T0SZ needs to be 12 instead of 16, * TASK_SIZE needs to reflect the new size. This patch implements the above when the support for 52-bit VAs is detected at early boot time. On arm64 userspace addresses translation is controlled by TTBR0_EL1. As well as userspace, TTBR0_EL1 controls: * The identity mapping, * EFI runtime code. It is possible to run a kernel with an identity mapping that has a larger VA size than userspace (and for this case __cpu_set_tcr_t0sz() would set TCR_EL1.T0SZ as appropriate). However, when the conditions for 52-bit userspace are met; it is possible to keep TCR_EL1.T0SZ fixed at 12. Thus in this patch, the TCR_EL1.T0SZ size changing logic is disabled. Signed-off-by: Steve Capper --- Changed in V4, pgd_index logic removed as we offset ttbr1 instead --- arch/arm64/Kconfig | 4 ++++ arch/arm64/include/asm/assembler.h | 7 +++---- arch/arm64/include/asm/mmu_context.h | 3 +++ arch/arm64/include/asm/processor.h | 14 +++++++++----- arch/arm64/kernel/head.S | 13 +++++++++++++ arch/arm64/mm/fault.c | 2 +- arch/arm64/mm/mmu.c | 1 + arch/arm64/mm/proc.S | 10 +++++++++- 8 files changed, 43 insertions(+), 11 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 787d7850e064..eab02d24f5d1 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -709,6 +709,10 @@ config ARM64_PA_BITS_52 endchoice +config ARM64_52BIT_VA + def_bool y + depends on ARM64_VA_BITS_48 && ARM64_64K_PAGES + config ARM64_PA_BITS int default 48 if ARM64_PA_BITS_48 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index e2fe378d2a63..243ec4f0c00f 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -342,11 +342,10 @@ alternative_endif .endm /* - * tcr_set_idmap_t0sz - update TCR.T0SZ so that we can load the ID map + * tcr_set_t0sz - update TCR.T0SZ so that we can load the ID map */ - .macro tcr_set_idmap_t0sz, valreg, tmpreg - ldr_l \tmpreg, idmap_t0sz - bfi \valreg, \tmpreg, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH + .macro tcr_set_t0sz, valreg, t0sz + bfi \valreg, \t0sz, #TCR_T0SZ_OFFSET, #TCR_TxSZ_WIDTH .endm /* diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 1e58bf58c22b..b125fafc611b 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -72,6 +72,9 @@ extern u64 idmap_ptrs_per_pgd; static inline bool __cpu_uses_extended_idmap(void) { + if (IS_ENABLED(CONFIG_ARM64_52BIT_VA)) + return false; + return unlikely(idmap_t0sz != TCR_T0SZ(VA_BITS)); } diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index fe95fd8b065e..b363fc705be4 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -19,11 +19,12 @@ #ifndef __ASM_PROCESSOR_H #define __ASM_PROCESSOR_H -#define TASK_SIZE_64 (UL(1) << VA_BITS) - -#define KERNEL_DS UL(-1) -#define USER_DS (TASK_SIZE_64 - 1) - +#define KERNEL_DS UL(-1) +#ifdef CONFIG_ARM64_52BIT_VA +#define USER_DS ((UL(1) << 52) - 1) +#else +#define USER_DS ((UL(1) << VA_BITS) - 1) +#endif /* CONFIG_ARM64_52IT_VA */ #ifndef __ASSEMBLY__ #ifdef __KERNEL__ @@ -48,6 +49,9 @@ #define DEFAULT_MAP_WINDOW_64 (UL(1) << VA_BITS) +extern u64 vabits_user; +#define TASK_SIZE_64 (UL(1) << vabits_user) + #ifdef CONFIG_COMPAT #define TASK_SIZE_32 UL(0x100000000) #define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index f60081be9a1b..5bc776b8ee5e 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -318,6 +318,19 @@ __create_page_tables: adrp x0, idmap_pg_dir adrp x3, __idmap_text_start // __pa(__idmap_text_start) +#ifdef CONFIG_ARM64_52BIT_VA + mrs_s x6, SYS_ID_AA64MMFR2_EL1 + and x6, x6, #(0xf << ID_AA64MMFR2_LVA_SHIFT) + mov x5, #52 + cbnz x6, 1f +#endif + mov x5, #VA_BITS +1: + adr_l x6, vabits_user + str x5, [x6] + dmb sy + dc ivac, x6 // Invalidate potentially stale cache line + /* * VA_BITS may be too small to allow for an ID mapping to be created * that covers system RAM if that is located sufficiently high in the diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 7d9571f4ae3d..5fe6d2e40e9b 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -160,7 +160,7 @@ void show_pte(unsigned long addr) pr_alert("%s pgtable: %luk pages, %u-bit VAs, pgdp = %p\n", mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K, - VA_BITS, mm->pgd); + mm == &init_mm ? VA_BITS : (int) vabits_user, mm->pgd); pgdp = pgd_offset(mm, addr); pgd = READ_ONCE(*pgdp); pr_alert("[%016lx] pgd=%016llx", addr, pgd_val(pgd)); diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 394b8d554def..f8fc393143ea 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -52,6 +52,7 @@ u64 idmap_t0sz = TCR_T0SZ(VA_BITS); u64 idmap_ptrs_per_pgd = PTRS_PER_PGD; +u64 vabits_user __ro_after_init; u64 kimage_voffset __ro_after_init; EXPORT_SYMBOL(kimage_voffset); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 2db1c491d45d..0cf86b17714c 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -450,7 +450,15 @@ ENTRY(__cpu_setup) ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ TCR_TBI0 | TCR_A1 - tcr_set_idmap_t0sz x10, x9 + +#ifdef CONFIG_ARM64_52BIT_VA + ldr_l x9, vabits_user + sub x9, xzr, x9 + add x9, x9, #64 +#else + ldr_l x9, idmap_t0sz +#endif + tcr_set_t0sz x10, x9 /* * Set the IPS bits in TCR_EL1. From patchwork Wed Dec 5 16:41:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Capper X-Patchwork-Id: 10714607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C1C5109C for ; Wed, 5 Dec 2018 16:42:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D7C02DE33 for ; Wed, 5 Dec 2018 16:42:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B48C2DE54; Wed, 5 Dec 2018 16:42:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D5E72DE3A for ; Wed, 5 Dec 2018 16:42:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 536BF6B7550; Wed, 5 Dec 2018 11:42:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4BF546B7551; Wed, 5 Dec 2018 11:42:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 388296B7552; Wed, 5 Dec 2018 11:42:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by kanga.kvack.org (Postfix) with ESMTP id 0A8376B7550 for ; Wed, 5 Dec 2018 11:42:13 -0500 (EST) Received: by mail-oi1-f199.google.com with SMTP id b18so12674905oii.1 for ; Wed, 05 Dec 2018 08:42:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zpJc6dPrTAqjFfHeF4Wbhsj+l8SDpE6gZ8EjXQ6zbPE=; b=M3lMTfEAbEejZgr0ez4J6y0T2UYlFlvI45aIXyZ9U60Q78HptWUMPCoAix7VL2PqNR 8AhdearwTD0GC7BN5LwuZNr2Lv/QuMnSuEOYxDRJqBeHeWNPFf9Zpyv5OdoVXObazriY awqqTkrq08Sx8+1ZF2eXxN/WmWkmGZAoMZQXQmH190G1nbjTgGrbxDZ39YPrKaoe1h7h QEAiq1sL7Luip4LXBjD3i8aLjWet1HQh3wJq7DTk6MP/UgIY/e+HVWbRbQFZnwe+KY1A IF+BcNYn0zUQCnUaOpCM2cTWPNJNS2WraJfXihtB6wdT2awZ5nYbvdMaYy1oCd2lDsy3 PSUg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com X-Gm-Message-State: AA+aEWZxsipNCXk/x7LSYfLwtHLY34mSrFAkMLftLQES+LaSe7n/7cSt P3V/rNKeEhVWm0BGPzgfVPDNfqfk/2Z78sdCLKMS+jTcnlCIovcp4uwMS1MyjTBPKJBwu29R043 fDIDzccfPdnGEnLi8JEQv6wgDZjHYxT+3CwWbhoiKEXm3IyQBxqa9hk+rdl7Okqe2gQ== X-Received: by 2002:a9d:6c3:: with SMTP id 61mr16866616otx.314.1544028132779; Wed, 05 Dec 2018 08:42:12 -0800 (PST) X-Google-Smtp-Source: AFSGD/WoDbNK9IZTgM3HMxTnrsBUW4mX0jSf8Z2pOWHJUmsHiWQElLClZdIgrhMkivjX3TE/P1b5 X-Received: by 2002:a9d:6c3:: with SMTP id 61mr16866575otx.314.1544028131999; Wed, 05 Dec 2018 08:42:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544028131; cv=none; d=google.com; s=arc-20160816; b=KHMLsbQWDE/Noof5ABMNNpdI6Tzo78MZ+CjEhtqWQo2DISpxg4z8dvMjS8U/JnGUvD K1nb8hPHpiIDp3xFj/okqjfH3fbgCiryi6OijBkgxroo7kdNghji4T1uKJ9G9tPy5UYd vxYgBx8YrCeg7uf9ep9sO0qGuvEQZwciK0aUfowIxgM4L5w4+WBNl+kTjS6eh9g60kwN 5wDT/5PPA/YJZWD/u+St1t2TKKBBoS8jlM7r+5sCeSstDrEkzxJhRolJiFw+1SVSGJZH 3RmawcyUJDGtG6A+YcaKNNnH8D61ayzumHe6+dCECoO2FG0tieub/57EanQTqDBGWzPv 75Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=zpJc6dPrTAqjFfHeF4Wbhsj+l8SDpE6gZ8EjXQ6zbPE=; b=frmSwnGyvY3DJ5hmPgXgxbCHmBfNzA39LzjZYKmPJYDS08LmlkKPKjSbQ6YWVo/Bbi LYYmMFouXLibFZcMobiVTtDwIzqas3AkOtGQFORGtY8h1U7oUbxEYwQYoeLeS/511cUz 81lW58zjTjUJ2Quq2Ugjsqx0veiESPepDXdTchHCFvk3GzmcUQd6ewyhkEqq9CVoHOSW CP+ifIs7dzq2Kt2nQhZA9a9xiKceKhn9k8wlhcwOV6kgVk/4jRsHvBaVigb9qUd6TRME KtvFY1NijFroDp3TOK3BBL8//Cski08METWLWuH4xiHFB/qs7nS7WhQMEV4E4TG9ZW27 1/2Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id c189si8702063oia.80.2018.12.05.08.42.11 for ; Wed, 05 Dec 2018 08:42:11 -0800 (PST) Received-SPF: pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steve.capper@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steve.capper@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 57A02A78; Wed, 5 Dec 2018 08:42:11 -0800 (PST) Received: from capper-debian.emea.arm.com (C02R32KKFVH8.manchester.arm.com [10.32.102.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 13AE33F5AF; Wed, 5 Dec 2018 08:42:09 -0800 (PST) From: Steve Capper To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, ard.biesheuvel@linaro.org, jcm@redhat.com, Steve Capper Subject: [PATCH V4 6/6] arm64: mm: Allow forcing all userspace addresses to 52-bit Date: Wed, 5 Dec 2018 16:41:45 +0000 Message-Id: <20181205164145.24568-7-steve.capper@arm.com> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181205164145.24568-1-steve.capper@arm.com> References: <20181205164145.24568-1-steve.capper@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On arm64 52-bit VAs are provided to userspace when a hint is supplied to mmap. This helps maintain compatibility with software that expects at most 48-bit VAs to be returned. In order to help identify software that has 48-bit VA assumptions, this patch allows one to compile a kernel where 52-bit VAs are returned by default on HW that supports it. This feature is intended to be for development systems only. Signed-off-by: Steve Capper Acked-by: Catalin Marinas --- arch/arm64/Kconfig | 13 +++++++++++++ arch/arm64/include/asm/elf.h | 4 ++++ arch/arm64/include/asm/processor.h | 9 ++++++++- 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index eab02d24f5d1..9f50dc8af110 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1165,6 +1165,19 @@ config ARM64_CNP at runtime, and does not affect PEs that do not implement this feature. +config ARM64_FORCE_52BIT + bool "Force 52-bit virtual addresses for userspace" + depends on ARM64_52BIT_VA && EXPERT + help + For systems with 52-bit userspace VAs enabled, the kernel will attempt + to maintain compatibility with older software by providing 48-bit VAs + unless a hint is supplied to mmap. + + This configuration option disables the 48-bit compatibility logic, and + forces all userspace addresses to be 52-bit on HW that supports it. One + should only enable this configuration option for stress testing userspace + memory management code. If unsure say N here. + endmenu config ARM64_SVE diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h index bc9bd9e77d9d..6adc1a90e7e6 100644 --- a/arch/arm64/include/asm/elf.h +++ b/arch/arm64/include/asm/elf.h @@ -117,7 +117,11 @@ * 64-bit, this is above 4GB to leave the entire 32-bit address * space open for things that want to use the area for 32-bit pointers. */ +#ifdef CONFIG_ARM64_FORCE_52BIT +#define ELF_ET_DYN_BASE (2 * TASK_SIZE_64 / 3) +#else #define ELF_ET_DYN_BASE (2 * DEFAULT_MAP_WINDOW_64 / 3) +#endif /* CONFIG_ARM64_FORCE_52BIT */ #ifndef __ASSEMBLY__ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index b363fc705be4..9abd91570b5b 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -65,8 +65,13 @@ extern u64 vabits_user; #define DEFAULT_MAP_WINDOW DEFAULT_MAP_WINDOW_64 #endif /* CONFIG_COMPAT */ -#define TASK_UNMAPPED_BASE (PAGE_ALIGN(DEFAULT_MAP_WINDOW / 4)) +#ifdef CONFIG_ARM64_FORCE_52BIT +#define STACK_TOP_MAX TASK_SIZE_64 +#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4)) +#else #define STACK_TOP_MAX DEFAULT_MAP_WINDOW_64 +#define TASK_UNMAPPED_BASE (PAGE_ALIGN(DEFAULT_MAP_WINDOW / 4)) +#endif /* CONFIG_ARM64_FORCE_52BIT */ #ifdef CONFIG_COMPAT #define AARCH32_VECTORS_BASE 0xffff0000 @@ -76,12 +81,14 @@ extern u64 vabits_user; #define STACK_TOP STACK_TOP_MAX #endif /* CONFIG_COMPAT */ +#ifndef CONFIG_ARM64_FORCE_52BIT #define arch_get_mmap_end(addr) ((addr > DEFAULT_MAP_WINDOW) ? TASK_SIZE :\ DEFAULT_MAP_WINDOW) #define arch_get_mmap_base(addr, base) ((addr > DEFAULT_MAP_WINDOW) ? \ base + TASK_SIZE - DEFAULT_MAP_WINDOW :\ base) +#endif /* CONFIG_ARM64_FORCE_52BIT */ extern phys_addr_t arm64_dma_phys_limit; #define ARCH_LOW_ADDRESS_LIMIT (arm64_dma_phys_limit - 1)