From patchwork Sat Sep 29 21:43:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Hutchings X-Patchwork-Id: 10621015 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B05C014BD for ; Sat, 29 Sep 2018 21:48:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98C8E29A4A for ; Sat, 29 Sep 2018 21:48:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8BB7429A5C; Sat, 29 Sep 2018 21:48:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B64FD29A4A for ; Sat, 29 Sep 2018 21:48:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 265DB8E0003; Sat, 29 Sep 2018 17:48:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1C4998E0001; Sat, 29 Sep 2018 17:48:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B36D8E0002; Sat, 29 Sep 2018 17:48:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id A47E38E0001 for ; Sat, 29 Sep 2018 17:48:38 -0400 (EDT) Received: by mail-wm1-f70.google.com with SMTP id t79-v6so6762605wmt.3 for ; Sat, 29 Sep 2018 14:48:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state :content-disposition:content-transfer-encoding:mime-version:from:to :cc:date:message-id:subject:in-reply-to; bh=4HMBOvlcUWyk7xZEaIBRe1B4pGi9QoZx+IzRtSEXq+Y=; b=Ay0xjNVMWx5pm3ULGJMvIXYV6k3Bvw1qYhivfLX66+mVvVOXtfOvBPmPTijWiEvmGk 6AqKJrzsRxh+yvJG16aotkcbllk3blRWBSeZ/FBQCPadjpix/b9Dfine6CCwzRTVpO/0 1hKraXzftRH9SmTS3g+4Fyk1ZsY/XZprO6E63B/OiykfkV7qyN3+tIMyXW2gjx6ChGB/ 7+CdoFlo9ZNp5g5WqhnyxdEzCHSCfq1hihYGTvjzKfZ23HqM+gdlexrs+ExfQWrrQlA6 p0AHZGlws+QYct16ljLUlVmZsrBobSQfXH6sg8NqiQtxt3ocTxkl3tfXeklQOGTRZ7RQ zJVg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ben@decadent.org.uk designates 88.96.1.126 as permitted sender) smtp.mailfrom=ben@decadent.org.uk X-Gm-Message-State: ABuFfohTq96TI4DHozdydtUNzqf6yjW4kOMA2myEBU/JFthpHpo54kJj nfJnyRIcR0mWsHz7v5cOEubkabi2I9dGN20RTJWLt+zGqdV3X314fm58AJ4tyFhzPc2dYZ5zxFL pyzW4fs1aqnVua7qGPamQqHn6PvFK3lN85WmzdlJbSmkUVznh8xAY36FSNLtDnrgCng== X-Received: by 2002:a1c:9141:: with SMTP id t62-v6mr4948368wmd.69.1538257718158; Sat, 29 Sep 2018 14:48:38 -0700 (PDT) X-Google-Smtp-Source: ACcGV617yNPMs4FYbJS3cMj3YRT/Hp1dRh3O5U9yCTky1YiyTXLDA5Uv+tR8YfFZ5oqWIdWCa1Ed X-Received: by 2002:a1c:9141:: with SMTP id t62-v6mr4948344wmd.69.1538257717040; Sat, 29 Sep 2018 14:48:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538257717; cv=none; d=google.com; s=arc-20160816; b=nlcaglqLVLOn1IoH6Bg39UoC1EYF/E3CyhDQXsHsC2YDfuP58UdNqF+FXrtIzZwE/K R9VYiy54hzxGGTB4bTe2DFDmN2QbNuJ3OtSEIikaMR8+Yhi50PzDuiACz5PwXb0yFa95 UWy7IlYaA50Ly6E0MF0VlGZDbJe472z6+vnBEDLABmsrZ/G8pXhBIOEhk2XHZQ9GujT5 IFdUYTdCtMPlt+OlsZwharcaPRbrI3iQojNbWs39cLS4yUgb/eu71q/SWMOO1qsQ4B3J 3fLzL1bBJD8OQHt4bbwBgVIEgufKQMwO4TmT+9CL2C5occ7B9eTemb5wkh02JoIMOPYD 1z6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:subject:message-id:date:cc:to:from:mime-version :content-transfer-encoding:content-disposition; bh=4HMBOvlcUWyk7xZEaIBRe1B4pGi9QoZx+IzRtSEXq+Y=; b=0VpLAEdZCi5iKN2QfXkdPjqUJFOM7B7hS+BqhPbFvBoNLGV9vRQ5ZN0zF985+1TMVT 5qjYA8aizfla1CFiBi8ZovNAcut+mxfmWsKVUPW3nqaWQRM3B/cAqfHqA7m6J/fYYnKi fjGpZJukt8GLJv8SFB1B1MG9CTtMgzJXifFc6fQOQGvyc9x8ddyiUd2jloNsUbpYeXlu XLZdzG4L8U+oC5h+nj4w8co1f34j50GTCZxtV0f11yWw9wzkJV1GHyIe+S6q+E6zgUnY KFkHqET3jkQNVnmfEG+j7ptvqJvL2Vg3+QrPUXVFNDQ90crDRWc2HuG1BQ30vuourwvS SAYQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ben@decadent.org.uk designates 88.96.1.126 as permitted sender) smtp.mailfrom=ben@decadent.org.uk Received: from shadbolt.e.decadent.org.uk (shadbolt.e.decadent.org.uk. [88.96.1.126]) by mx.google.com with ESMTPS id x11-v6si7315680wrv.22.2018.09.29.14.48.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Sep 2018 14:48:36 -0700 (PDT) Received-SPF: pass (google.com: domain of ben@decadent.org.uk designates 88.96.1.126 as permitted sender) client-ip=88.96.1.126; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ben@decadent.org.uk designates 88.96.1.126 as permitted sender) smtp.mailfrom=ben@decadent.org.uk Received: from [2a02:8011:400e:2:cbab:f00:c93f:614] (helo=deadeye) by shadbolt.decadent.org.uk with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1g6N66-0000H3-51; Sat, 29 Sep 2018 22:48:22 +0100 Received: from ben by deadeye with local (Exim 4.91) (envelope-from ) id 1g6N5y-0006wG-Fg; Sat, 29 Sep 2018 22:48:14 +0100 Content-Disposition: inline MIME-Version: 1.0 From: Ben Hutchings To: linux-kernel@vger.kernel.org, stable@vger.kernel.org CC: akpm@linux-foundation.org, "H. Peter Anvin" , "Andy Lutomirski" , "Denys Vlasenko" , "Peter Zijlstra" , linux-mm@kvack.org, mhocko@suse.com, "Brian Gerst" , "Toshi Kani" , "Luis R. Rodriguez" , "Ingo Molnar" , "Dave Hansen" , dave.hansen@intel.com, "Linus Torvalds" , "Josh Poimboeuf" , "Thomas Gleixner" , "Borislav Petkov" , "Dave Hansen" Date: Sat, 29 Sep 2018 22:43:07 +0100 Message-ID: X-Mailer: LinuxStableQueue (scripts by bwh) Subject: [PATCH 3.16 092/131] x86/mm: Move swap offset/type up in PTE to work around erratum In-Reply-To: X-SA-Exim-Connect-IP: 2a02:8011:400e:2:cbab:f00:c93f:614 X-SA-Exim-Mail-From: ben@decadent.org.uk X-SA-Exim-Scanned: No (on shadbolt.decadent.org.uk); SAEximRunCond expanded to false X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP 3.16.59-rc1 review patch. If anyone has any objections, please let me know. ------------------ From: Dave Hansen commit 00839ee3b299303c6a5e26a0a2485427a3afcbbf upstream. This erratum can result in Accessed/Dirty getting set by the hardware when we do not expect them to be (on !Present PTEs). Instead of trying to fix them up after this happens, we just allow the bits to get set and try to ignore them. We do this by shifting the layout of the bits we use for swap offset/type in our 64-bit PTEs. It looks like this: bitnrs: | ... | 11| 10| 9|8|7|6|5| 4| 3|2|1|0| names: | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U|W|P| before: | OFFSET (9-63) |0|X|X| TYPE(1-5) |0| after: | OFFSET (14-63) | TYPE (9-13) |0|X|X|X| X| X|X|X|0| Note that D was already a don't care (X) even before. We just move TYPE up and turn its old spot (which could be hit by the A bit) into all don't cares. We take 5 bits away from the offset, but that still leaves us with 50 bits which lets us index into a 62-bit swapfile (4 EiB). I think that's probably fine for the moment. We could theoretically reclaim 5 of the bits (1, 2, 3, 4, 7) but it doesn't gain us anything. Signed-off-by: Dave Hansen Cc: Andrew Morton Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: Luis R. Rodriguez Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Toshi Kani Cc: dave.hansen@intel.com Cc: linux-mm@kvack.org Cc: mhocko@suse.com Link: http://lkml.kernel.org/r/20160708001911.9A3FD2B6@viggo.jf.intel.com Signed-off-by: Ingo Molnar [bwh: Backported to 3.16: Bit 9 may be reserved for PAGE_BIT_NUMA, which no longer exists upstream. Adjust the bit numbers accordingly, incorporating commit ace7fab7a6cd "x86/mm: Fix swap entry comment and macro".] Signed-off-by: Ben Hutchings --- a/arch/x86/include/asm/pgtable_64.h +++ b/arch/x86/include/asm/pgtable_64.h @@ -162,23 +162,37 @@ static inline int pgd_large(pgd_t pgd) { #define pte_offset_map(dir, address) pte_offset_kernel((dir), (address)) #define pte_unmap(pte) ((void)(pte))/* NOP */ -/* Encode and de-code a swap entry */ -#define SWP_TYPE_BITS 5 +/* + * Encode and de-code a swap entry + * + * | ... | 11| 10| 9|8|7|6|5| 4| 3|2|1|0| <- bit number + * | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U|W|P| <- bit names + * | OFFSET (15->63) | TYPE (10-14) | 0 |0|X|X|X| X| X|X|X|0| <- swp entry + * + * G (8) is aliased and used as a PROT_NONE indicator for + * !present ptes. We need to start storing swap entries above + * there. We also need to avoid using A and D because of an + * erratum where they can be incorrectly set by hardware on + * non-present PTEs. + */ #ifdef CONFIG_NUMA_BALANCING /* Automatic NUMA balancing needs to be distinguishable from swap entries */ -#define SWP_OFFSET_SHIFT (_PAGE_BIT_PROTNONE + 2) +#define SWP_TYPE_FIRST_SHIFT (_PAGE_BIT_PROTNONE + 2) #else -#define SWP_OFFSET_SHIFT (_PAGE_BIT_PROTNONE + 1) +#define SWP_TYPE_FIRST_SHIFT (_PAGE_BIT_PROTNONE + 1) #endif +#define SWP_TYPE_BITS 5 +/* Place the offset above the type: */ +#define SWP_OFFSET_FIRST_BIT (SWP_TYPE_FIRST_BIT + SWP_TYPE_BITS) #define MAX_SWAPFILES_CHECK() BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS) -#define __swp_type(x) (((x).val >> (_PAGE_BIT_PRESENT + 1)) \ +#define __swp_type(x) (((x).val >> (SWP_TYPE_FIRST_BIT)) \ & ((1U << SWP_TYPE_BITS) - 1)) -#define __swp_offset(x) ((x).val >> SWP_OFFSET_SHIFT) +#define __swp_offset(x) ((x).val >> SWP_OFFSET_FIRST_BIT) #define __swp_entry(type, offset) ((swp_entry_t) { \ - ((type) << (_PAGE_BIT_PRESENT + 1)) \ - | ((offset) << SWP_OFFSET_SHIFT) }) + ((type) << (SWP_TYPE_FIRST_BIT)) \ + | ((offset) << SWP_OFFSET_FIRST_BIT) }) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val((pte)) }) #define __swp_entry_to_pte(x) ((pte_t) { .pte = (x).val })