From patchwork Thu May 17 11:06:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Dufour X-Patchwork-Id: 10406379 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E0FCA60247 for ; Thu, 17 May 2018 11:07:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEEA428A07 for ; Thu, 17 May 2018 11:07:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C30D428A69; Thu, 17 May 2018 11:07:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7A1628A5B for ; Thu, 17 May 2018 11:07:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE5886B04AA; Thu, 17 May 2018 07:07:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ABC796B04AB; Thu, 17 May 2018 07:07:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 985406B04AC; Thu, 17 May 2018 07:07:04 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70]) by kanga.kvack.org (Postfix) with ESMTP id 4089F6B04AA for ; Thu, 17 May 2018 07:07:04 -0400 (EDT) Received: by mail-wm0-f70.google.com with SMTP id d82-v6so1727536wmd.4 for ; Thu, 17 May 2018 04:07:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:in-reply-to:references:message-id; bh=kcnMFDqd1q4hV8W97BnhEMBLZ9nGtMCT1ol5AnvaFoA=; b=ebwZRsdbn3VerSNuS2hy7IedWTiD5tcdSe7neLOAjbujFmc7rar+gtERLogfNKK9kS sr2x7wJk8uWFsuCKbWG36wwfefAVRSeKZqJehIcbxMR/dJEjMLwdfSBtgxCKdPpfxUaQ zOnIjs5Z+LmNKepcXnb9co9ZpUpqhQWLRhx1WKp1384QoHGeFmrZuQVvyvRRp1QKhVAa c7zXiaseQyhSX8SnO3aYUkyeCHshUjqJrTM9kJkbkZRllFjU02D+yGsUMJoVHxKe297x o8ai55g3Oju+T4tIuFInj7qt+Pgteg53WOWOFZJdAd7xHZ4t1YjCVyeMPvZuMgBZIHls 4U1A== X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of ldufour@linux.vnet.ibm.com) smtp.mailfrom=ldufour@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com X-Gm-Message-State: ALKqPwf7TTuhmGhEuSYthX/D5XUSlEZp7PphW3e7Yt11bEZ1+J+WxhZs rrgxgIiZrh2Sh5vvigpL/RqmabU+3+fQcWRji9vY8EP0dRPzatC4TjE4PixczWtZk/RA0fclpyY iJwddjVbn5d2yMHKO1SsFQNOM8H1YKHDBXhzR4Jut0n1aojfH/6VY62QJ+yE/F1A= X-Received: by 2002:a50:8f65:: with SMTP id 92-v6mr6366501edy.281.1526555223839; Thu, 17 May 2018 04:07:03 -0700 (PDT) X-Google-Smtp-Source: AB8JxZraulC9LxsxiPprfwo6+RPNP4jde4+UUXSIHlGyXJ0t1yEVxhV1qV3WfDrHitxeR3MtaXd8 X-Received: by 2002:a50:8f65:: with SMTP id 92-v6mr6366449edy.281.1526555223126; Thu, 17 May 2018 04:07:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526555223; cv=none; d=google.com; s=arc-20160816; b=shJry0HQGYPLXNPjcLGsY8OhUSXLjyHhUmcGrn8UTQYI8VKX7Pnsz+yp2MofwsnHA4 b3mRF07eojuh06N/EtdvWCcl1oWaSIxnVgbwpvi0+QXTOUYhz/+mN6o9UTIpBs3N9ubp p7+iulYZn4OAg7EhwKgFl1blpEeyZ2dvP+IdKVc1qZfVckBCHkGXHHdVOtRzN25belOV brDPNXy6IEscc6292s/lGzPVB6sJYxHmzI/zXmCCWCKQALt5O1tUlJ0biRLI/Oe1Ik6w jjoaAMRc9j9jYZcT9pAXnocC62hNlaucjZywdFtI3sZqAgcvQ5rBl2+IKAne13y7mNf7 A4bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:references:in-reply-to:date:subject:cc:to:from :arc-authentication-results; bh=kcnMFDqd1q4hV8W97BnhEMBLZ9nGtMCT1ol5AnvaFoA=; b=tQXofOF7ZIQ9ONg4+F6L40UDDCUS7nLgYjLwYK3Zq/Ck0SOEABLZPq2rQOjBE1a2Ax beaqp1KXKy6YjqJ110mBuq9YjLZgaeCmGe48YLBglUIE7VjNFbbcIgnafjgxFUdxCLTi iXbYNAHGiKkq98ueZ3j/IKt6DVnwRPQ/1tnTI30dYF2s+0xRIEXNEPVcNvuxE8fA9MWa Ygmve8frW6k4t9MInYHGctj8XyrEMe9G3pV3oe9IJ5TGq7dV6ClyDa9/vLu2y8yrzkDw M9puhfs0XEtXfGRl377KQ4qWkTdXB/f/TJ8/RToNlEdAQ1mSxxkfJDNW9a+F+1mfdJ+A wa+g== ARC-Authentication-Results: i=1; mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of ldufour@linux.vnet.ibm.com) smtp.mailfrom=ldufour@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id j19-v6si2900982edh.234.2018.05.17.04.07.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 17 May 2018 04:07:03 -0700 (PDT) Received-SPF: neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of ldufour@linux.vnet.ibm.com) client-ip=148.163.158.5; Authentication-Results: mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of ldufour@linux.vnet.ibm.com) smtp.mailfrom=ldufour@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4HB4CPe054174 for ; Thu, 17 May 2018 07:07:01 -0400 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 2j17kxaw60-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 17 May 2018 07:07:01 -0400 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 17 May 2018 12:06:59 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 17 May 2018 12:06:49 +0100 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4HB6mM48454404; Thu, 17 May 2018 11:06:49 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BDE174C052; Thu, 17 May 2018 11:58:37 +0100 (BST) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7622D4C044; Thu, 17 May 2018 11:58:36 +0100 (BST) Received: from nimbus.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.33]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 17 May 2018 11:58:36 +0100 (BST) From: Laurent Dufour To: akpm@linux-foundation.org, mhocko@kernel.org, peterz@infradead.org, kirill@shutemov.name, ak@linux.intel.com, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox , khandual@linux.vnet.ibm.com, aneesh.kumar@linux.vnet.ibm.com, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner , Ingo Molnar , hpa@zytor.com, Will Deacon , Sergey Senozhatsky , sergey.senozhatsky.work@gmail.com, Andrea Arcangeli , Alexei Starovoitov , kemi.wang@intel.com, Daniel Jordan , David Rientjes , Jerome Glisse , Ganesh Mahendran , Minchan Kim , Punit Agrawal , vinayak menon , Yang Shi Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, paulmck@linux.vnet.ibm.com, Tim Chen , linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v11 06/26] mm: introduce pte_spinlock for FAULT_FLAG_SPECULATIVE Date: Thu, 17 May 2018 13:06:13 +0200 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1526555193-7242-1-git-send-email-ldufour@linux.vnet.ibm.com> References: <1526555193-7242-1-git-send-email-ldufour@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18051711-0044-0000-0000-00000553750C X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18051711-0045-0000-0000-00002894E8EA Message-Id: <1526555193-7242-7-git-send-email-ldufour@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-17_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805170100 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When handling page fault without holding the mmap_sem the fetch of the pte lock pointer and the locking will have to be done while ensuring that the VMA is not touched in our back. So move the fetch and locking operations in a dedicated function. Signed-off-by: Laurent Dufour --- mm/memory.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a55e72c8e469..fa0d9493acac 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2298,6 +2298,13 @@ int apply_to_page_range(struct mm_struct *mm, unsigned long addr, } EXPORT_SYMBOL_GPL(apply_to_page_range); +static inline bool pte_spinlock(struct vm_fault *vmf) +{ + vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); + spin_lock(vmf->ptl); + return true; +} + static inline bool pte_map_lock(struct vm_fault *vmf) { vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, @@ -3814,8 +3821,8 @@ static int do_numa_page(struct vm_fault *vmf) * validation through pte_unmap_same(). It's of NUMA type but * the pfn may be screwed if the read is non atomic. */ - vmf->ptl = pte_lockptr(vma->vm_mm, vmf->pmd); - spin_lock(vmf->ptl); + if (!pte_spinlock(vmf)) + return VM_FAULT_RETRY; if (unlikely(!pte_same(*vmf->pte, vmf->orig_pte))) { pte_unmap_unlock(vmf->pte, vmf->ptl); goto out; @@ -4008,8 +4015,8 @@ static int handle_pte_fault(struct vm_fault *vmf) if (pte_protnone(vmf->orig_pte) && vma_is_accessible(vmf->vma)) return do_numa_page(vmf); - vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); - spin_lock(vmf->ptl); + if (!pte_spinlock(vmf)) + return VM_FAULT_RETRY; entry = vmf->orig_pte; if (unlikely(!pte_same(*vmf->pte, entry))) goto unlock;