From patchwork Thu Jun 6 18:44:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980271 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E34343E8C for ; Thu, 6 Jun 2019 18:44:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D544428A5E for ; Thu, 6 Jun 2019 18:44:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BFFDE28A7C; Thu, 6 Jun 2019 18:44:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AB60128A7E for ; Thu, 6 Jun 2019 18:44:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B13E6B0279; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 35C206B027D; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 274686B027E; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id E11C26B0279 for ; Thu, 6 Jun 2019 14:44:47 -0400 (EDT) Received: by mail-qt1-f199.google.com with SMTP id o16so2886232qtj.6 for ; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lL1gh8nawE6RFwsPdkPriGSJ+VnlR7Td1463Tqz8utE=; b=YEpz0rj4Ok/HROHP2BgxlpInnpQDk63LZ/lEbE3kcj7J6u/9bgCC52hTYkSvM7Y5mm Im0bgrRyhUVbRcnTBp4yWXhcGinNO65w4A29ATxU/MbNkJJK+M18BevhgeSTUaPdMeUu R+JmuXGSLgCTBZj/okuzqPKllUswhoS1qx2YgCw70UTSa0BGixvaPqm1kzy+TrX7tpTd zpv8S125uo3X4Jr7pcA5PRQaWj20zxZ2McCt9QEw7CqLu3xHLfgUv0cEwpCY9EihRIuB 1+927sBp15d2jpJEgHfkEayazBUAQgA+j3v/roLUARjhLYInT3s7SdnhGyoZ5a5ATjji vGow== X-Gm-Message-State: APjAAAWlAMzpUt28nixmvr88BoMBFQcB1hRBcvVNbsR+f1Q/HtVc5eT+ 4d0RFb6cGZBq/FxMZ56u+FUGMKYPp1OYfFFy9Ki1ko2ZBmthq8hGeDpkQwSHfeAoXjEGIvTIJ95 A8bteM/w8I9G/igloT16+F7IemWsZ6So+9eg5X4qUdtw8+hVqzvateeuOYlU73Uhspw== X-Received: by 2002:a0c:81e7:: with SMTP id 36mr7255112qve.5.1559846687661; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) X-Received: by 2002:a0c:81e7:: with SMTP id 36mr7255081qve.5.1559846687055; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846687; cv=none; d=google.com; s=arc-20160816; b=ZRTlH+59H/h4lfYD3ad3wZFxSFdHy3xy+4jQQWzw0Kp0bA5MxC4CKqeM9BP0Mx0svK iRWYvvCNeMGMf9fo/hb6EJN5EBz4hPlcL9n5fREYd2zqpxgwwRxQYIZSI1/g4ry0E+AQ mO4fMJqqiPYVBp7Kn/G8eVlqVnBkTgUiV94rZ12jEWHV4y91x60eVHN/zlMIvl019+kq KmDBYyLPL1sIpZbwHZ+FS336koneIymPk+vcXNlmYWlO4aDprboBvV+vMPCezkmrs/KO jKa+cpJw8q2XE4RsyBYoLS+dMzfLq8RKZxV93bW4L3b25VboAKIZS3E6HNidWgWgVlEP Ni+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=lL1gh8nawE6RFwsPdkPriGSJ+VnlR7Td1463Tqz8utE=; b=rTN/+/1Uz0KcPCmK26famY/T2AihnkEQ6y2DlcoQrz8ScNKQSYSUaDqn0+EXScCTMQ HERAz2fUAJ+cfQl2J0nk1XgNAjEUWemd7lb9M68JF9v6+qI3tdnXx8DMmczQvnXusKlI HGYmFbI99lvRgZ2gR+bZ5YgYuugMcYiqARWM7Z2B7E7DyzILfRC+BI/cMTF+mTGPaNqW mFHJKSwBysWdhrGyqPH8Ui/IuqFmGVoZn6WLlIIWX0XchUQqhw0zEryaUqojm+1QqazX 92hoOLHS/lzYdmJFxnQCkZguY9fELg9ZrdU0TyfNmXUo+PuGylFccQg6ridiW1zaBeMZ xX9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=i1eoUPW+; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l12sor1433048qkg.113.2019.06.06.11.44.46 for (Google Transport Security); Thu, 06 Jun 2019 11:44:47 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=i1eoUPW+; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lL1gh8nawE6RFwsPdkPriGSJ+VnlR7Td1463Tqz8utE=; b=i1eoUPW+Np1ntz3bbKM3DpfMNN3rtSLjFDcmDB0vnq8UBH26kk63yNvtdKWmYVv2TV Pi+BVA4ti/NZ/Kxq6fbwoxDlR7kj35FAE/DOaimNUrZ5r1sTA5O/bAcr13jRwmRbnpxo TDFODNWEBAT/btsMWcqDTmLq2sZ6BFWqMBX8F+5SG9cP5jttZex0XY8bZAT3yEvhQllA Ti/dQpvuTYxWENCc1huucv5i9yrAjKo9CBNJ/Lo8NX9PMfBSb4jhB683uChR8oVThjIV ISzCmuSiYW97tizaIYodsqJaggibXGQKh2/eAvL6GclckuApULlhfSiRO/Br537Ceq9G qQdQ== X-Google-Smtp-Source: APXvYqysUYNwLpFGpXB7aK6DBYGI6cTSoPBzuRqVbpub4N7BK9TEIAgc5KayvZk47O3Fq3EpoC7BwA== X-Received: by 2002:a37:6f81:: with SMTP id k123mr4055833qkc.321.1559846686738; Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id e128sm1194796qkf.90.2019.06.06.11.44.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008I5-Dx; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 01/11] mm/hmm: fix use after free with struct hmm in the mmu notifiers Date: Thu, 6 Jun 2019 15:44:28 -0300 Message-Id: <20190606184438.31646-2-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe mmu_notifier_unregister_no_release() is not a fence and the mmu_notifier system will continue to reference hmm->mn until the srcu grace period expires. Resulting in use after free races like this: CPU0 CPU1 __mmu_notifier_invalidate_range_start() srcu_read_lock hlist_for_each () // mn == hmm->mn hmm_mirror_unregister() hmm_put() hmm_free() mmu_notifier_unregister_no_release() hlist_del_init_rcu(hmm-mn->list) mn->ops->invalidate_range_start(mn, range); mm_get_hmm() mm->hmm = NULL; kfree(hmm) mutex_lock(&hmm->lock); Use SRCU to kfree the hmm memory so that the notifiers can rely on hmm existing. Get the now-safe hmm struct through container_of and directly check kref_get_unless_zero to lock it against free. Signed-off-by: Jason Gunthorpe Reviewed-by: Ralph Campbell --- v2: - Spell 'free' properly (Jerome/Ralph) --- include/linux/hmm.h | 1 + mm/hmm.c | 25 +++++++++++++++++++------ 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 092f0234bfe917..688c5ca7068795 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -102,6 +102,7 @@ struct hmm { struct mmu_notifier mmu_notifier; struct rw_semaphore mirrors_sem; wait_queue_head_t wq; + struct rcu_head rcu; long notifiers; bool dead; }; diff --git a/mm/hmm.c b/mm/hmm.c index 8e7403f081f44a..547002f56a163d 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -113,6 +113,11 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) return NULL; } +static void hmm_free_rcu(struct rcu_head *rcu) +{ + kfree(container_of(rcu, struct hmm, rcu)); +} + static void hmm_free(struct kref *kref) { struct hmm *hmm = container_of(kref, struct hmm, kref); @@ -125,7 +130,7 @@ static void hmm_free(struct kref *kref) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); - kfree(hmm); + mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu); } static inline void hmm_put(struct hmm *hmm) @@ -153,10 +158,14 @@ void hmm_mm_destroy(struct mm_struct *mm) static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) { - struct hmm *hmm = mm_get_hmm(mm); + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); struct hmm_mirror *mirror; struct hmm_range *range; + /* hmm is in progress to free */ + if (!kref_get_unless_zero(&hmm->kref)) + return; + /* Report this HMM as dying. */ hmm->dead = true; @@ -194,13 +203,15 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) static int hmm_invalidate_range_start(struct mmu_notifier *mn, const struct mmu_notifier_range *nrange) { - struct hmm *hmm = mm_get_hmm(nrange->mm); + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); struct hmm_mirror *mirror; struct hmm_update update; struct hmm_range *range; int ret = 0; - VM_BUG_ON(!hmm); + /* hmm is in progress to free */ + if (!kref_get_unless_zero(&hmm->kref)) + return 0; update.start = nrange->start; update.end = nrange->end; @@ -245,9 +256,11 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, static void hmm_invalidate_range_end(struct mmu_notifier *mn, const struct mmu_notifier_range *nrange) { - struct hmm *hmm = mm_get_hmm(nrange->mm); + struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); - VM_BUG_ON(!hmm); + /* hmm is in progress to free */ + if (!kref_get_unless_zero(&hmm->kref)) + return; mutex_lock(&hmm->lock); hmm->notifiers--; From patchwork Thu Jun 6 18:44:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980279 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A17A81398 for ; Thu, 6 Jun 2019 18:44:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9260428A5E for ; Thu, 6 Jun 2019 18:44:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86F1128A74; Thu, 6 Jun 2019 18:44:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D3D028A5E for ; Thu, 6 Jun 2019 18:44:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 050CD6B027F; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 001366B0281; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0C1D6B0280; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id B52D76B027D for ; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id l185so2763499qkd.14 for ; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cKkqHG6lA8ec9OmOBivsmV6gRcE/obxU2CCqIDGlTpM=; b=IAPT+lwiUnQlOTuQ/7r9lRnGaeJCWUAvL6QzfQJyCnVfJilHXBCd/872tNjgfqiOzq RRkcsMU6DfReo9WQEYqyc8ooamfDIEJqyhu9/KJhqS9QiWNVQDpVK3gLixThXAtKIZc/ VGTrA3uqDLapWahTR73wXuO5peVKbkkylQFoGFvSwN3VDModnlGoz875RxDrYaxXu0be Ulab+r7y4lwDmYAJofinNCTRWxw0QRzaHK/snGVN1fzxyK54ejIiQb1dkYzWj2qMdx7e 1iGJd/nQiQpwyumHkEwK7GytNnmb+mUP/O3P+oioC5zN8jV3zRpc6XFAsEazWgky1sdB Nx7A== X-Gm-Message-State: APjAAAXQT2uLd8denINGRCssENuy4FzNS9hilEVyFyxQZmNjm3mSA9vv NPzohmW91bcQcKQ7XhXz90EGsj9sK6W/JXhY/Kra8zhDAJ6PKpVUYicspRE/kZjIp+1cR8UnOXF NPS6ZVaVPdkjpQuUZ1sy5k9LhDcZVDTXPyZ+CmqrB4ze+Vd7sKf575SOlVoOvgvUaxA== X-Received: by 2002:a05:620a:14a8:: with SMTP id x8mr40354465qkj.35.1559846688484; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) X-Received: by 2002:a05:620a:14a8:: with SMTP id x8mr40354402qkj.35.1559846687367; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846687; cv=none; d=google.com; s=arc-20160816; b=CxcEe4b9voM7U5CR4HbsQeAFjPNb46NJ9qQSU4uGud4yrbPgsUYtJK/KN9PkL7Vucb XxUbsm3bWS7IdowEXN2UpKZQUAwVJOUQNyujKUHIrX7Zywi1oDAqjQHYvZ13d8zjqqch l1uWxGtUJN2eIrMsWewiQ167cMSH2Aa1fkvG/a/hJRsNlchjefYy2vqlvH2KAc7bUVv3 JzK2TTmP6ijjWc2zwcapJJ0u7QSgxk9aD7wW/RyzQssaw3JGowlYEsmCkLXq6SKztw48 Vq3XCVRLpDQPt+ZruwsLkHU0RVrEorXvC6/bEu5b7p0/KjEROaHLl75vPPrjM0J0Glfn dAEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=cKkqHG6lA8ec9OmOBivsmV6gRcE/obxU2CCqIDGlTpM=; b=TZPv+YQKghHzpB+s60ETG5eAmjcIWCdnrB6XBHTTMWSGqWm+xM5guJzmUACXE7BqD0 trcTCwi/KN91Gh3kfiFjQwmkYf4wXqin4F/21VRKHgd+mSmAbCS+ZXSsRJNXwC9vTDsI yhKRT6V7u/SYpHqT6BPRzRZKMvwRRWs1cxTuxHhdIF8YthxkykxjdkaVCUywRc6a+gKc kP0e8c6zr9VgigrlSwsK8WF9K5+Yh4Vy8jD0l1L5+w2j1igrcA9l4wwd7YQmllrAMFLl dJ5KFgecFYmR9H6l1ZEblTqjSixC/j2FQrCBddNZaw5gwZaEzqY71cYBYjUmT0ERGO8p PAzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=jwj7m66y; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l66sor1449557qkd.125.2019.06.06.11.44.47 for (Google Transport Security); Thu, 06 Jun 2019 11:44:47 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=jwj7m66y; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cKkqHG6lA8ec9OmOBivsmV6gRcE/obxU2CCqIDGlTpM=; b=jwj7m66yktUzAAVuBGGSCtyJS/egSnF1B/vxHFcHYcF2Uik/EqL4UNPU57byfvPnPV EUOhSE6Yc2ULsjPzk2Q0cPgGDWJT6t2SoK+wEhbAq1wIixyAii3w6eS/3F4aOcLSPwis wISfz8dx+YNMVHjshe7Qu1Sb1slKfIiBQMNHt3ytiNJzaiH2eoJOAMMz+9NSZ0N4cZIj IOJsM/44sVtvL3oMoPVTtng3fKxXbLORwQK9ZTj6pDXhMqEyH4epdWP8UfEJY1SfdN0a cUwOJj4KyRl8nUp5tdavSJ2m0nYwjfI9rz5uUmGGXTAadrKsObhys9ToUrWKVFbjE5E+ wb4w== X-Google-Smtp-Source: APXvYqwVkPweqvQyzdMm8hrdxDc8fcdFbLwJPBzLxuUMc6ON/X0Bc6zX6ARY8mPOX6I9iWyD4gOp2g== X-Received: by 2002:a37:a0e:: with SMTP id 14mr21589009qkk.203.1559846687100; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id e66sm1557234qtb.55.2019.06.06.11.44.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008IB-G5; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 02/11] mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_register Date: Thu, 6 Jun 2019 15:44:29 -0300 Message-Id: <20190606184438.31646-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Ralph observes that hmm_range_register() can only be called by a driver while a mirror is registered. Make this clear in the API by passing in the mirror structure as a parameter. This also simplifies understanding the lifetime model for struct hmm, as the hmm pointer must be valid as part of a registered mirror so all we need in hmm_register_range() is a simple kref_get. Suggested-by: Ralph Campbell Signed-off-by: Jason Gunthorpe Reviewed-by: Ralph Campbell Reviewed-by: Ira Weiny Signed-off-by: Jason Gunthorpe --- v2 - Include the oneline patch to nouveau_svm.c --- drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +- include/linux/hmm.h | 7 ++++--- mm/hmm.c | 15 ++++++--------- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouveau/nouveau_svm.c index 93ed43c413f0bb..8c92374afcf227 100644 --- a/drivers/gpu/drm/nouveau/nouveau_svm.c +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -649,7 +649,7 @@ nouveau_svm_fault(struct nvif_notify *notify) range.values = nouveau_svm_pfn_values; range.pfn_shift = NVIF_VMM_PFNMAP_V0_ADDR_SHIFT; again: - ret = hmm_vma_fault(&range, true); + ret = hmm_vma_fault(&svmm->mirror, &range, true); if (ret == 0) { mutex_lock(&svmm->mutex); if (!hmm_vma_range_done(&range)) { diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 688c5ca7068795..2d519797cb134a 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -505,7 +505,7 @@ static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror) * Please see Documentation/vm/hmm.rst for how to use the range API. */ int hmm_range_register(struct hmm_range *range, - struct mm_struct *mm, + struct hmm_mirror *mirror, unsigned long start, unsigned long end, unsigned page_shift); @@ -541,7 +541,8 @@ static inline bool hmm_vma_range_done(struct hmm_range *range) } /* This is a temporary helper to avoid merge conflict between trees. */ -static inline int hmm_vma_fault(struct hmm_range *range, bool block) +static inline int hmm_vma_fault(struct hmm_mirror *mirror, + struct hmm_range *range, bool block) { long ret; @@ -554,7 +555,7 @@ static inline int hmm_vma_fault(struct hmm_range *range, bool block) range->default_flags = 0; range->pfn_flags_mask = -1UL; - ret = hmm_range_register(range, range->vma->vm_mm, + ret = hmm_range_register(range, mirror, range->start, range->end, PAGE_SHIFT); if (ret) diff --git a/mm/hmm.c b/mm/hmm.c index 547002f56a163d..8796447299023c 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -925,13 +925,13 @@ static void hmm_pfns_clear(struct hmm_range *range, * Track updates to the CPU page table see include/linux/hmm.h */ int hmm_range_register(struct hmm_range *range, - struct mm_struct *mm, + struct hmm_mirror *mirror, unsigned long start, unsigned long end, unsigned page_shift) { unsigned long mask = ((1UL << page_shift) - 1UL); - struct hmm *hmm; + struct hmm *hmm = mirror->hmm; range->valid = false; range->hmm = NULL; @@ -945,15 +945,12 @@ int hmm_range_register(struct hmm_range *range, range->start = start; range->end = end; - hmm = hmm_get_or_create(mm); - if (!hmm) - return -EFAULT; - /* Check if hmm_mm_destroy() was call. */ - if (hmm->mm == NULL || hmm->dead) { - hmm_put(hmm); + if (hmm->mm == NULL || hmm->dead) return -EFAULT; - } + + range->hmm = hmm; + kref_get(&hmm->kref); /* Initialize range to track CPU page table updates. */ mutex_lock(&hmm->lock); From patchwork Thu Jun 6 18:44:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980281 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3E7F1398 for ; Thu, 6 Jun 2019 18:44:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D663728A5E for ; Thu, 6 Jun 2019 18:44:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9DBA28A74; Thu, 6 Jun 2019 18:44:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55CDE28A5E for ; Thu, 6 Jun 2019 18:44:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50C6C6B027E; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 496A56B0282; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04E036B027E; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by kanga.kvack.org (Postfix) with ESMTP id CAFCC6B027E for ; Thu, 6 Jun 2019 14:44:48 -0400 (EDT) Received: by mail-qt1-f198.google.com with SMTP id q13so2869984qtj.15 for ; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bW6W0dzfTSD+G2qWpxSPnT6FDkRgs6qYralNDwCWWpM=; b=T3bmauwurK43ci2hK7odx/4S4nG6/m0vC62BQe4K/94VfZ/TbgzmeYeqaunjsmjm8z PiDomoC0LyTLNjUZH/W7B22KmWjN5spalVbSrRIbmK0FZCMuH/WkDi3BZ6MnlKWZfKd8 UxV3RyDIo73krIr2BQ5+0PPjR6p0QGtzgfDFO3x3nAbBtl1wEgPe5qs6zb3JCkwhdR1n 9FR7AHX0IjKm0YJshgBev+vVzpF/tB9J8EF/NZ9XoFK5Qu3kSMJ1yJEhFLoR9CjbZFEd wgghTk3GB6SKCyrTtiJ/CUHx1c96wG5nY+kNwSiFGi26LED9aDys32R/ZkOjuusb4gpf BvxA== X-Gm-Message-State: APjAAAXyQuNRtGmYzlUB8U3T9b/OwACks6JDPO0JtyhSz61yMmEmsRDm pSEnEJUK6B4h7RyombPbw1CqlxaHfxfIte+DZBSPi+tyRZDr9E4+ePg4UkdYsbpUQNIIjm2r1jw hYWS7GFA4EZHIXLHZwkCcQqbffO4/RxFdkp8dnZBAaVB7Lks3mg2ShjTXMmCU51aLpA== X-Received: by 2002:ac8:2bd4:: with SMTP id n20mr32689931qtn.131.1559846688575; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) X-Received: by 2002:ac8:2bd4:: with SMTP id n20mr32689889qtn.131.1559846687783; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846687; cv=none; d=google.com; s=arc-20160816; b=n0XeLZJ83GEugy9vT1WT7tupNRcRAhUWdfWm+k/icZqJHo2MmDxXfkXHr4UP1F2vy3 HDuQCZqA08BhV3NecNJa26nKn1E1W0rU7fvGlZhdriSYNjCBjbGXmzUGFxcIukufVpFA eKOub/UXFOQv7fydkiHMzhZRNX0gQVuB4DNo7FmZdMR8uIWhOmhahBs0XQm7/pKM0vx/ BjzfEpGXPdzsY9JPjV/6MlOXSxzJNnqHzpuvDj2v6oG6dYTzXS5S8Q/XOKijqbOA55xY U0HcqB6mJPR/EAU5m6mIXFngCcK7RB2dcYwmMqC0y/JYJkby7nw1WcFEziGo1qh2qn5L 6Kiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=bW6W0dzfTSD+G2qWpxSPnT6FDkRgs6qYralNDwCWWpM=; b=GIl8tJYT6IxK6uplRXo0QZaWbScIGI9/vi+Px0JqBp0nva+0rgXzBNaQ93QgOQgiCC VULES9rSkqZAWOh+sVn4XH/dgPSuL0dVKLL/6cnDRtaee/ho2X2BHq9hf4/WF5B8VTEB kDDtLeWS08M18rtcu0sO/+qFvcrosiTIazedawACwexrQX3ZEx+FvCRp+NE9tULEv2fC p+FOCl5UKgRvUd0wai5B2Y9wuv4OYNKlMPTNiwsfe6Kv+cLjCnwTs9AGOa+H/qeqN7Va GhSA5hjDVi+/8FYjFyvEovk+UZMlWl4jn7v/7HifFetM29doCF9tKflKj9ga2HjFXFi0 ky6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=EJVyVO6o; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u39sor3150870qte.45.2019.06.06.11.44.47 for (Google Transport Security); Thu, 06 Jun 2019 11:44:47 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=EJVyVO6o; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bW6W0dzfTSD+G2qWpxSPnT6FDkRgs6qYralNDwCWWpM=; b=EJVyVO6oJlvHu7MqG8ir6ye0d2E52VzdqGkXefNtzJsbNH/e2vUDKVO4zKcSXAEFek RGTHSuR2YRZ1JntKN58PGjvSeSfQolEl6TZyDu5mOklbvcsTapHOZDax5t2QrdVtX4to S11U99ZURLmGhJncgXoMHt954BFGp+lQHi/nCQ/NGsZzpVCFFG4kyutLM4kf1TOb9nOO tio0c5Uz+BelXNUYQJIQjh7rB+6spPYFY8/2kbAPv75GMFedDdWbqiZJxGEeE5bGDsyy mjVC0gpMoewb2qe/MbOH7NBFg26yQBOwn3DW+fWWa28YBIKRIyCP/q4ODTS6P3lKmvV+ taYA== X-Google-Smtp-Source: APXvYqzZrmfOR3muOAR1prnBDkAE4ixgQwkLWPMt0LNdVl4lLNKXUZcw+vPND9ViR47fQ9OlYet1CA== X-Received: by 2002:aed:2a43:: with SMTP id k3mr42504564qtf.301.1559846687527; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id c5sm1192064qtj.27.2019.06.06.11.44.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008IH-HE; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 03/11] mm/hmm: Hold a mmgrab from hmm to mm Date: Thu, 6 Jun 2019 15:44:30 -0300 Message-Id: <20190606184438.31646-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe So long a a struct hmm pointer exists, so should the struct mm it is linked too. Hold the mmgrab() as soon as a hmm is created, and mmdrop() it once the hmm refcount goes to zero. Since mmdrop() (ie a 0 kref on struct mm) is now impossible with a !NULL mm->hmm delete the hmm_hmm_destroy(). Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: Ralph Campbell Reviewed-by: Ira Weiny --- v2: - Fix error unwind paths in hmm_get_or_create (Jerome/Jason) --- include/linux/hmm.h | 3 --- kernel/fork.c | 1 - mm/hmm.c | 22 ++++------------------ 3 files changed, 4 insertions(+), 22 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 2d519797cb134a..4ee3acabe5ed22 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -586,14 +586,11 @@ static inline int hmm_vma_fault(struct hmm_mirror *mirror, } /* Below are for HMM internal use only! Not to be used by device driver! */ -void hmm_mm_destroy(struct mm_struct *mm); - static inline void hmm_mm_init(struct mm_struct *mm) { mm->hmm = NULL; } #else /* IS_ENABLED(CONFIG_HMM_MIRROR) */ -static inline void hmm_mm_destroy(struct mm_struct *mm) {} static inline void hmm_mm_init(struct mm_struct *mm) {} #endif /* IS_ENABLED(CONFIG_HMM_MIRROR) */ diff --git a/kernel/fork.c b/kernel/fork.c index b2b87d450b80b5..588c768ae72451 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -673,7 +673,6 @@ void __mmdrop(struct mm_struct *mm) WARN_ON_ONCE(mm == current->active_mm); mm_free_pgd(mm); destroy_context(mm); - hmm_mm_destroy(mm); mmu_notifier_mm_destroy(mm); check_mm(mm); put_user_ns(mm->user_ns); diff --git a/mm/hmm.c b/mm/hmm.c index 8796447299023c..cc7c26fda3300e 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -82,6 +83,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) hmm->notifiers = 0; hmm->dead = false; hmm->mm = mm; + mmgrab(hmm->mm); spin_lock(&mm->page_table_lock); if (!mm->hmm) @@ -109,6 +111,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); error: + mmdrop(hmm->mm); kfree(hmm); return NULL; } @@ -130,6 +133,7 @@ static void hmm_free(struct kref *kref) mm->hmm = NULL; spin_unlock(&mm->page_table_lock); + mmdrop(hmm->mm); mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu); } @@ -138,24 +142,6 @@ static inline void hmm_put(struct hmm *hmm) kref_put(&hmm->kref, hmm_free); } -void hmm_mm_destroy(struct mm_struct *mm) -{ - struct hmm *hmm; - - spin_lock(&mm->page_table_lock); - hmm = mm_get_hmm(mm); - mm->hmm = NULL; - if (hmm) { - hmm->mm = NULL; - hmm->dead = true; - spin_unlock(&mm->page_table_lock); - hmm_put(hmm); - return; - } - - spin_unlock(&mm->page_table_lock); -} - static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); From patchwork Thu Jun 6 18:44:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980285 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CDBCF1398 for ; Thu, 6 Jun 2019 18:45:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC71A28A5E for ; Thu, 6 Jun 2019 18:45:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AD9EB28A74; Thu, 6 Jun 2019 18:45:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C62428A5E for ; Thu, 6 Jun 2019 18:45:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFB776B027D; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD7B06B0280; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 903076B0281; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 6E34F6B0280 for ; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id i196so2754754qke.20 for ; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cde22D2D/SQ7DjRMRDYakpeqSceiEmLYAy9R4sDRFZ4=; b=aNq3y32ocGy7+kOvmds1SNkj5iVCSMhoV2bbIbpSHKZPKtiIO6voTfI+hzB/UfweEa 4gkHmABbDioVbYnnjlmI8nm+eNjLTBuowjYESQrcWgZTGVItG6dHl3bDi8WQLGW8RbMW EFfHsaxXIJfOHSjFiDPfdU71BhNVdFPzYQB/uymweA2hXegIlF4vxblD2Pm/0Tcw5eDq H/qPZCZjr0fNZ1iHP8fd3idLPeBHliqMdNZv0TiNeGXGpPO8l4hL3jqIJbCymu/nnObW 6dGT5bjpTtdv/wzejm6hF8bH95gJgvYQQKrXKM2aWNL/lkw8nYP9mdygWh9/AF2bKFWc +Ciw== X-Gm-Message-State: APjAAAWND16cmlxwn8LIrju/XpwXrPs3OVHU9a8N7TZJjFt+XSeKHPzJ FqORTLEXdNSqcKUS9dS+EyQ/7kkKOIdnyoBpnanpCCHYMbF0z/SRPjmMyq4Ee0RyLCVmYYhIICS 6LXklcOCv+ihHjfYBJPZdndxldOgVwqXd/u7nCCKjbI5klRBPGeGBfS6RIohIDjlj/g== X-Received: by 2002:ae9:c21a:: with SMTP id j26mr20775598qkg.310.1559846689197; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) X-Received: by 2002:ae9:c21a:: with SMTP id j26mr20775557qkg.310.1559846688458; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846688; cv=none; d=google.com; s=arc-20160816; b=u9aWdgIHZoDqL5T9OgGMqobef88P7aqEpbIPI1co03wzmiNruW0Wa6nJU4gau7rtsL Dg26n2+RNNz/b7t1zw8PMBpieLJ35swnt798j0CE7UKH+oDPTlEBeVrC14gATgFqeESt dCassprbhcdtvcyCPNZ3k6PW8D7tUuFxwpB/yswezg9mXYd+fDJgADd3XmPeC5npL7rw OjV70G0l7eZYqefWXadFYPH4i0KYUez432YiftX+BTtojNH9aeNVaOuMB1tQhTWlbOF2 pqq2rLASowtkgHpIDf77ElbHV8VoHCU4FXfsSCoIUWLxxV8z/I93rk7wxJnZsaBlTgZB mz+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=cde22D2D/SQ7DjRMRDYakpeqSceiEmLYAy9R4sDRFZ4=; b=hV+Ug9nhQFQKcrHIL5kHYEgMA1FC64z6LCzJFv+DT9xRDBcd5Yf41AcNhzrS6opHEE 8eHoXdpUVf6rlqz7Mn45gAHn17hYo0+XN8sV9y5C/baghPzt9w6/Fyu3bRElIG84hofa Qac3wHkSyXpPwwsOsDB3MITXFeHg6Nv/hvy6nqlbg9lSNjo3Ll5a9Kdc63pw+PYjph8k iHZysYoMSPme60G3/0MQD6jJqmGvxvZrd/HnEnGrXJZ/io5whG6/NRw+qR12j9TwIhlI gDzzEth3x5Dd0Dp1/oxdf7ccbQrWtOF4EP1nBBR60tF4XOkCSe8xJCJFyX6Nqn6NhkGF CroQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=lSL5uWTC; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y2sor907928qkl.107.2019.06.06.11.44.48 for (Google Transport Security); Thu, 06 Jun 2019 11:44:48 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=lSL5uWTC; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cde22D2D/SQ7DjRMRDYakpeqSceiEmLYAy9R4sDRFZ4=; b=lSL5uWTCwCDgnbM/9/jrfQbxzPhWl8p/lDEgf4FT0uXH6Y+QewAJIhobM0Y5pnTV9Q mHJJmWUwI+A1PUB/WHWHue/Bz0y86gLvYZNesbGU1EQLEO4zSV4kHiD4h1a3k/MFDCdt WER1Mq2EaZX18rAxQeUToKW/nL69Y1jxs8Z6SnX3i9uxlnWxabF1Y6RuKi6OeGO9ieDQ MKnU5o9rvxa7nvv/y/bNKbuZLHNH9r7rBleushaJ37/rgvNJIPoVwIFEEjE9KU47+m6B Zk1tFH65d0SU9drWp7Mv6gBGKQ18TW6fwuehe84ewVEWflPwK/zZvOf9mFM3yphUj1/7 R/wA== X-Google-Smtp-Source: APXvYqxTSq7xbg9zxLiy0kEDCOrZTnZLTLbqUahJa+pbRkNf/2oRcK31kQh3sfGGk+OlBqVQE/baDQ== X-Received: by 2002:a37:4fca:: with SMTP id d193mr40455150qkb.298.1559846688185; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id o38sm1731656qto.96.2019.06.06.11.44.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008IN-IO; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 04/11] mm/hmm: Simplify hmm_get_or_create and make it reliable Date: Thu, 6 Jun 2019 15:44:31 -0300 Message-Id: <20190606184438.31646-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe As coded this function can false-fail in various racy situations. Make it reliable by running only under the write side of the mmap_sem and avoiding the false-failing compare/exchange pattern. Also make the locking very easy to understand by only ever reading or writing mm->hmm while holding the write side of the mmap_sem. Signed-off-by: Jason Gunthorpe Reviewed-by: Ralph Campbell Reviewed-by: Ira Weiny --- v2: - Fix error unwind of mmgrab (Jerome) - Use hmm local instead of 2nd container_of (Jerome) --- mm/hmm.c | 80 ++++++++++++++++++++------------------------------------ 1 file changed, 29 insertions(+), 51 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index cc7c26fda3300e..dc30edad9a8a02 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -40,16 +40,6 @@ #if IS_ENABLED(CONFIG_HMM_MIRROR) static const struct mmu_notifier_ops hmm_mmu_notifier_ops; -static inline struct hmm *mm_get_hmm(struct mm_struct *mm) -{ - struct hmm *hmm = READ_ONCE(mm->hmm); - - if (hmm && kref_get_unless_zero(&hmm->kref)) - return hmm; - - return NULL; -} - /** * hmm_get_or_create - register HMM against an mm (HMM internal) * @@ -64,11 +54,20 @@ static inline struct hmm *mm_get_hmm(struct mm_struct *mm) */ static struct hmm *hmm_get_or_create(struct mm_struct *mm) { - struct hmm *hmm = mm_get_hmm(mm); - bool cleanup = false; + struct hmm *hmm; - if (hmm) - return hmm; + lockdep_assert_held_exclusive(&mm->mmap_sem); + + if (mm->hmm) { + if (kref_get_unless_zero(&mm->hmm->kref)) + return mm->hmm; + /* + * The hmm is being freed by some other CPU and is pending a + * RCU grace period, but this CPU can NULL now it since we + * have the mmap_sem. + */ + mm->hmm = NULL; + } hmm = kmalloc(sizeof(*hmm), GFP_KERNEL); if (!hmm) @@ -83,57 +82,36 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) hmm->notifiers = 0; hmm->dead = false; hmm->mm = mm; - mmgrab(hmm->mm); - - spin_lock(&mm->page_table_lock); - if (!mm->hmm) - mm->hmm = hmm; - else - cleanup = true; - spin_unlock(&mm->page_table_lock); - if (cleanup) - goto error; - - /* - * We should only get here if hold the mmap_sem in write mode ie on - * registration of first mirror through hmm_mirror_register() - */ hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; - if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) - goto error_mm; + if (__mmu_notifier_register(&hmm->mmu_notifier, mm)) { + kfree(hmm); + return NULL; + } + mmgrab(hmm->mm); + mm->hmm = hmm; return hmm; - -error_mm: - spin_lock(&mm->page_table_lock); - if (mm->hmm == hmm) - mm->hmm = NULL; - spin_unlock(&mm->page_table_lock); -error: - mmdrop(hmm->mm); - kfree(hmm); - return NULL; } static void hmm_free_rcu(struct rcu_head *rcu) { - kfree(container_of(rcu, struct hmm, rcu)); + struct hmm *hmm = container_of(rcu, struct hmm, rcu); + + down_write(&hmm->mm->mmap_sem); + if (hmm->mm->hmm == hmm) + hmm->mm->hmm = NULL; + up_write(&hmm->mm->mmap_sem); + mmdrop(hmm->mm); + + kfree(hmm); } static void hmm_free(struct kref *kref) { struct hmm *hmm = container_of(kref, struct hmm, kref); - struct mm_struct *mm = hmm->mm; - - mmu_notifier_unregister_no_release(&hmm->mmu_notifier, mm); - spin_lock(&mm->page_table_lock); - if (mm->hmm == hmm) - mm->hmm = NULL; - spin_unlock(&mm->page_table_lock); - - mmdrop(hmm->mm); + mmu_notifier_unregister_no_release(&hmm->mmu_notifier, hmm->mm); mmu_notifier_call_srcu(&hmm->rcu, hmm_free_rcu); } From patchwork Thu Jun 6 18:44:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980283 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BFE711515 for ; Thu, 6 Jun 2019 18:45:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B20BA28A5E for ; Thu, 6 Jun 2019 18:45:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A5C2A28A74; Thu, 6 Jun 2019 18:45:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B08A528A5E for ; Thu, 6 Jun 2019 18:44:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 906D96B0283; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 88E2E6B0282; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 509D26B027D; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 0D8A96B0280 for ; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id n126so2748296qkc.18 for ; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=i7+8VVGKVPXEVnSFQY2/eN+heRnF32oClEuR9RZcz4c=; b=ZAnaYYj46DLWLeZQMLUGac0Cse0ZPNgDJe1JYHzTFCxaMRvgZlT2cA4WySk0oOgGaN O4okVW/4x0TfGIRRWn4uxOKseHk2qaxKUqJQb4eJ0pogXAjFtxu9IeVmO0u31818dpPE Hcorl4w2BlHNW75n5BsKWejzc7V9SM1b5an01+ulth9mttIVvQq2jvR455Jax2n8Hph6 zb9dYJCYtfmMJlWEX7729oEDgUwJPKwWGI3j6iViNQpHoLAxWBC8nQDvCTBFD1/d1++9 Y3M3L+yHFka6QGZmC3Jic/jWJt5HegtrFgXUdx4fSOm6GfFqMDGWKVaMq6T3ObfAtQrC pwgw== X-Gm-Message-State: APjAAAVBDs43v7ae8fFmtHwYude1psCEwzexQXX3+jBwH0ah17UcgzPx XiZn1I4YixUAyBWpff4yBUGNPkdFhMg726wLCs1Y/lUh0kOBS2ORDIM7Vm48d1cBwUd2MNtxP3B azFcssY3M3q/UN2ttuzhOvXdy6birJmTRME7bcxguQ5ufEnOKivQCDDCBs762Ubs9EQ== X-Received: by 2002:a0c:95ed:: with SMTP id t42mr7485056qvt.70.1559846688818; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) X-Received: by 2002:a0c:95ed:: with SMTP id t42mr7485011qvt.70.1559846688141; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846688; cv=none; d=google.com; s=arc-20160816; b=LIjZQ//q6oTlxWzXX6CKLHGsPkXlpxwjrk81xeVBf/yv1e/+L8eXm5JVElgNPtT+by fOoPya/tLpBjRXJm5nU5YdZasVf8Gr6Z9sDOd5RuDQDdcC533a4FrqS3s/CHKdhlH1sA LJdH2iBZIwospbhwaLNNyH6eQVBgaZH5RpC7xgeJt8u9zDNpms6d0fmLwoY6Mq3Hm0GI mPZX4vkbZmMPHqJZIn9EHG1QGGGTVFWxgaFDcUZ5SBf0FJVERSKYymxeFyCVCgEuvVGV fWosT8ruiT1fYwsdFjopu0p70XH/WlkNMM9BvrjCcoytw7r/A6qd2LmqkUftk0J6OrpZ 9Mhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=i7+8VVGKVPXEVnSFQY2/eN+heRnF32oClEuR9RZcz4c=; b=uCDbdvRQGmK4gxzZkZlfyqFrNwUHSCqBXzgM4MF97zyTR3ASK9o+mAijN+ifs+z3OF 4N9XWYp35FO2p/8Tr5Y/CXccw+Mk9/MM4y0ilao2VLeyhvBLKa0kovS+EXlYPEDjxK89 V85avDkYNXe5LdFNJkOZXuC4JgazuVoeDePU6zrey0NjvjWeKZ9Zm8q2ZWShSnEL2bye cyXJwIus8lXdsvgc2nTpTIKXR9kT/3J/Y1sWfjoTCkEcwTsakrlr40nmw+FgdGbhDklU eR8mOh18eYm2hI6jj1s9rmPJsYROwOzwiYnk1l6prWjiB+PE3fjG3EtbqxM6hVUCx/lt Emng== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=awv3zzYY; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id s64sor1455652qkh.114.2019.06.06.11.44.48 for (Google Transport Security); Thu, 06 Jun 2019 11:44:48 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=awv3zzYY; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i7+8VVGKVPXEVnSFQY2/eN+heRnF32oClEuR9RZcz4c=; b=awv3zzYYkAzknK7/NkCAiHR5B250y04djNp+WoYpexe7Uej2IPZ5cqDxrJR0439HDp Ip+tkYncoM6synNyttQstYPfXOCfvqJ+MFaPA020vji6f3V+ijeewe00T5WLlGVbBkip ta/ZcbdijmmX7p+PqFRh9FOoQWxBygNZCZjmbhPstNfn/mX4xsgnW1BaTUSYo/a8jPup YkMIRPOwMicKl54A6Y5BoXPtSil+jxKqN6rTDrLm8CETdgRy4WQzp8exudHeoEcxgA4t JqZxvdcH0B1B5+3aq+YyPAmAifoH1FgyXvEu+EF0wwHDyoaCdNCX+I1ObXgN34um9W9p meJg== X-Google-Smtp-Source: APXvYqzcNAbWl+BqHtgTaqiaVwnoaAOHy/kM0yjt/wGvoT6pCtDHNXa8FJTB1/3g0VUWQOgRUAVd7Q== X-Received: by 2002:a05:620a:1ee:: with SMTP id x14mr39905952qkn.70.1559846687875; Thu, 06 Jun 2019 11:44:47 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id f6sm1303617qkk.79.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008IT-Km; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 05/11] mm/hmm: Remove duplicate condition test before wait_event_timeout Date: Thu, 6 Jun 2019 15:44:32 -0300 Message-Id: <20190606184438.31646-6-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe The wait_event_timeout macro already tests the condition as its first action, so there is no reason to open code another version of this, all that does is skip the might_sleep() debugging in common cases, which is not helpful. Further, based on prior patches, we can no simplify the required condition test: - If range is valid memory then so is range->hmm - If hmm_release() has run then range->valid is set to false at the same time as dead, so no reason to check both. - A valid hmm has a valid hmm->mm. Also, add the READ_ONCE for range->valid as there is no lock held here. Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: Ralph Campbell --- include/linux/hmm.h | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 4ee3acabe5ed22..2ab35b40992b24 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -218,17 +218,9 @@ static inline unsigned long hmm_range_page_size(const struct hmm_range *range) static inline bool hmm_range_wait_until_valid(struct hmm_range *range, unsigned long timeout) { - /* Check if mm is dead ? */ - if (range->hmm == NULL || range->hmm->dead || range->hmm->mm == NULL) { - range->valid = false; - return false; - } - if (range->valid) - return true; - wait_event_timeout(range->hmm->wq, range->valid || range->hmm->dead, + wait_event_timeout(range->hmm->wq, range->valid, msecs_to_jiffies(timeout)); - /* Return current valid status just in case we get lucky */ - return range->valid; + return READ_ONCE(range->valid); } /* From patchwork Thu Jun 6 18:44:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980287 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68F8A14B6 for ; Thu, 6 Jun 2019 18:45:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5968128A5E for ; Thu, 6 Jun 2019 18:45:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4BB6328A74; Thu, 6 Jun 2019 18:45:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B544128A5E for ; Thu, 6 Jun 2019 18:45:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 24E206B0280; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1D6656B0281; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09EFC6B0282; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id D737F6B0281 for ; Thu, 6 Jun 2019 14:44:49 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id l11so2846594qtp.22 for ; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vDRIilFVldnvkbkMJDKjjlCneF+91QuQaIvOaI3TwIs=; b=HmFnN82E+p7OkX/hzxZz2I0tAbK9WRNRnYyoHqUAh14z6YSNObUHDFtCRCwUuOcXcM QQC29sB5o+aWof7CFxIeQqx63rdogl+EgMKPIC1IY55WKh4KfhBmX8vxNwsmb0axodhj wqdC67upX41Wf8R/0WNnqNmHjpmpNb39Vjj8jHvhlMa2bko1Bs4ElVq8eZhUx0JqGJCI H6k5iDzyJcR8ytVmj9sO0vfzTovZutUkS/H85c9fXNoNhhu2Ze2MPHH/IhI7CAqUzSNZ 4CfuGAHE23memUy+TSEUEbS28FJHp8eE/hfxjIKthP+MxI2qBsCEIsf6U6ePcQR5fvpE 7+Nw== X-Gm-Message-State: APjAAAXycY5MYlrCNotwZDFwWnRyNJDNdWu+LWEcVRb+qX8lAyQUnHGn yUUqxEfX74FjWW6gEZjyoZtWLwwbPzyEdL12+G63+LnmEmFs1VaeZYXVHJZeJvS2NbP5bJ0D5i+ vpId52PCKRDEjiDmeUJB0ClUW9PCUCYx/GtmnNYgd1rDvAXbzYpHc83vQIZ3xKsvkLA== X-Received: by 2002:ac8:3301:: with SMTP id t1mr36740577qta.209.1559846689606; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) X-Received: by 2002:ac8:3301:: with SMTP id t1mr36740539qta.209.1559846688830; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846688; cv=none; d=google.com; s=arc-20160816; b=jsj5B3d7x2IH3hcH5Pp/F3+XCCpujYPyGtdsAGnuNajMbOARKbJea6xZlBPvk2uQE+ xbAzZCcmVgtHwoGo2wJDA+2LHtx+0F6uAsQpqtOeqCoJ1lX2JoUIZYctpy5nEuyD5vUf TygkUssSVP/sNiPpmMl+ANuDjiXtV5jZnYS5AxN056h3RJpEo9UaI/2tCYHx6picvISv IBOCCkBdJa9dP+6hqaQsOP6si+uyUtQxYJRfwppCJG0uBy0AvFpOjBZCoMFsJzpeFdIF yPQ3jv+92MV2FHSwzirqJPatDKmptWMglMJpLw+2itUUIyhwiA48pbfh0Vkn1jtVmCJK 7u5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=vDRIilFVldnvkbkMJDKjjlCneF+91QuQaIvOaI3TwIs=; b=HSZhy/qQ+HI6YlkFGDzuL003wwrlumK0qs+6WAtdsKhzF12/nfHQo0XJS+R9P3kJwp xjZm6hQI9CjxDEZxRLq/Vw4HtGqv0Z3z+HrT1NPx02qII/heOdPDlcFMWkUO2GCthkZz yBHyKThbC7ILLOnSkMT86OS80u3asIPgRwEECnVQCvRJfHbwM0V5GSaVv3nkyVQ565DD OfKRgqqEGjEQqTODUR1Pl5ZdC7q1DihGIC/m/cbprNN2bZ7w57LNDYVr7RC8TxHVwERa Rx00IXI4Py1Xz+vkI8QOBcgDs0bob7H2FFo322GLYKVmpV2bdYKDkBd5OVrskBIfmUn4 Ht4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=QCSARRqk; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r2sor3143973qti.71.2019.06.06.11.44.48 for (Google Transport Security); Thu, 06 Jun 2019 11:44:48 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=QCSARRqk; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vDRIilFVldnvkbkMJDKjjlCneF+91QuQaIvOaI3TwIs=; b=QCSARRqkANiDuo9Zt0gh3n3vfALB6ijEiRwqIMaXCX3/qGzEYVq5zDKzohmDrx2kEq E/nLyL1hf/Yw2YVKQK2mevK/3x7wbrlWVevgrKFbIiCf3/EB6QRExm5LWe5UtxC9rV5d UQwAlzEs9kATjD+x85Cuz49AhNxPFC7gKS60FnQfbxtiMvkSooOuUBX+Iv5TIFtZIcqQ Aee5zzJxne/q8crQAD9Is+y1upCG3grwwPRQuj39icbmd1uTk0pCmr+5K/a+fe+W+cWn Ye97kdTtyZ2qAfyBP9hoRiiA8P1jinSjAUnj4sUnxQAhfyk59Kal+WLdZQytIQm+fu8L ZYtQ== X-Google-Smtp-Source: APXvYqxDHwC4C9hnk7VytBkovRfN98/hasmwmCm/dELdXMnufTdP+P+aFJn1ExmavzkvjsH0ryaMNg== X-Received: by 2002:ac8:28dd:: with SMTP id j29mr42963672qtj.34.1559846688561; Thu, 06 Jun 2019 11:44:48 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id 77sm871850qkd.59.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008IZ-M7; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 06/11] mm/hmm: Hold on to the mmget for the lifetime of the range Date: Thu, 6 Jun 2019 15:44:33 -0300 Message-Id: <20190606184438.31646-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Range functions like hmm_range_snapshot() and hmm_range_fault() call find_vma, which requires hodling the mmget() and the mmap_sem for the mm. Make this simpler for the callers by holding the mmget() inside the range for the lifetime of the range. Other functions that accept a range should only be called if the range is registered. This has the side effect of directly preventing hmm_release() from happening while a range is registered. That means range->dead cannot be false during the lifetime of the range, so remove dead and hmm_mirror_mm_is_alive() entirely. Signed-off-by: Jason Gunthorpe Reviewed-by: Ralph Campbell --- v2: - Use Jerome's idea of just holding the mmget() for the range lifetime, rework the patch to use that as as simplification to remove dead in one step --- include/linux/hmm.h | 26 -------------------------- mm/hmm.c | 28 ++++++++++------------------ 2 files changed, 10 insertions(+), 44 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 2ab35b40992b24..0e20566802967a 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -91,7 +91,6 @@ * @mirrors_sem: read/write semaphore protecting the mirrors list * @wq: wait queue for user waiting on a range invalidation * @notifiers: count of active mmu notifiers - * @dead: is the mm dead ? */ struct hmm { struct mm_struct *mm; @@ -104,7 +103,6 @@ struct hmm { wait_queue_head_t wq; struct rcu_head rcu; long notifiers; - bool dead; }; /* @@ -469,30 +467,6 @@ struct hmm_mirror { int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm); void hmm_mirror_unregister(struct hmm_mirror *mirror); -/* - * hmm_mirror_mm_is_alive() - test if mm is still alive - * @mirror: the HMM mm mirror for which we want to lock the mmap_sem - * Return: false if the mm is dead, true otherwise - * - * This is an optimization, it will not always accurately return false if the - * mm is dead; i.e., there can be false negatives (process is being killed but - * HMM is not yet informed of that). It is only intended to be used to optimize - * out cases where the driver is about to do something time consuming and it - * would be better to skip it if the mm is dead. - */ -static inline bool hmm_mirror_mm_is_alive(struct hmm_mirror *mirror) -{ - struct mm_struct *mm; - - if (!mirror || !mirror->hmm) - return false; - mm = READ_ONCE(mirror->hmm->mm); - if (mirror->hmm->dead || !mm) - return false; - - return true; -} - /* * Please see Documentation/vm/hmm.rst for how to use the range API. */ diff --git a/mm/hmm.c b/mm/hmm.c index dc30edad9a8a02..f67ba32983d9f1 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -80,7 +80,6 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) mutex_init(&hmm->lock); kref_init(&hmm->kref); hmm->notifiers = 0; - hmm->dead = false; hmm->mm = mm; hmm->mmu_notifier.ops = &hmm_mmu_notifier_ops; @@ -124,20 +123,17 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); struct hmm_mirror *mirror; - struct hmm_range *range; /* hmm is in progress to free */ if (!kref_get_unless_zero(&hmm->kref)) return; - /* Report this HMM as dying. */ - hmm->dead = true; - - /* Wake-up everyone waiting on any range. */ mutex_lock(&hmm->lock); - list_for_each_entry(range, &hmm->ranges, list) - range->valid = false; - wake_up_all(&hmm->wq); + /* + * Since hmm_range_register() holds the mmget() lock hmm_release() is + * prevented as long as a range exists. + */ + WARN_ON(!list_empty(&hmm->ranges)); mutex_unlock(&hmm->lock); down_write(&hmm->mirrors_sem); @@ -909,8 +905,8 @@ int hmm_range_register(struct hmm_range *range, range->start = start; range->end = end; - /* Check if hmm_mm_destroy() was call. */ - if (hmm->mm == NULL || hmm->dead) + /* Prevent hmm_release() from running while the range is valid */ + if (!mmget_not_zero(hmm->mm)) return -EFAULT; range->hmm = hmm; @@ -955,6 +951,7 @@ void hmm_range_unregister(struct hmm_range *range) /* Drop reference taken by hmm_range_register() */ range->valid = false; + mmput(hmm->mm); hmm_put(hmm); range->hmm = NULL; } @@ -982,10 +979,7 @@ long hmm_range_snapshot(struct hmm_range *range) struct vm_area_struct *vma; struct mm_walk mm_walk; - /* Check if hmm_mm_destroy() was call. */ - if (hmm->mm == NULL || hmm->dead) - return -EFAULT; - + lockdep_assert_held(&hmm->mm->mmap_sem); do { /* If range is no longer valid force retry. */ if (!range->valid) @@ -1080,9 +1074,7 @@ long hmm_range_fault(struct hmm_range *range, bool block) struct mm_walk mm_walk; int ret; - /* Check if hmm_mm_destroy() was call. */ - if (hmm->mm == NULL || hmm->dead) - return -EFAULT; + lockdep_assert_held(&hmm->mm->mmap_sem); do { /* If range is no longer valid force retry. */ From patchwork Thu Jun 6 18:44:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9767C14B6 for ; Thu, 6 Jun 2019 18:45:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89B6F28A5E for ; Thu, 6 Jun 2019 18:45:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C7F328A74; Thu, 6 Jun 2019 18:45:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F3A128A5E for ; Thu, 6 Jun 2019 18:45:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 88EE86B0281; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 81A6D6B0282; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A6626B0284; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 344656B0282 for ; Thu, 6 Jun 2019 14:44:50 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id s9so2863440qtn.14 for ; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GDhglHPFaykanjjoF1gAri47ISjb+fimKQAKOR+h9MI=; b=tO4FD/SbHgiETxLGb9hLMpAwzlHPiZHkk0+0wkJibu2dgPrDC3cX5/k850N6ngjB5P CnYfy50/9rUk/kLoCepBZdk1baGcvd87SrmwxwAIT8dSXdk+WxwvikByXvJ3FNIy5dBJ 7FLglnDyvwMqxpn8h/gnpOL+1yrgedr/IlMhZUq6y4lokzPX4vKPO4ChK4HHnAsI7Ujv 7zMfQWHZ+gn/qYZEahBvg8etA6OP73t3/GV5zjt1Elk1m8NdRM3TANACMLLkWLsY+CyJ SnIw+7qItAJqQnIBysB8lXtBquxR0VI68LYwNd/v7z1IX4BPZ2U4gPlUKcV/lmCKvNbT MIBA== X-Gm-Message-State: APjAAAUHCYoyZHTbuzGlRa9aSVIZ6VYUjhDq9BHZ8iCOW++nbzPb5fpZ FAvIR8KIs+lwdh2szWpUkiKe6sFiN4QHhtVjdk1znsbDLz9qJ97eCWoQu5HXaeJtDAUHGyjSXei h/zW1Zj7QEc7jDW1ITkSyFPdWYkNs4qi9xmGb8DhOaviMLWpKg/vpI9TE3HuhA8afwA== X-Received: by 2002:a0c:d604:: with SMTP id c4mr40051398qvj.27.1559846690000; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) X-Received: by 2002:a0c:d604:: with SMTP id c4mr40051358qvj.27.1559846689293; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846689; cv=none; d=google.com; s=arc-20160816; b=xJW0NbnkGBRTP+AlZMbMyibBV5NnWze1mtRqICpJ0weC54uCMYqjzlUO/CaVvr09ey H7Wyij8J1R6FMa+jjA/lQ5yXHNQj9pnEqJ0YVwdsWR+VfjwryPscMoJHALblKWXvL2Vy rD1M/M0Veyv86Jv+QhfhoeyjgI+HD5CmqSmNlY7MzaL/oCCb4VEcZdA1H0LHlInHzIex J0NwWV0Hogt18GXe7oLtjiD44RMfyV4fi2LRa6kxAIRHGy5VWd5hiNFPjumSKt+nWxEq qGrC0jhi5nuDnPc5PVLJyUqN1oz3bbiNl/ngOBhxkWVq2FG6QAhc7gj1VzqUZ6QV+Qzz 8DLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=GDhglHPFaykanjjoF1gAri47ISjb+fimKQAKOR+h9MI=; b=wMT981rFFxjntoBzu+wQTNIIBn5R4xsK4aQ9WgrZmcj1PrTXYqzBfvlqa0BWQ9DAne SfilKSdJodi7cE8Yv/CrUGBZ6w9EgjzG5eKwzwGgN0IDkx+bAmP9ZdPxMj6JNUI8YFap 9vodllTTPou17ntcrZO6p9tT7YegmjwIrw8BenI9JJKVn3g9wYb3jE3l3o4YYE8fkOIM sLf3uFkmpxOrzdk+Xx7vmAJw5bIHvx/fID2e7ktcpxUva4Y1qxs9XadZTn0TlIQdk9sV 9ukwBMrKCnTWOKAyPqsDCzk5+2ywt5Lon1j8THdDMWA0asooi8xxSSYm30PgAuGz6Zxn SIsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=PxwcFgRp; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j34sor3178805qte.42.2019.06.06.11.44.49 for (Google Transport Security); Thu, 06 Jun 2019 11:44:49 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=PxwcFgRp; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GDhglHPFaykanjjoF1gAri47ISjb+fimKQAKOR+h9MI=; b=PxwcFgRpu1yKtgG+aowdw5x+8tofPqN/iG+sZlMS+1BEgLEa2wfuOLDwkMIpgzpwcd 07df04JSrSYFDPlOBaTxTsHwx7kBZmstHd0a4oVfpGuC/g/Bhpf3Lt5nSoYeD6uPrpsQ QII6+kEeEkW/MUSdxlhOd0i2T95D5HdQdwlL7Ozqf48joL6SQYTkyLT5peB4+TMiTij9 aIRgom5VV0qkHk9b2ZOSyXv7P9Z7i69saCcXZ47LPPfmx+ZPD9nsxaz5ycev8w2P6F6R TEDZZY+v7m2CrZJVilSqXEyPa+l6AO++F7IzKVzBSzLlDLqpq+DAzeUJPhu5fEg2sxO9 4usQ== X-Google-Smtp-Source: APXvYqyc+KhO/w9mb/yqATHQoXsEv3qzyCLAVUEuPTK/WR5bBDKxmiMb1le/dUnjSV/gm3c3d4B8Pg== X-Received: by 2002:ac8:f0a:: with SMTP id e10mr39961260qtk.325.1559846689071; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id w34sm1260252qth.81.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008If-OA; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 07/11] mm/hmm: Use lockdep instead of comments Date: Thu, 6 Jun 2019 15:44:34 -0300 Message-Id: <20190606184438.31646-8-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe So we can check locking at runtime. Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: Ralph Campbell Acked-by: Souptick Joarder --- v2 - Fix missing & in lockdeps (Jason) --- mm/hmm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index f67ba32983d9f1..c702cd72651b53 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -254,11 +254,11 @@ static const struct mmu_notifier_ops hmm_mmu_notifier_ops = { * * To start mirroring a process address space, the device driver must register * an HMM mirror struct. - * - * THE mm->mmap_sem MUST BE HELD IN WRITE MODE ! */ int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm) { + lockdep_assert_held_exclusive(&mm->mmap_sem); + /* Sanity check */ if (!mm || !mirror || !mirror->ops) return -EINVAL; From patchwork Thu Jun 6 18:44:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980291 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B0121398 for ; Thu, 6 Jun 2019 18:45:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1BB328A5E for ; Thu, 6 Jun 2019 18:45:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E5CF828A74; Thu, 6 Jun 2019 18:45:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 964EA28A5E for ; Thu, 6 Jun 2019 18:45:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66A276B0282; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 61DBF6B0284; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BC9A6B0285; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id 2E16A6B0282 for ; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) Received: by mail-qt1-f199.google.com with SMTP id t11so2885432qtc.9 for ; Thu, 06 Jun 2019 11:44:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=W85iZVgjurkkiB2a7Xxec+qLci2n6gFQqX8RW+I6lJo=; b=oHZjIpUIUZNp3ghWTRDJQQLd/disIkhkgMBaxe14KTtCqmTUKDVO2WHKNMS0RcCJEO YNsp3OVcyG1OjE47BThwVtoZXH2RymLQoL+XH/JqQSANYIueRX4d9u/XDwXCp1BL5evl +YJGQucO54c8sUEo/FemGR1JdWhGmOIXqqpuMXHssmwXq3NQbIrsy7J1e1c7ZoWRvlpr MyPEWXGDbhEQYUNC2x5KUg2XwbqKvSXKogVsemQfrzizYIhiTRSYaY4pgJ2uOewgPBaR vMOv2Gt3k68Wjj8mov1dIgEXXRKosetVqwliBYO2p+JDbCsN9UEvwysfg343Z3ipsgBw iuuQ== X-Gm-Message-State: APjAAAXeuHuOQ8EliKim3x7TOjBQBTk4IoggT0rdey7NDBekRQ+5frPP sZuM5nVL/D0dPPcBMzHRcnaUNG1Vm6AFvRkHzS1yEDmLhxzfJNJo/zzdoLEvqDEBCpLiJK6/Cgx H3ifzSo17Hq18kGOplQ6XvOAv5hPTHtxl87hYrPjKZ+3QA0khlxziVtIaeG1DlQzrXg== X-Received: by 2002:a37:bd45:: with SMTP id n66mr38370801qkf.81.1559846690946; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) X-Received: by 2002:a37:bd45:: with SMTP id n66mr38370768qkf.81.1559846690424; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846690; cv=none; d=google.com; s=arc-20160816; b=VP0wpbm+Akpj2UNnTCxayJVMvns4Nq/oe9VGkpt+Bx7BHvo9E4fhI2m5DxvWrVKzI7 GKREu8xlpqoMiNU/g47EEw8cJ/P+6RTk+DbgoCT5JkKU/uvOFrES23ydTd0y6mumEp14 FQRUsAOqrtpXquIj/uYZFNk57gvoDI0/JeyheG+HlkgwezyP3wwKRfSBeGeFRTi7e+XZ a7sa0M/aXx830FMErh/W6ArI3eaUPM7lzI6nN+NxIPiT8Mok8KXDTFmLYalyMZjsNR2L N4+9fSg6lleC/HhK1l43A56vPb6mXasgKRcyFKOhfZ8Gcvr1Xhw7YtRv6FsHdeHy/Ijw ysiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=W85iZVgjurkkiB2a7Xxec+qLci2n6gFQqX8RW+I6lJo=; b=fwV6NKmnIFpTMy/IrdVxbSzP/FWW9sWerXuvUQ2s1kIvGmFnzaYH3CQgcKWcayGt29 pH7tCU11PXHU05drxlDKbRU67vt41iH1sg15Ql6vSIWczW0+SHCHRh1zEnylTCdb53Pb gq6xtOjx5rct+w/Zm2E5Ue8/fZYeYAwpXhyNVmyyc29tDx04zM/omn6LU7OpcsAKYSqw 7r1frfZQJJF7oHK4TrdEyb0Mxt0BDkkuMzq0ehKJltGt67OX1T3FlaZOtbjHPXQ3GqEv ytV623J+etWjgAxfUJS+tn+rQXtjdIoyT9t1CqFgrtyxPV+56KtsEAZNckwrVZRU5lbh xtrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=EIOU+fw3; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.41 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id c8sor2108084qvj.32.2019.06.06.11.44.50 for (Google Transport Security); Thu, 06 Jun 2019 11:44:50 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=EIOU+fw3; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.41 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=W85iZVgjurkkiB2a7Xxec+qLci2n6gFQqX8RW+I6lJo=; b=EIOU+fw3tMulwBY+jl6o/Pd7uN1ahiKKDLJKphj0SFrY3mGFeHTn1W2VV4oFCcXr7j cWmZ0gDNudc595ZVgI5l/RmqYCimcWM0wl22l9mGjVDztbCe051fJtkZna0E33TmapUB sttvDBWVXq87dQ+6H75k5wM3twYt5qFdBFS/o7++FbtFqaun0Q5YuEHy1UH3O9BInXzS pIETiNwzZOWJnzWU1GfKDiH4km5IjkASxVzt5h0rLlByvN4VwKEz5TKRl7R8CRCPPHgC iZGZpOcpwV/Xe9ToHHJdY4NMDuY9EmTPikVd4pQhwsuT6b7FpmiV8btfYq7kPkezkCcc rh8A== X-Google-Smtp-Source: APXvYqwcFvSukVvOu/ecnBzPeUt++yQE3oiNuMCzOwlWEIQcAIcfXxOQH4AMIgicmlTTSzjQ5g0HvQ== X-Received: by 2002:a0c:989d:: with SMTP id f29mr21429512qvd.209.1559846690185; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id p37sm1643204qtc.35.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008Il-PR; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 08/11] mm/hmm: Remove racy protection against double-unregistration Date: Thu, 6 Jun 2019 15:44:35 -0300 Message-Id: <20190606184438.31646-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe No other register/unregister kernel API attempts to provide this kind of protection as it is inherently racy, so just drop it. Callers should provide their own protection, it appears nouveau already does, but just in case drop a debugging POISON. Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: Ralph Campbell --- mm/hmm.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index c702cd72651b53..6802de7080d172 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -284,18 +284,13 @@ EXPORT_SYMBOL(hmm_mirror_register); */ void hmm_mirror_unregister(struct hmm_mirror *mirror) { - struct hmm *hmm = READ_ONCE(mirror->hmm); - - if (hmm == NULL) - return; + struct hmm *hmm = mirror->hmm; down_write(&hmm->mirrors_sem); list_del_init(&mirror->list); - /* To protect us against double unregister ... */ - mirror->hmm = NULL; up_write(&hmm->mirrors_sem); - hmm_put(hmm); + memset(&mirror->hmm, POISON_INUSE, sizeof(mirror->hmm)); } EXPORT_SYMBOL(hmm_mirror_unregister); From patchwork Thu Jun 6 18:44:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980297 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E37F1398 for ; Thu, 6 Jun 2019 18:45:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 553B028A25 for ; Thu, 6 Jun 2019 18:45:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 485BB28A49; Thu, 6 Jun 2019 18:45:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DBBF328A25 for ; Thu, 6 Jun 2019 18:45:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3626B6B0286; Thu, 6 Jun 2019 14:45:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 299C86B0287; Thu, 6 Jun 2019 14:45:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0ED5E6B0288; Thu, 6 Jun 2019 14:45:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id DBA6A6B0286 for ; Thu, 6 Jun 2019 14:44:59 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id k13so2784626qkj.4 for ; Thu, 06 Jun 2019 11:44:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=m5r23PQEATRs0yuo7EiwMQdVkzY1U+YCz5n+ShEeoJ4=; b=o5HYA0YDcOKvnJ+of5HeyjSx7gD5OJinuMKqRwfZ0pNvCPFGWGCfqWZ91g2PdO43WN RDadySjgdfmR+ACfUw9TwjPN95mlxLxlDH+rBbEbawVcmHTE0Faud7bJwocKuqlU8buj XcQtKd8ufy746Ldff1Nm0APaIfSOM/ZcUESk3Yt1eH422gqpvljSS+GcJFRf90/Ft0p2 1NLa3MdUzUvOQFVdlUmJ8pjqsq93f2IgchqqRGuxrABl7ZDwvsjRlHBQ7BxaSW1nbABF Dp1gSXP2X98H4XjkJHvPck7TCLAv3Mr2Iy5W1guL/ZfUuTiZhY+B9LZE62n4PiBx9Jvo wi+g== X-Gm-Message-State: APjAAAWs0TF/gd0C2BXX8aJK0Hm+hggSN+GPLfO+AMN+9a2B5Hlb8SNF /FgAysaz4M8l4LbDlaG8fxiR/EG7Rw/Z1MqOvbX3cxUjE7HgACkNsT2ud5g5tJaiT1Rd/35WTdb u2EnWkFxQlpXO3eSbklbciSmZKSsYComh6KANILHL8Al1wHH5POJRSl88z/ozhEJ08g== X-Received: by 2002:a0c:8b49:: with SMTP id d9mr39265537qvc.63.1559846699484; Thu, 06 Jun 2019 11:44:59 -0700 (PDT) X-Received: by 2002:a0c:8b49:: with SMTP id d9mr39264968qvc.63.1559846690575; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846689; cv=none; d=google.com; s=arc-20160816; b=ef6idjr38wp9bZ2JqlDX54v2LCBOr920g0oygAmC7fiMZ7wZKganvKR8X+69+eEd8Q 3eSwbB0O1JBwCzXZBcVnZkF355We5axKXD8tfa5jZE6okFFpLyIGeaAOeSXQUsaDSE/g danZqHYqlrUvQpit94T07XPuAasX9oyAGsiJMmVC1jN4+pyEvcgnyIBYC/o+FzrgfQMn WaWDkJWlOpHLhS60IhcqBAs9h82oUvzpBqBXPDirleP+RVQtaPO0g82MOlOyIeceVYFr IUZLJjQ0wlAXwtE6lF4hD++RZ+n7QmlhsqFR2DNGgKTDwWmLVOz8DrpG/x36yuU2PAmJ aY5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=m5r23PQEATRs0yuo7EiwMQdVkzY1U+YCz5n+ShEeoJ4=; b=n+XAh67Co7tDY2TQSGO2pXyDuMVy3a63ZoB2LEG7+0OjEh6opuPT+2WkI82SES88f9 antgoYsbvyuv+9Ob9DkSNd2hdie4bzumYbdN1oId+dW+3lfmp2/lYmj5t28eTHL/KFJM QuTDrfKDLr0q4it4yg+zfBrX/lIjkX7sm+dgW8kQmlIzYb60K1GLc9xiCLce80W8HITW duMdfzTJeD3JvaGR1ahOcqItorCgbVHbyKRZAd7MvjNi2yqIIT1+ZX49LS4vxU4FIeSN RnmmPsZTqpy5mVvXa+nQQYJHwByE0RW98q8PIlkizcsRtmbXepzjcBHgmGFarBlxQa/e O/Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b="NWo/t5Eg"; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id o65sor1458877qkd.6.2019.06.06.11.44.49 for (Google Transport Security); Thu, 06 Jun 2019 11:44:49 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b="NWo/t5Eg"; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m5r23PQEATRs0yuo7EiwMQdVkzY1U+YCz5n+ShEeoJ4=; b=NWo/t5Eg8vkPop9rmLEJM24/TPCJSDGLkfJSg++y8tqe0YH7o0F0c+BLBxXYqlWbHn TQBM+rvo5UiyMgZHvqK7Pvgn4ARc08XbQ20YAC3fqCirAP5zyCBTuaRKGYQxA5+ez9MY 5dtAcK3pg10dTEX2QJmbgVOe2q4izOeIa/IhaKfFTg1LtlOqd5CgPkliph8c0pVndS4n r9g5yTZjE2Bz4PyNamQroNF8IT3K4ojSe1Y0KMRcIsSxPFsRL0WgkcnPYGMbxXNaACfs /YPv4ioHw6zlpgsrcrlglyXYrBLyXki3e7XQt67BaOsoljLrmJ+pampgcg1bCjvTHEk3 u1yw== X-Google-Smtp-Source: APXvYqyDTfgtep66c/jLgBTZULmgoKmkAN7hyfuI/DrREbp9gQ0U06D4ocKYi5xCroJDWLKbh/kYjQ== X-Received: by 2002:ae9:c30e:: with SMTP id n14mr34724569qkg.220.1559846689590; Thu, 06 Jun 2019 11:44:49 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id s64sm1267327qkb.56.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008Ir-QZ; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 09/11] mm/hmm: Poison hmm_range during unregister Date: Thu, 6 Jun 2019 15:44:36 -0300 Message-Id: <20190606184438.31646-10-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Trying to misuse a range outside its lifetime is a kernel bug. Use WARN_ON and poison bytes to detect this condition. Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: Ralph Campbell Reviewed-by: Ira Weiny --- v2 - Keep range start/end valid after unregistration (Jerome) --- mm/hmm.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 6802de7080d172..c2fecb3ecb11e1 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -937,7 +937,7 @@ void hmm_range_unregister(struct hmm_range *range) struct hmm *hmm = range->hmm; /* Sanity check this really should not happen. */ - if (hmm == NULL || range->end <= range->start) + if (WARN_ON(range->end <= range->start)) return; mutex_lock(&hmm->lock); @@ -948,7 +948,10 @@ void hmm_range_unregister(struct hmm_range *range) range->valid = false; mmput(hmm->mm); hmm_put(hmm); - range->hmm = NULL; + + /* The range is now invalid, leave it poisoned. */ + range->valid = false; + memset(&range->hmm, POISON_INUSE, sizeof(range->hmm)); } EXPORT_SYMBOL(hmm_range_unregister); From patchwork Thu Jun 6 18:44:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980293 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 664AE14B6 for ; Thu, 6 Jun 2019 18:45:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5868D28A5E for ; Thu, 6 Jun 2019 18:45:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C4E828A74; Thu, 6 Jun 2019 18:45:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4E1A28A5E for ; Thu, 6 Jun 2019 18:45:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED07F6B0284; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E58BF6B0285; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D209A6B0286; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id AAEEA6B0284 for ; Thu, 6 Jun 2019 14:44:51 -0400 (EDT) Received: by mail-qk1-f200.google.com with SMTP id i196so2754819qke.20 for ; Thu, 06 Jun 2019 11:44:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IcIuNwf0zSIjR7XDITYyS7Upr/ibuj30Et6ThDetGUk=; b=EzD+WOAkzAWuTtOF06iIbrbfGOzjcJmv7syFvk+5ziTfR9kTPjEHk/EZ02lrRRXk+h yq/z49XY7NFY99QkV7+LnefqcQJt6TmKhOSG9LO40GJbMLd0/UFswxGLff0QqKHawu+o Q0OprvDmXEBkRV+vry1ByKWkXBSHZh70YrrX2wyGSK+jZ2pSbm73rRQxvoeNqFEs0PVb BWTjyVors2tX7qLghwdf0OMA+LnZSUbC/STQCb4LxsINMMwfwCNK4alNQNsk60YK1q6l QsPusuOq5RdpkzGNCpT8tBi8gxDfROlUrYwQ1zI50RmhLSq9DWICigWgdpxA9Ux40Pe8 ahrw== X-Gm-Message-State: APjAAAVtgJYpf/AJXeotf1qlInJvfLP5UMV9zS8zVm2W55UwL5hFAnbK +0nraM01K/i4g815nWv4kf9hAj+cKQTFRj/Sze7ENqM2+vhtuGsDd53y7SmN5zlHqsfVeQlq+ly E8ZT50k58bHNpdBqPG0kY1WzakzKP/fAIPOUwRALfQnIUT0EF71FhwoMnWobnf/0oSQ== X-Received: by 2002:a0c:c164:: with SMTP id i33mr21776331qvh.37.1559846691452; Thu, 06 Jun 2019 11:44:51 -0700 (PDT) X-Received: by 2002:a0c:c164:: with SMTP id i33mr21776284qvh.37.1559846690715; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846690; cv=none; d=google.com; s=arc-20160816; b=rCd41AaZ3o2PPH58BGXi2GummjpzGA4bSk8ObvA+vbAPjh7pWD1zqm+GvJJ5clKzOS Sj+ZC+Mj6Ow/POM2Be8KMIbQkF+KVHd5Csj/TwO8XAF4eGGqIthanq0ncUCctBJVkdhC SiYY3dkT1hjSefzZVX4cmq4U43DVTvuLaqpS8qBbF5AHFsgVwV0oUmp40GmtsOvgozlX 7iuruWaBucxydGcIiJHMYfDd0LR+WFUgowTn+l03xrBLvpanmcIQ5MVjnKcgoRX14w25 /2cR5dCVc9tn78LJH1Quh8C1dDu+SohWPg3b9R/fruMZkQ53W715toEwwWDCnfH6I9YJ hDGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=IcIuNwf0zSIjR7XDITYyS7Upr/ibuj30Et6ThDetGUk=; b=gcnxqQxrwwI+Ua0+nukZHNTctTkFx+0Gn9uAof6Guhqk7vB4txnof9L2Z0oWi46QBj LEemWXgDqt0lEJlCadQtfmatQORca4/RR4XTW4DChdbao6vA9fclcSkd/TomMoe/pXnS ATFfWcRQWepj0HIeqeZstgbEPapDO3KjbS7jwFSUh1wiTj5E9QC1IZLcA04dqHm4ftM0 XBR3M0ldE22KUHBrZlNlO2yH9LubsW9stx8EvZrI2COVtv8TGZyE5AxNKwpF7DDTloxw zEwCZJFBvXFgkv4v5BKQCt+j42rBoeNAdyEsjzTL9r6K7OLzeMrZ208ie3eg9hkyXEom byvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=Rvt6PYlH; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b11sor3160196qtc.50.2019.06.06.11.44.50 for (Google Transport Security); Thu, 06 Jun 2019 11:44:50 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=Rvt6PYlH; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IcIuNwf0zSIjR7XDITYyS7Upr/ibuj30Et6ThDetGUk=; b=Rvt6PYlHJVz8B4LawxVuziHGCwRJD2Qr2qMiY3NEDUJNQxIcviDgQFIUHpNOBK6bPf MIIBJmiWVVkfUMbHMVhNgYzIahm6Aa7GUFAb3gMyDxztH5h7Eb+fHHV92q7izqEBwwUK enoyn9lRGevzhavWgW36BHvjzT31gUq2EFNwzhKeKkZTpVNTzzVvik0L2TrM3EnBxw1d HwLNV0REutZrfhliSSQTwWrC1elVFzjojR+FeHHh1CsZjd+KT2xql8gm7opCTrlBcvvX NVsKXd0JcUf5SXTHZ3LhocSEFVCL79+sS0K2915hv11I6abm0hUCKJT6F6DnKNs8vXBi N3Ug== X-Google-Smtp-Source: APXvYqykn/mITkIwjkB65/z/6RCijqiu6Ix2iI3uboXOjS2+VibgJIHgW9EaDOqfs4y2pX6u4b1Wyw== X-Received: by 2002:aed:33e6:: with SMTP id v93mr42686308qtd.157.1559846690472; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id y8sm1656836qth.22.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008Ix-Sq; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 10/11] mm/hmm: Do not use list*_rcu() for hmm->ranges Date: Thu, 6 Jun 2019 15:44:37 -0300 Message-Id: <20190606184438.31646-11-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This list is always read and written while holding hmm->lock so there is no need for the confusing _rcu annotations. Signed-off-by: Jason Gunthorpe Reviewed-by: Jérôme Glisse Reviewed-by: Ralph Campbell Acked-by: Souptick Joarder Reviewed-by: Ira Weiny --- mm/hmm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index c2fecb3ecb11e1..709d138dd49027 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -911,7 +911,7 @@ int hmm_range_register(struct hmm_range *range, mutex_lock(&hmm->lock); range->hmm = hmm; - list_add_rcu(&range->list, &hmm->ranges); + list_add(&range->list, &hmm->ranges); /* * If there are any concurrent notifiers we have to wait for them for @@ -941,7 +941,7 @@ void hmm_range_unregister(struct hmm_range *range) return; mutex_lock(&hmm->lock); - list_del_rcu(&range->list); + list_del(&range->list); mutex_unlock(&hmm->lock); /* Drop reference taken by hmm_range_register() */ From patchwork Thu Jun 6 18:44:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10980295 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EEF8314B6 for ; Thu, 6 Jun 2019 18:45:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEB1B28A5E for ; Thu, 6 Jun 2019 18:45:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D2B3528A74; Thu, 6 Jun 2019 18:45:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A21128A5E for ; Thu, 6 Jun 2019 18:45:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F1796B0285; Thu, 6 Jun 2019 14:44:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 929666B0286; Thu, 6 Jun 2019 14:44:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81A8D6B0287; Thu, 6 Jun 2019 14:44:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 62F586B0285 for ; Thu, 6 Jun 2019 14:44:52 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id n126so2748409qkc.18 for ; Thu, 06 Jun 2019 11:44:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nVAUpO+Ga9NYec9bVUlta7oVxGoK2kbyT+1hUTcsX2M=; b=Ph8hbVx7rUc3HCIZtyYa+YSbb5/H56M+kPe900gY+wjXs00seXefzbOlMFQF85jYPU 9C4MZPeOG3JG6oRQqG3ZEgoojaw0ve3EXIRIGd6BJdO7FAi2Uss8tQFPJPvcuH1//ViR zmuuS3jhHOhzatFXfVvWRQpvlqXHbx+3CLe9lUHQRNgLyiWscJLA9SAg0mEcLsWZJGg/ j4B1wrxGLkjHxCi+Ve0smzIjWiSFbisQ1mGjqWltiMqE/83s78a9HGTsQEzjdrJ1M3Od kw/VChtBva77I9mNNCm8WGl0A2kW+cw2MhaSRj5Isuucwge1e1ycwd8/aXFLEwt6QTEH tOsQ== X-Gm-Message-State: APjAAAWNglYJjg0a+6JIho3rNg57BYcK+0JlT3qZurAwMU9kffFbLjKI oWjXys3zDqIj4JYn8xBVqtT83+mCKeFN7gM3s7gZ4sGMGv9Xfu1wBLkQoTxsAAvq1zsu3/BfoGI f2+nAzyyO7B47FgX1G1wzz6LoBJVttrdKTd9xyhTXbnyfqpha9MKlMaKv80x76xw+/Q== X-Received: by 2002:ac8:2d69:: with SMTP id o38mr28339674qta.169.1559846692179; Thu, 06 Jun 2019 11:44:52 -0700 (PDT) X-Received: by 2002:ac8:2d69:: with SMTP id o38mr28339618qta.169.1559846691179; Thu, 06 Jun 2019 11:44:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559846691; cv=none; d=google.com; s=arc-20160816; b=TV1S35aakmMFMVAk9ETx9EJ6+zLA+HOz6+17O9reb/xCh1hSrxypl9Q/8760XZbDZL DN6WNW5oJCvZfCRJTl1nlVzCfcMmr0GeHI/bjDPu09DUiccMG3jCcoKelW9xl7NHS2HV N737VNFW/BgUqu77YARQSJf2a64HwHH0cR7iQB232xWbrPJ4mk59O+IU1qXOLeJhS0Qe TxDQjWAD15dZib36TYYWkLsjsyV7Mu60kOb4xKtYCdlFnSBPYiRiUOsGa412UU9o/Ofp AOVLEYSdQDHmAIEAsPNiMsW8eis/VU46ZiOM7FyGp6TzxeaRw70iituTOMfCOo//Mt9m Illw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=nVAUpO+Ga9NYec9bVUlta7oVxGoK2kbyT+1hUTcsX2M=; b=KT1Nj6NShsr7O8RUldkkexCpEqQzhpp6rD0i5LegdQLenqv28d3P+XRwxgR/Q0Da0B hf+uFHx+GvFZ1grptTQ1w3qSV3rD5YNKAR6I/hDoj4yY4P03xEGXbDemPEzKZg965xXI Y2qBaUHH9kqpjDg9Ip0wWWS8ddbyRAyouqDviSCjzyc5SaGiZESJGNZmyNN4bkCMuQEv mnKNdQgieQ2068QK2KvIUjaDr5AeWziUd/y+hPyf3Azrn1b/vl+FpHQyQcJgQXX/BXbX qQZhpQo3+oRNd3XTAh8yR4bCh0G9wi2sSBF8A2/NBoKO9MPm906vmSKHOPVDyazA3JEc cEfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=WgY47IGZ; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id o18sor1430337qke.38.2019.06.06.11.44.51 for (Google Transport Security); Thu, 06 Jun 2019 11:44:51 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=WgY47IGZ; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nVAUpO+Ga9NYec9bVUlta7oVxGoK2kbyT+1hUTcsX2M=; b=WgY47IGZKwCscEhmp/e1SH9Qdm9WcIPMWuHYPXHgf1Mke+ckY9UbBST6Yg3085YiGZ dcNR69Qrxwy8qK6hBe1Ue5L8mtizg8fgMYwpWlMDyzW4sJZM/YSdC7J0JCTA28fxl76n +LUXoIcNaktF9sMJ+8Tn9DSETKfpEaKYxfe0XcgkeYX9s25CuXN7zKQgSo93XiKMF8Ia uyDd1lBZZfguzvqyPsXI94pwdw1QaF2JfejyOIr3xXYPHSXV060VLQByKbXFlIqKGo61 qP5nXZ0Y1Bzv69Slh4GEkWmlgkAOHNc2qWHPy706QC/VjevthD4d+IaR7G2877aUIVh2 s12Q== X-Google-Smtp-Source: APXvYqy0lCJbH1BvPYOo0rOmLYXG/g+I9HtsZNodsB/KhCTBbD6BMVIIs/zFfGrbgs1cd5AeX+0slQ== X-Received: by 2002:a05:620a:16cc:: with SMTP id a12mr32024122qkn.256.1559846690927; Thu, 06 Jun 2019 11:44:50 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id q36sm1951613qtc.12.2019.06.06.11.44.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 06 Jun 2019 11:44:46 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hYxNV-0008J3-Ud; Thu, 06 Jun 2019 15:44:45 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe Subject: [PATCH v2 hmm 11/11] mm/hmm: Remove confusing comment and logic from hmm_release Date: Thu, 6 Jun 2019 15:44:38 -0300 Message-Id: <20190606184438.31646-12-jgg@ziepe.ca> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe hmm_release() is called exactly once per hmm. ops->release() cannot accidentally trigger any action that would recurse back onto hmm->mirrors_sem. This fixes a use after-free race of the form: CPU0 CPU1 hmm_release() up_write(&hmm->mirrors_sem); hmm_mirror_unregister(mirror) down_write(&hmm->mirrors_sem); up_write(&hmm->mirrors_sem); kfree(mirror) mirror->ops->release(mirror) The only user we have today for ops->release is an empty function, so this is unambiguously safe. As a consequence of plugging this race drivers are not allowed to register/unregister mirrors from within a release op. Signed-off-by: Jason Gunthorpe --- mm/hmm.c | 28 +++++++++------------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 709d138dd49027..3a45dd3d778248 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -136,26 +136,16 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) WARN_ON(!list_empty(&hmm->ranges)); mutex_unlock(&hmm->lock); - down_write(&hmm->mirrors_sem); - mirror = list_first_entry_or_null(&hmm->mirrors, struct hmm_mirror, - list); - while (mirror) { - list_del_init(&mirror->list); - if (mirror->ops->release) { - /* - * Drop mirrors_sem so the release callback can wait - * on any pending work that might itself trigger a - * mmu_notifier callback and thus would deadlock with - * us. - */ - up_write(&hmm->mirrors_sem); + down_read(&hmm->mirrors_sem); + list_for_each_entry(mirror, &hmm->mirrors, list) { + /* + * Note: The driver is not allowed to trigger + * hmm_mirror_unregister() from this thread. + */ + if (mirror->ops->release) mirror->ops->release(mirror); - down_write(&hmm->mirrors_sem); - } - mirror = list_first_entry_or_null(&hmm->mirrors, - struct hmm_mirror, list); } - up_write(&hmm->mirrors_sem); + up_read(&hmm->mirrors_sem); hmm_put(hmm); } @@ -287,7 +277,7 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror) struct hmm *hmm = mirror->hmm; down_write(&hmm->mirrors_sem); - list_del_init(&mirror->list); + list_del(&mirror->list); up_write(&hmm->mirrors_sem); hmm_put(hmm); memset(&mirror->hmm, POISON_INUSE, sizeof(mirror->hmm)); From patchwork Fri Jun 7 16:05:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10982321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 793DE6C5 for ; Fri, 7 Jun 2019 16:06:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6427E28857 for ; Fri, 7 Jun 2019 16:06:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 584D7288ED; Fri, 7 Jun 2019 16:06:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A726E288B3 for ; Fri, 7 Jun 2019 16:06:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 802B76B0007; Fri, 7 Jun 2019 12:06:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7B24B6B000C; Fri, 7 Jun 2019 12:06:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A10B6B000E; Fri, 7 Jun 2019 12:06:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by kanga.kvack.org (Postfix) with ESMTP id 4C5566B0007 for ; Fri, 7 Jun 2019 12:06:00 -0400 (EDT) Received: by mail-qt1-f198.google.com with SMTP id q26so2247014qtr.3 for ; Fri, 07 Jun 2019 09:06:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :message-id:references:mime-version:content-disposition:in-reply-to :user-agent; bh=mDQI7vDRgVEqfEUUaCyaBBK+4bOYH5UfB7o3hA1mJXA=; b=R8XbfYvbiL/I3Y58CoC0Tjy0wnuQ3drbs3jZZ9s+VzMn3f5vJUIkU+EE0KsDaMe1Gx kscYIdLF3vXNzTT4Dawxz11xR0sa8GkYOzqPSNyV6MbgWJ6+sy9yIZ710Z5apWpBUu+r U/2iqh9jRdENRSa1qmAEH0Gu5TP9l/+5xUVUTL0PwbCJyqY4MSCVG+le8BfFTUiQHIb9 tWE114H2/yn77dUXTUpB0YQZ4O9N/Su/6yOlDgoht4Xw2HbKENYt1mAVXwkaGNss68uu ke51MeHRExx6FM4ifZ2Z4gkXA5EHkzX5EeuSHOJjBD/fF0365aB6ODRNZ7kqd/PgvOto eo9g== X-Gm-Message-State: APjAAAU7+QnVZ84RiQb6Y/YHpNyTlw79ZKI2l7Qi9T7DPn703S4rUM8Q Tf+SEPCwiVh4jsIu2RK9WESteKNxdykkIsAMXPsIcvQhR+DXwdqgU88tIUFhAsv4uU3ZyEd1bdP Z/rb35CSMNW5V9ddWB83daTZnUigQ/jfxl26cr4vP3tS6seMnbXaFJijrWp6UzcX5Bw== X-Received: by 2002:ac8:28c2:: with SMTP id j2mr45987407qtj.103.1559923560021; Fri, 07 Jun 2019 09:06:00 -0700 (PDT) X-Received: by 2002:ac8:28c2:: with SMTP id j2mr45987316qtj.103.1559923558871; Fri, 07 Jun 2019 09:05:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559923558; cv=none; d=google.com; s=arc-20160816; b=LWNNSKq2fzP0Pz6ykMLc3oXQo+MAz/4mu53RpQS81Nx1E6b9ETz9z+z1snorSj62ZX NYRf0pTAsMlUtNadpF/YLRUVYzizrbXStrP7nP5KQ5Wa6wWwtwyZlrM3UuP3AgEjPJft tDmGNlfmGbozGS/PO0CSPrOzkSova1ThbuLiJghryieTNlIkoyY28XiKS077EQuJ/8oR /zhk1v24BX78/r8YjCbpOvFI1ss4evY4D23UV6o9ZuA9ov5zjVTzTcuOrT1TgAuqcqdR hT6BDenCRuhzXX0VgXI5eSfsf9DKeCZk16mxSPFT0eq8lfal41QI0Qb2dawpH5G3/ZHB EIQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:dkim-signature; bh=mDQI7vDRgVEqfEUUaCyaBBK+4bOYH5UfB7o3hA1mJXA=; b=UGYGkM6VZMfCc8KAsHsr5HAEgThs9JwrHzezJuMPnajxEeFn29z9lZSph6N6rLdYcl IZ7OUeWfy3vJHu2Q1nSFjZrDE5eDgP9qwhQ95THeox71rMvCYgLe992In937FnyGu22A blMaCMIny3IouEMK/d42e1Wu/5MxjM88pT8c1WbJxK1gGriFMXCohcVn+oxx2F24RCRR EQf3k7IrCPQ90tk7RHV175CBm1XCqkVPsTbcFJSu6p8uEmcr4Axgjdyx/dAMIgDuuAWa T6Dvgvb15ATaTL9lyFXbdegs734bcu0N1/4ddAjhj/O4Rx4Qu0aYM7DqO7P/TUCMl99Z zZrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=h6KpvG8k; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j34sor2937948qte.42.2019.06.07.09.05.58 for (Google Transport Security); Fri, 07 Jun 2019 09:05:58 -0700 (PDT) Received-SPF: pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@ziepe.ca header.s=google header.b=h6KpvG8k; spf=pass (google.com: domain of jgg@ziepe.ca designates 209.85.220.65 as permitted sender) smtp.mailfrom=jgg@ziepe.ca DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=mDQI7vDRgVEqfEUUaCyaBBK+4bOYH5UfB7o3hA1mJXA=; b=h6KpvG8kEPwZ4LEZggj55gXJyseLol+sT11VPWvdlbvHuRB/SDV9CdXfnBVmjGGNUR rOaB3khRLP9k43hFFlM7JgJrB+xap09v5WEvcNpCcDm+R7q6tjjOEBwesAVq48cJ/83g So0nJITGzjxRk4D0uPQp5n7PvmlL4uFKrl/48pOQIl6ZN2eKDvmwWgXQZlPWvyAnWlIw yU5ZwpruBe/84KRjpLYqcD9PBjC1jDUG3G9Ohbk+ythU2LvunFU4UkOUnBzmx4hF6Ao0 Lnzva0XY+uFw0Y+ynwlkcWlHyp6MBQc24OEdhN/d2qSSoB3U9Jw6KgBVG8SRZERqZzkb 2hlw== X-Google-Smtp-Source: APXvYqwIT1CmduZzPGFrLNflexTC0zIShRnLBlA8OsJP0/v7GZFL6ZhIHAbjBC1Kw7kXZZYVQlIuiQ== X-Received: by 2002:ac8:1af4:: with SMTP id h49mr38593897qtk.183.1559923558110; Fri, 07 Jun 2019 09:05:58 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-156-34-55-100.dhcp-dynamic.fibreop.ns.bellaliant.net. [156.34.55.100]) by smtp.gmail.com with ESMTPSA id n124sm1260323qkf.31.2019.06.07.09.05.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 07 Jun 2019 09:05:57 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1hZHNN-00008j-5Z; Fri, 07 Jun 2019 13:05:57 -0300 Date: Fri, 7 Jun 2019 13:05:57 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH v2 12/11] mm/hmm: Fix error flows in hmm_invalidate_range_start Message-ID: <20190607160557.GA335@ziepe.ca> References: <20190606184438.31646-1-jgg@ziepe.ca> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190606184438.31646-1-jgg@ziepe.ca> User-Agent: Mutt/1.9.4 (2018-02-28) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP If the trylock on the hmm->mirrors_sem fails the function will return without decrementing the notifiers that were previously incremented. Since the caller will not call invalidate_range_end() on EAGAIN this will result in notifiers becoming permanently incremented and deadlock. If the sync_cpu_device_pagetables() required blocking the function will not return EAGAIN even though the device continues to touch the pages. This is a violation of the mmu notifier contract. Switch, and rename, the ranges_lock to a spin lock so we can reliably obtain it without blocking during error unwind. The error unwind is necessary since the notifiers count must be held incremented across the call to sync_cpu_device_pagetables() as we cannot allow the range to become marked valid by a parallel invalidate_start/end() pair while doing sync_cpu_device_pagetables(). Signed-off-by: Jason Gunthorpe Reviewed-by: Ralph Campbell --- include/linux/hmm.h | 2 +- mm/hmm.c | 77 +++++++++++++++++++++++++++------------------ 2 files changed, 48 insertions(+), 31 deletions(-) I almost lost this patch - it is part of the series, hasn't been posted before, and wasn't sent with the rest, sorry. diff --git a/include/linux/hmm.h b/include/linux/hmm.h index bf013e96525771..0fa8ea34ccef6d 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -86,7 +86,7 @@ struct hmm { struct mm_struct *mm; struct kref kref; - struct mutex lock; + spinlock_t ranges_lock; struct list_head ranges; struct list_head mirrors; struct mmu_notifier mmu_notifier; diff --git a/mm/hmm.c b/mm/hmm.c index 4215edf737ef5b..10103a24e9b7b3 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -68,7 +68,7 @@ static struct hmm *hmm_get_or_create(struct mm_struct *mm) init_rwsem(&hmm->mirrors_sem); hmm->mmu_notifier.ops = NULL; INIT_LIST_HEAD(&hmm->ranges); - mutex_init(&hmm->lock); + spin_lock_init(&hmm->ranges_lock); kref_init(&hmm->kref); hmm->notifiers = 0; hmm->mm = mm; @@ -114,18 +114,19 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); struct hmm_mirror *mirror; + unsigned long flags; /* Bail out if hmm is in the process of being freed */ if (!kref_get_unless_zero(&hmm->kref)) return; - mutex_lock(&hmm->lock); + spin_lock_irqsave(&hmm->ranges_lock, flags); /* * Since hmm_range_register() holds the mmget() lock hmm_release() is * prevented as long as a range exists. */ WARN_ON(!list_empty(&hmm->ranges)); - mutex_unlock(&hmm->lock); + spin_unlock_irqrestore(&hmm->ranges_lock, flags); down_read(&hmm->mirrors_sem); list_for_each_entry(mirror, &hmm->mirrors, list) { @@ -141,6 +142,23 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) hmm_put(hmm); } +static void notifiers_decrement(struct hmm *hmm) +{ + lockdep_assert_held(&hmm->ranges_lock); + + hmm->notifiers--; + if (!hmm->notifiers) { + struct hmm_range *range; + + list_for_each_entry(range, &hmm->ranges, list) { + if (range->valid) + continue; + range->valid = true; + } + wake_up_all(&hmm->wq); + } +} + static int hmm_invalidate_range_start(struct mmu_notifier *mn, const struct mmu_notifier_range *nrange) { @@ -148,6 +166,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, struct hmm_mirror *mirror; struct hmm_update update; struct hmm_range *range; + unsigned long flags; int ret = 0; if (!kref_get_unless_zero(&hmm->kref)) @@ -158,12 +177,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, update.event = HMM_UPDATE_INVALIDATE; update.blockable = mmu_notifier_range_blockable(nrange); - if (mmu_notifier_range_blockable(nrange)) - mutex_lock(&hmm->lock); - else if (!mutex_trylock(&hmm->lock)) { - ret = -EAGAIN; - goto out; - } + spin_lock_irqsave(&hmm->ranges_lock, flags); hmm->notifiers++; list_for_each_entry(range, &hmm->ranges, list) { if (update.end < range->start || update.start >= range->end) @@ -171,7 +185,7 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, range->valid = false; } - mutex_unlock(&hmm->lock); + spin_unlock_irqrestore(&hmm->ranges_lock, flags); if (mmu_notifier_range_blockable(nrange)) down_read(&hmm->mirrors_sem); @@ -179,16 +193,26 @@ static int hmm_invalidate_range_start(struct mmu_notifier *mn, ret = -EAGAIN; goto out; } + list_for_each_entry(mirror, &hmm->mirrors, list) { - int ret; + int rc; - ret = mirror->ops->sync_cpu_device_pagetables(mirror, &update); - if (!update.blockable && ret == -EAGAIN) + rc = mirror->ops->sync_cpu_device_pagetables(mirror, &update); + if (rc) { + if (WARN_ON(update.blockable || rc != -EAGAIN)) + continue; + ret = -EAGAIN; break; + } } up_read(&hmm->mirrors_sem); out: + if (ret) { + spin_lock_irqsave(&hmm->ranges_lock, flags); + notifiers_decrement(hmm); + spin_unlock_irqrestore(&hmm->ranges_lock, flags); + } hmm_put(hmm); return ret; } @@ -197,23 +221,14 @@ static void hmm_invalidate_range_end(struct mmu_notifier *mn, const struct mmu_notifier_range *nrange) { struct hmm *hmm = container_of(mn, struct hmm, mmu_notifier); + unsigned long flags; if (!kref_get_unless_zero(&hmm->kref)) return; - mutex_lock(&hmm->lock); - hmm->notifiers--; - if (!hmm->notifiers) { - struct hmm_range *range; - - list_for_each_entry(range, &hmm->ranges, list) { - if (range->valid) - continue; - range->valid = true; - } - wake_up_all(&hmm->wq); - } - mutex_unlock(&hmm->lock); + spin_lock_irqsave(&hmm->ranges_lock, flags); + notifiers_decrement(hmm); + spin_unlock_irqrestore(&hmm->ranges_lock, flags); hmm_put(hmm); } @@ -866,6 +881,7 @@ int hmm_range_register(struct hmm_range *range, { unsigned long mask = ((1UL << page_shift) - 1UL); struct hmm *hmm = mirror->hmm; + unsigned long flags; range->valid = false; range->hmm = NULL; @@ -887,7 +903,7 @@ int hmm_range_register(struct hmm_range *range, kref_get(&hmm->kref); /* Initialize range to track CPU page table updates. */ - mutex_lock(&hmm->lock); + spin_lock_irqsave(&hmm->ranges_lock, flags); range->hmm = hmm; list_add(&range->list, &hmm->ranges); @@ -898,7 +914,7 @@ int hmm_range_register(struct hmm_range *range, */ if (!hmm->notifiers) range->valid = true; - mutex_unlock(&hmm->lock); + spin_unlock_irqrestore(&hmm->ranges_lock, flags); return 0; } @@ -914,13 +930,14 @@ EXPORT_SYMBOL(hmm_range_register); void hmm_range_unregister(struct hmm_range *range) { struct hmm *hmm = range->hmm; + unsigned long flags; if (WARN_ON(range->end <= range->start)) return; - mutex_lock(&hmm->lock); + spin_lock_irqsave(&hmm->ranges_lock, flags); list_del(&range->list); - mutex_unlock(&hmm->lock); + spin_unlock_irqrestore(&hmm->ranges_lock, flags); /* Drop reference taken by hmm_range_register() */ range->valid = false;