From patchwork Tue Jan 29 17:47:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10786671 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 986D06C2 for ; Tue, 29 Jan 2019 17:47:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 826A32D0C6 for ; Tue, 29 Jan 2019 17:47:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 75D412D0C9; Tue, 29 Jan 2019 17:47:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7B522D0C6 for ; Tue, 29 Jan 2019 17:47:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4047C8E0003; Tue, 29 Jan 2019 12:47:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 38EA28E0001; Tue, 29 Jan 2019 12:47:42 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 254A88E0003; Tue, 29 Jan 2019 12:47:42 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id EE0018E0001 for ; Tue, 29 Jan 2019 12:47:41 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id t18so25706427qtj.3 for ; Tue, 29 Jan 2019 09:47:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9RSF3wYM68LRkU5/mjhuSa1XXoUTuAX+TG7X8cIVENc=; b=jP7+hrY4DMrtfcSv7/aa5rK1MQJehFoPv3Mm0Tzqz2OOMqdTr2gT6i2+upCt2txVGF dWn9JRnupnI453hLT1HtWvT5UVwWmdxiF8Lc16PzYT+eIeQk/oFnAExERHZgz6a2yKB/ xHHQSU/gYb1i7R2G63VdQ89bqQwrCcPD+ze/RNLiWbbm4XKCcc6d7bZv3z5xnRMcU/hU BpOVmH9aF+C27BSrP639CncmERhg2HxNgCpwKRt17JCPVnWrAB2hz5O1ViaFVfDxKpc2 z/4e8399UJYZX+1rATqBoN6zDNIiLxlz74Hb/3zwJEZW93d9hmo1+3pEGNKWhvsCychb hCqw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukcfnaDoeTPlLzwBg1e1M/gjvpC8mV4ty9shqEJQtVMRbJehLzDi Op6Cf+Kms7u8yFz75oYEaVIKr/qBli9wCPUzXIlNR0xW7jAdsAxecLy7FvtleokLEB8KMXpW/bl p6SrUbXHVQad+QQM5Imu6Wtkrx/6uvm5B2lkcI3sjYfoMD2NXe+hSSK0XAox1A53UPw== X-Received: by 2002:a37:72c3:: with SMTP id n186mr23710460qkc.340.1548784061748; Tue, 29 Jan 2019 09:47:41 -0800 (PST) X-Google-Smtp-Source: ALg8bN6r0BG1n5G8lq9hLfa0aMzaXyJljHHhNU3gclOLazjI5xWSyDffVDHsagA6+zQKL+z4aewd X-Received: by 2002:a37:72c3:: with SMTP id n186mr23710426qkc.340.1548784061223; Tue, 29 Jan 2019 09:47:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548784061; cv=none; d=google.com; s=arc-20160816; b=Qi3p903kYoG2MN1QkhVat9joEgmcRJV2ow3CM3DOkIz3P0O5NOYYDDacgrebqjieFS 69qGee7GVC5TU4OPvVun+uNJGXMCTsUqFoBysO4DV2gO7Up749L6boH/0R3BPFnlNCo9 VjpD4G6UquL+DKi/QFKpwKX4TQz7etgZljbQt/dSKO5QGn62tnxLMc4+fRrI19jAEjp9 L5ZuIxU1/mDLJJiOcmmT9+59lF97cv7bI7Bx8XVwLQxoXS5A8N4nMcWrWEVQIeSLzPOk Q8WWTzWooQ82OarFkk/DKhXs5BxqouYLKm7EY8xEIVJze1ODVf+tarSQS02gvad1QHoS 2sVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=9RSF3wYM68LRkU5/mjhuSa1XXoUTuAX+TG7X8cIVENc=; b=cvRLSfmjN9TBcHdU0sA7Q6Oca/kxF1XTbYUOaBWsjr2UjzSadqbx4V4jeW296JOzWz 4WZrUlEMCiPBJY18LuYVrdFcqpW+tu3Vzuhk4eV80nQFFn+D6wIeKZiE8Zzyavm2WTTd 2mIZopkgXf8CTs3AWVfZ5yQIsPhOZjwaPI/zVf3GMpBcn3RQBjc99uJJkE8byTG6/83o STJgLcJavNA9SeLI3pKqaiWBDn7n3YSkeaUiFdEjCkEgU+lepokfbQR1PDOuyvY3QSch 7Oluj5LhlJZbOb4Ot+NO7iwVqkhpcD38+oYxPloZavO3zuZiudzjRgsDYarj2HzQp3Yp Rdig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a2si3077316qtg.254.2019.01.29.09.47.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 09:47:41 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 105AFA7885; Tue, 29 Jan 2019 17:47:40 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-2.rdu2.redhat.com [10.10.122.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id CC26B5D985; Tue, 29 Jan 2019 17:47:37 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Logan Gunthorpe , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , Jason Gunthorpe , linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , iommu@lists.linux-foundation.org Subject: [RFC PATCH 1/5] pci/p2p: add a function to test peer to peer capability Date: Tue, 29 Jan 2019 12:47:24 -0500 Message-Id: <20190129174728.6430-2-jglisse@redhat.com> In-Reply-To: <20190129174728.6430-1-jglisse@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 29 Jan 2019 17:47:40 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse device_test_p2p() return true if two devices can peer to peer to each other. We add a generic function as different inter-connect can support peer to peer and we want to genericaly test this no matter what the inter-connect might be. However this version only support PCIE for now. Signed-off-by: Jérôme Glisse Cc: Logan Gunthorpe Cc: Greg Kroah-Hartman Cc: Rafael J. Wysocki Cc: Bjorn Helgaas Cc: Christian Koenig Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: linux-kernel@vger.kernel.org Cc: linux-pci@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Joerg Roedel Cc: iommu@lists.linux-foundation.org --- drivers/pci/p2pdma.c | 27 +++++++++++++++++++++++++++ include/linux/pci-p2pdma.h | 6 ++++++ 2 files changed, 33 insertions(+) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index c52298d76e64..620ac60babb5 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -797,3 +797,30 @@ ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev, return sprintf(page, "%s\n", pci_name(p2p_dev)); } EXPORT_SYMBOL_GPL(pci_p2pdma_enable_show); + +bool pci_test_p2p(struct device *devA, struct device *devB) +{ + struct pci_dev *pciA, *pciB; + bool ret; + int tmp; + + /* + * For now we only support PCIE peer to peer but other inter-connect + * can be added. + */ + pciA = find_parent_pci_dev(devA); + pciB = find_parent_pci_dev(devB); + if (pciA == NULL || pciB == NULL) { + ret = false; + goto out; + } + + tmp = upstream_bridge_distance(pciA, pciB, NULL); + ret = tmp < 0 ? false : true; + +out: + pci_dev_put(pciB); + pci_dev_put(pciA); + return false; +} +EXPORT_SYMBOL_GPL(pci_test_p2p); diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index bca9bc3e5be7..7671cc499a08 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -36,6 +36,7 @@ int pci_p2pdma_enable_store(const char *page, struct pci_dev **p2p_dev, bool *use_p2pdma); ssize_t pci_p2pdma_enable_show(char *page, struct pci_dev *p2p_dev, bool use_p2pdma); +bool pci_test_p2p(struct device *devA, struct device *devB); #else /* CONFIG_PCI_P2PDMA */ static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, u64 offset) @@ -97,6 +98,11 @@ static inline ssize_t pci_p2pdma_enable_show(char *page, { return sprintf(page, "none\n"); } + +static inline bool pci_test_p2p(struct device *devA, struct device *devB) +{ + return false; +} #endif /* CONFIG_PCI_P2PDMA */ From patchwork Tue Jan 29 17:47:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10786675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64AD0184E for ; Tue, 29 Jan 2019 17:47:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4F1C72D0C6 for ; Tue, 29 Jan 2019 17:47:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 43A6F2D0CB; Tue, 29 Jan 2019 17:47:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF5422D0C6 for ; Tue, 29 Jan 2019 17:47:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 876EE8E0004; Tue, 29 Jan 2019 12:47:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 81FDE8E0001; Tue, 29 Jan 2019 12:47:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67CB48E0004; Tue, 29 Jan 2019 12:47:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id 3A6568E0001 for ; Tue, 29 Jan 2019 12:47:44 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id w28so22296658qkj.22 for ; Tue, 29 Jan 2019 09:47:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PoGzAvv3e6DxlL080Riy/GDZsRCScs9tPJNjMYSpaMs=; b=b7bnmDCsYPhWQmg3oQWkx6uBX2DBAvFO95v9vLdJhFvLT3lIDJRULKrNC9Q6k0Up+G HsPK+xFRYr4wteNsvnGh+NApA+vV1J1BPhZvycKNvO2FaI5Ia4CmIS6+utd/cAFKa72/ Eii9IRqQaL/5guEz6KbzKaV+4X1OCGCWTsiPl2PzOGc7Bj73tGdDVWHagMLjb4h5DrYd RWQ8xZjWr1pfC0wlLaEDCyzLOFWOi+VKQRIJwbzSmERpLLA2wBSvl4KaRDFFaBGQkiaP x5JbLFr0Ku6CZXgEEbFidwjLelTtn9mOrebVj9RfQX55xC6lRENqggri0pk1QfzI8NM1 AalQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukc8WpKRxH8mvHQmEDBPtYIA79UU0PpEcWo1ZlSCzxpaOhDb85X+ MniG8eDiDaR0epE+TbPUeo6ecfXXwCEwCfag5w2XVbQxHdQOZlORnTs0gHYhpx7Hlk9Re2ZYbnJ 2SqJEYb5eYsJ1m2HQ8as3j9SJ5i8weFFtXmIY5j1YVnX5Zn+XQyHqt4dHUGJ543+ynw== X-Received: by 2002:ac8:2276:: with SMTP id p51mr27427103qtp.200.1548784064020; Tue, 29 Jan 2019 09:47:44 -0800 (PST) X-Google-Smtp-Source: ALg8bN7uO6e8h9CB2LQ4viawjNAUDphxNAChg68RYoTcQCdSK1cBI1l7YpL/Q3Fo22O+9EOmDN3u X-Received: by 2002:ac8:2276:: with SMTP id p51mr27427059qtp.200.1548784063419; Tue, 29 Jan 2019 09:47:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548784063; cv=none; d=google.com; s=arc-20160816; b=x3fA14HifSwreX7hLMEh93hob42joui+CPOoVBg6acc3fC24cKmRDSQxcwD53NsYJr nLYVOzL0ja5Wqt28YmcrWJrlYiTfPbwAXhFJjFhOMXw5z8fPET6uyHGP24hFkdz0gMpk FoGuCMhwBZHwbBSpGvT87HwLyJo7ihBtDht4WG4I5gGINOUw9W5lAmUI4ktDsv5jKMxY gR64kWzl1UizBjti/N3+CQYg3GO1ffUC4lmmn7T8n7ngSvTxTQYyxChSstYAcauucFcf uSFIAwSNdU3K1xobh1Fx3sTvyWvA0YznMvQ1FhhO40VvjnivlZuvp4tKn4N6mQNGFKpg XupQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=PoGzAvv3e6DxlL080Riy/GDZsRCScs9tPJNjMYSpaMs=; b=im7GBsGuoCa/kzIIuoU3JBg5uioFLBmwPMTptid9n2laxcCO57CHOEcPtR7ytCH/6Q 7YPIDzpXYaMGIB2sPldr3CBUPgZxZlPjAbVSxVZk7m/yqQvSru0N/Ven8ReZQddwRtyW 7/eBRH29yZBg6niI+nxLFoDK5wG5a3Ko624F+plbzhT+dyitid+GVdqSggevgFsnypLK PaBvPqP1NIj6iSSse/IWubd0e4NaZBDF0d0gTNa+0HkzAUEqCVPC6jjkaw/7HVHZK9li L4POnqUp6hZY61xa1S++HuRGt5pt1vvvl8FFhF7SjT/F9ZKi3blmnVRZBVSS2ugXhLFS OC6A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id x74si8227588qka.175.2019.01.29.09.47.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 09:47:43 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4BDFEC0753CE; Tue, 29 Jan 2019 17:47:42 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-2.rdu2.redhat.com [10.10.122.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 38BF05D98E; Tue, 29 Jan 2019 17:47:40 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Logan Gunthorpe , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , Jason Gunthorpe , linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , iommu@lists.linux-foundation.org Subject: [RFC PATCH 2/5] drivers/base: add a function to test peer to peer capability Date: Tue, 29 Jan 2019 12:47:25 -0500 Message-Id: <20190129174728.6430-3-jglisse@redhat.com> In-Reply-To: <20190129174728.6430-1-jglisse@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 29 Jan 2019 17:47:42 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse device_test_p2p() return true if two devices can peer to peer to each other. We add a generic function as different inter-connect can support peer to peer and we want to genericaly test this no matter what the inter-connect might be. However this version only support PCIE for now. Signed-off-by: Jérôme Glisse Cc: Logan Gunthorpe Cc: Greg Kroah-Hartman Cc: Rafael J. Wysocki Cc: Bjorn Helgaas Cc: Christian Koenig Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: linux-kernel@vger.kernel.org Cc: linux-pci@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Joerg Roedel Cc: iommu@lists.linux-foundation.org --- drivers/base/core.c | 20 ++++++++++++++++++++ include/linux/device.h | 1 + 2 files changed, 21 insertions(+) diff --git a/drivers/base/core.c b/drivers/base/core.c index 0073b09bb99f..56023b00e108 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "base.h" #include "power/power.h" @@ -3167,3 +3168,22 @@ void device_set_of_node_from_dev(struct device *dev, const struct device *dev2) dev->of_node_reused = true; } EXPORT_SYMBOL_GPL(device_set_of_node_from_dev); + +/** + * device_test_p2p - test if two device can peer to peer to each other + * @devA: device A + * @devB: device B + * Returns: true if device can peer to peer to each other, false otherwise + */ +bool device_test_p2p(struct device *devA, struct device *devB) +{ + /* + * For now we only support PCIE peer to peer but other inter-connect + * can be added. + */ + if (pci_test_p2p(devA, devB)) + return true; + + return false; +} +EXPORT_SYMBOL_GPL(device_test_p2p); diff --git a/include/linux/device.h b/include/linux/device.h index 6cb4640b6160..0d532d7f0779 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -1250,6 +1250,7 @@ extern int device_online(struct device *dev); extern void set_primary_fwnode(struct device *dev, struct fwnode_handle *fwnode); extern void set_secondary_fwnode(struct device *dev, struct fwnode_handle *fwnode); void device_set_of_node_from_dev(struct device *dev, const struct device *dev2); +bool device_test_p2p(struct device *devA, struct device *devB); static inline int dev_num_vf(struct device *dev) { From patchwork Tue Jan 29 17:47:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10786681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2B2D1390 for ; Tue, 29 Jan 2019 17:47:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD12C2D095 for ; Tue, 29 Jan 2019 17:47:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A08542D0C7; Tue, 29 Jan 2019 17:47:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 771AA2D095 for ; Tue, 29 Jan 2019 17:47:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8CCC8E0005; Tue, 29 Jan 2019 12:47:47 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E3D998E0001; Tue, 29 Jan 2019 12:47:47 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDFF08E0005; Tue, 29 Jan 2019 12:47:47 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id 9D6108E0001 for ; Tue, 29 Jan 2019 12:47:47 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id y27so22485608qkj.21 for ; Tue, 29 Jan 2019 09:47:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AnAd/qqNQuLsEqhueFCN4TriXeuQ14Ot5LqO5HoQXJc=; b=pC4IIsQW0pm/jqoANUmoay98F2lJg9wgIsWQeIFC6hcCH/NMv2M1rAZ3lw20ZPggm+ L8E1IBzFXhGvL2zPqxGWo2riptxrttG39uM4sEHFWIlmQzblE1G8aQlKK035G0D8LoHD LaP2TgA403tK4+mu3DdXUj3rz/8qmP2Vkf54R+GHo8ivmO/5z95Ccv5NuFl6iY6ku2qp vyW9o6kXssPtyhBFWDX4bTrEd6ntPg/ZwV7E78V6DXnwOSSgNXrDiKh561oavDjrOhB1 tj09oPMV4cdPdJmwYRE2XNqkvJAaZG9KiEscu6jW4B7nsPzyes0WWYa9PLYoW2c1IUcs U0Ag== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukeSzQuHo/FgDVc1kZDVXZGif7yS1XxuJNvNjS7ZwUHO2u7NY5o0 zyUqp/b1B/ExLAP4S3BoTVl/iXwSnwmKC7srX7OdAnKq/XY6DYjSDzBINwd7LVh5QqAOgkyKQjy vXjHSCRWroCZWMMjoELG5H9ZAb53As43kTlnck9EkXQ4IYqDNdFu9YoXhcGWa68L6sg== X-Received: by 2002:a37:8c04:: with SMTP id o4mr23493402qkd.165.1548784067397; Tue, 29 Jan 2019 09:47:47 -0800 (PST) X-Google-Smtp-Source: ALg8bN7cFVc2Akl6Zsm++Df9z3XB5ECvoJCqkqa5OMNgs97b9UGJb6Itg+Sw4sd+yxcSWTSNYMPZ X-Received: by 2002:a37:8c04:: with SMTP id o4mr23493380qkd.165.1548784066827; Tue, 29 Jan 2019 09:47:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548784066; cv=none; d=google.com; s=arc-20160816; b=smkUJCxWja5wHvZODaEa26G9uxVIKceqH7DenK0P215R3uKFbUUSughBJ/VjhEuHGr HwjBFOj2q+LUqbwd/OApita0ut31lx2L4d37B832YLIurR5hkifPuVH0ntbe6Saj1fYw n8H5XeRz1Om7YCReXOs1fI9vRpwTB6G5OXN/rd8AAdtMY47m/8ckbv3q8bnyOaTSkHjw Gn7qfEGgyCSpWV72PkZI3fhGLtw87TqqL0OGQD5qjvwoLr7y/PrqOodJ8XScL83yYM5o +jVJic9Z11mIKWlNYVZzx4wd63RKqD1kbqDbTrvtrdHF9XoHp6LCFP4r46h7nCkR5nb8 U26A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=AnAd/qqNQuLsEqhueFCN4TriXeuQ14Ot5LqO5HoQXJc=; b=C+Y6I6mA7PcSfYKjo3a4lhGATt0km5HXqH1sZJi3L09FEo4a4X8GBCbr4fD6cXH5n6 Q3T30JsyQhl03eEdyZcAjmoh5d/WTScGk7ly6mVC3W2Mum4GNAmyGbfsVgTO17ST5NXY 5ChImvelFnSQqG8Wajm16Mqu9WQo4Imv4dRKnOyJ+26nIgFSEwQ9b5AgwurXxnGXTFah +nZopvs09nc5xZaYTTXrVxzunqQJuuvwHhc/6ECojDBcaAwqi2LCORzjBlDJN3xVPDE8 RAKsFvU4ZD3fduOZW2V+hfFuhWC5qHrTx41+V90DP1QEMH/7tbqPnoijtAoBBoV18Qft LC4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id v5si5803993qka.83.2019.01.29.09.47.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 09:47:46 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 92F9FC073D6F; Tue, 29 Jan 2019 17:47:45 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-2.rdu2.redhat.com [10.10.122.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id 93F8318A75; Tue, 29 Jan 2019 17:47:42 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Logan Gunthorpe , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , Jason Gunthorpe , linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , iommu@lists.linux-foundation.org Subject: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma Date: Tue, 29 Jan 2019 12:47:26 -0500 Message-Id: <20190129174728.6430-4-jglisse@redhat.com> In-Reply-To: <20190129174728.6430-1-jglisse@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 29 Jan 2019 17:47:46 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Allow mmap of device file to export device memory to peer to peer devices. This will allow for instance a network device to access a GPU memory or to access a storage device queue directly. The common case will be a vma created by userspace device driver that is then share to another userspace device driver which call in its kernel device driver to map that vma. The vma does not need to have any valid CPU mapping so that only peer to peer device might access its content. Or it could have valid CPU mapping too in that case it should point to same memory for consistency. Note that peer to peer mapping is highly platform and device dependent and it might not work in all the cases. However we do expect supports for this to grow on more hardware platform. This patch only adds new call backs to vm_operations_struct bulk of code light within common bus driver (like pci) and device driver (both the exporting and importing device). Current design mandate that the importer must obey mmu_notifier and invalidate any peer to peer mapping anytime a notification of invalidation happens for a range that have been peer to peer mapped. This allows exporter device to easily invalidate mapping for any importer device. Signed-off-by: Jérôme Glisse Cc: Logan Gunthorpe Cc: Greg Kroah-Hartman Cc: Rafael J. Wysocki Cc: Bjorn Helgaas Cc: Christian Koenig Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: linux-kernel@vger.kernel.org Cc: linux-pci@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Joerg Roedel Cc: iommu@lists.linux-foundation.org --- include/linux/mm.h | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 80bb6408fe73..1bd60a90e575 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -429,6 +429,44 @@ struct vm_operations_struct { pgoff_t start_pgoff, pgoff_t end_pgoff); unsigned long (*pagesize)(struct vm_area_struct * area); + /* + * Optional for device driver that want to allow peer to peer (p2p) + * mapping of their vma (which can be back by some device memory) to + * another device. + * + * Note that the exporting device driver might not have map anything + * inside the vma for the CPU but might still want to allow a peer + * device to access the range of memory corresponding to a range in + * that vma. + * + * FOR PREDICTABILITY IF DRIVER SUCCESSFULY MAP A RANGE ONCE FOR A + * DEVICE THEN FURTHER MAPPING OF THE SAME IF THE VMA IS STILL VALID + * SHOULD ALSO BE SUCCESSFUL. Following this rule allow the importing + * device to map once during setup and report any failure at that time + * to the userspace. Further mapping of the same range might happen + * after mmu notifier invalidation over the range. The exporting device + * can use this to move things around (defrag BAR space for instance) + * or do other similar task. + * + * IMPORTER MUST OBEY mmu_notifier NOTIFICATION AND CALL p2p_unmap() + * WHEN A NOTIFIER IS CALL FOR THE RANGE ! THIS CAN HAPPEN AT ANY + * POINT IN TIME WITH NO LOCK HELD. + * + * In below function, the device argument is the importing device, + * the exporting device is the device to which the vma belongs. + */ + long (*p2p_map)(struct vm_area_struct *vma, + struct device *device, + unsigned long start, + unsigned long end, + dma_addr_t *pa, + bool write); + long (*p2p_unmap)(struct vm_area_struct *vma, + struct device *device, + unsigned long start, + unsigned long end, + dma_addr_t *pa); + /* notification that a previously read-only page is about to become * writable, if an error is returned it will cause a SIGBUS */ vm_fault_t (*page_mkwrite)(struct vm_fault *vmf); From patchwork Tue Jan 29 17:47:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10786693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3D291390 for ; Tue, 29 Jan 2019 17:47:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C01B2D095 for ; Tue, 29 Jan 2019 17:47:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8FC2D2D0C9; Tue, 29 Jan 2019 17:47:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA9032D0C6 for ; Tue, 29 Jan 2019 17:47:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 897DE8E0007; Tue, 29 Jan 2019 12:47:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 846428E0001; Tue, 29 Jan 2019 12:47:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 712DC8E0007; Tue, 29 Jan 2019 12:47:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by kanga.kvack.org (Postfix) with ESMTP id 43A898E0001 for ; Tue, 29 Jan 2019 12:47:55 -0500 (EST) Received: by mail-qt1-f198.google.com with SMTP id n45so25666155qta.5 for ; Tue, 29 Jan 2019 09:47:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=YRbAZ+lp+j59t1y/XLZmXGP23A29YFgjirRM+nR5vIY=; b=lx+i5esDbB0SwHsaRZK2b8cpGHmiZYEAA/DGR2Mct/oNla+oZ/2rTOXEQIFgGI6mHt 5IcQ0UhWKVo1bZ5B0wxrlkoGmM2RfWKb4/wCPdQvEJNvVnyJwk2YGR+LSpF020BskhNV q+Wedvza0XDh7qUdJcOGJoOp5aq5mtSl758bKLPs9L3q7P4mj8GHivj6zFnqpD8HvZcJ x4H6vWDL1iEgtULmU6Yr7c8Ig9fmCrcMGwsO1y9tpPJ7WpY7/w8U15OWRBhFetIhNCgU X2dISLiNWQUDjJit4pQ8y9RmAG6Cp8au4nr2hHI+20PG6A/iDD2y7bmmzBvIYOiMpe1T gGPA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukfDF+UJpACjcsmtDk/5Db6LWY4ra0MV6Skvkz80iI6cUeZ9mGXj JH/wjZ+O031IyzbkqletDR/MvsUbiaxEVrPISLbWE5gdy2Mo8rF5o1FtJqNin/knpdBMFwAEjQ2 4vK27wG+ichVCnNSnrfhzLHzYopA6o0F5/EFm8wAk0n2Awxv3At8RoOJxOdGUeIx2/Q== X-Received: by 2002:a37:ac05:: with SMTP id e5mr25044863qkm.102.1548784075038; Tue, 29 Jan 2019 09:47:55 -0800 (PST) X-Google-Smtp-Source: ALg8bN687GOkaVOFM5ky3mQj/xiW7WzaLlx58xLQIGlURP9dHYxytcRoyHjzhMLmXO0z7REIS+oZ X-Received: by 2002:a37:ac05:: with SMTP id e5mr25044834qkm.102.1548784074441; Tue, 29 Jan 2019 09:47:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548784074; cv=none; d=google.com; s=arc-20160816; b=FI+q8wxjib7RA2AHUNIwVXvnTTlTocqFpQlsz5fRMKDLgmcQYQqmcJN3UPdj8SXdlw 0T3A8WR3Mv4hS9YitA9JI7EwaFdqeNgaqiumjJURakv7h+TOVgXuMZb0p1H1TXpNQEvs sbbDb/uzkks0qn2ArrMU6AA3eAAIBCheyg3qmCAvw8mETmgqcpr9vuaV+1Y3EHObGHHV jbXGdIJ4duw2GZXm08acB6kKaaCrKaJjoSsGVEPnQk8vpFa48jll7d3LGAPeMDyqSIlR bx7nXIxrIT0buGhzzgzdfH+h6lFrbvOipoyyU8fOVGd+45fj1ic61XEuWjV2LJC0QkAx QB7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=YRbAZ+lp+j59t1y/XLZmXGP23A29YFgjirRM+nR5vIY=; b=BH06sDtbZz7Hh3XL+e5MVXfQ7Od7OMqbcHKVjnr/Pl/Q7jKB/R1zCjQWJwjRJn+8me 9jCAk2IQYXMd0UrGPJfHp5oD6338r+cpo7Rz7BT6g8ExzR1jaMD93IEk/tN1kEu9Sp+5 iZjYYG2ot+k1lN7UPwGOVQFWSpiVtkrkDxCBqY94CM86Dj8E1i/hTwMaYuD6glsQddlM l90C/1e06wifnlZERrYQDUQDBcZ3puPe3lqZ4PRQdig73m2WJ3rxD3W8aXMirxEIAIkh 5ILK/xsX44E3i0uBDUAhoiEcv9xRk3rnlKzGIznkx71LHQvSdMaWdiy89zQ8Ip6a/Aca HxKA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r29si1918305qtr.290.2019.01.29.09.47.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 09:47:54 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C7A34C079C49; Tue, 29 Jan 2019 17:47:47 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-2.rdu2.redhat.com [10.10.122.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id B23F35D97A; Tue, 29 Jan 2019 17:47:45 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Logan Gunthorpe , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , Jason Gunthorpe , linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , iommu@lists.linux-foundation.org Subject: [RFC PATCH 4/5] mm/hmm: add support for peer to peer to HMM device memory Date: Tue, 29 Jan 2019 12:47:27 -0500 Message-Id: <20190129174728.6430-5-jglisse@redhat.com> In-Reply-To: <20190129174728.6430-1-jglisse@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 29 Jan 2019 17:47:53 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Signed-off-by: Jérôme Glisse Cc: Logan Gunthorpe Cc: Greg Kroah-Hartman Cc: Rafael J. Wysocki Cc: Bjorn Helgaas Cc: Christian Koenig Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: linux-pci@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Joerg Roedel Cc: iommu@lists.linux-foundation.org --- include/linux/hmm.h | 47 +++++++++++++++++++++++++++++++++ mm/hmm.c | 63 +++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 105 insertions(+), 5 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 4a1454e3efba..7a3ac182cc48 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -710,6 +710,53 @@ struct hmm_devmem_ops { const struct page *page, unsigned int flags, pmd_t *pmdp); + + /* + * p2p_map() - map page for peer to peer between device + * @devmem: device memory structure (see struct hmm_devmem) + * @range: range of virtual address that is being mapped + * @device: device the range is being map to + * @addr: first virtual address in the range to consider + * @pa: device address (where actual mapping is store) + * Returns: number of page successfuly mapped, 0 otherwise + * + * Map page belonging to devmem to another device for peer to peer + * access. Device can decide not to map in which case memory will + * be migrated to main memory. + * + * Also there is no garantee that all the pages in the range does + * belongs to the devmem so it is up to the function to check that + * every single page does belong to devmem. + * + * Note for now we do not care about error exect error, so on failure + * function should just return 0. + */ + long (*p2p_map)(struct hmm_devmem *devmem, + struct hmm_range *range, + struct device *device, + unsigned long addr, + dma_addr_t *pas); + + /* + * p2p_unmap() - unmap page from peer to peer between device + * @devmem: device memory structure (see struct hmm_devmem) + * @range: range of virtual address that is being mapped + * @device: device the range is being map to + * @addr: first virtual address in the range to consider + * @pa: device address (where actual mapping is store) + * Returns: number of page successfuly unmapped, 0 otherwise + * + * Unmap page belonging to devmem previously map with p2p_map(). + * + * Note there is no garantee that all the pages in the range does + * belongs to the devmem so it is up to the function to check that + * every single page does belong to devmem. + */ + unsigned long (*p2p_unmap)(struct hmm_devmem *devmem, + struct hmm_range *range, + struct device *device, + unsigned long addr, + dma_addr_t *pas); }; /* diff --git a/mm/hmm.c b/mm/hmm.c index 1a444885404e..fd49b1e116d0 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -1193,16 +1193,19 @@ long hmm_range_dma_map(struct hmm_range *range, dma_addr_t *daddrs, bool block) { - unsigned long i, npages, mapped, page_size; + unsigned long i, npages, mapped, page_size, addr; long ret; +again: ret = hmm_range_fault(range, block); if (ret <= 0) return ret ? ret : -EBUSY; + mapped = 0; + addr = range->start; page_size = hmm_range_page_size(range); npages = (range->end - range->start) >> range->page_shift; - for (i = 0, mapped = 0; i < npages; ++i) { + for (i = 0; i < npages; ++i, addr += page_size) { enum dma_data_direction dir = DMA_FROM_DEVICE; struct page *page; @@ -1226,6 +1229,29 @@ long hmm_range_dma_map(struct hmm_range *range, goto unmap; } + if (is_device_private_page(page)) { + struct hmm_devmem *devmem = page->pgmap->data; + + if (!devmem->ops->p2p_map || !devmem->ops->p2p_unmap) { + /* Fall-back to main memory. */ + range->default_flags |= + range->flags[HMM_PFN_DEVICE_PRIVATE]; + goto again; + } + + ret = devmem->ops->p2p_map(devmem, range, device, + addr, daddrs); + if (ret <= 0) { + /* Fall-back to main memory. */ + range->default_flags |= + range->flags[HMM_PFN_DEVICE_PRIVATE]; + goto again; + } + mapped += ret; + i += ret; + continue; + } + /* If it is read and write than map bi-directional. */ if (range->pfns[i] & range->values[HMM_PFN_WRITE]) dir = DMA_BIDIRECTIONAL; @@ -1242,7 +1268,9 @@ long hmm_range_dma_map(struct hmm_range *range, return mapped; unmap: - for (npages = i, i = 0; (i < npages) && mapped; ++i) { + npages = i; + addr = range->start; + for (i = 0; (i < npages) && mapped; ++i, addr += page_size) { enum dma_data_direction dir = DMA_FROM_DEVICE; struct page *page; @@ -1253,6 +1281,18 @@ long hmm_range_dma_map(struct hmm_range *range, if (dma_mapping_error(device, daddrs[i])) continue; + if (is_device_private_page(page)) { + struct hmm_devmem *devmem = page->pgmap->data; + unsigned long inc; + + inc = devmem->ops->p2p_unmap(devmem, range, device, + addr, &daddrs[i]); + BUG_ON(inc > npages); + mapped += inc; + i += inc; + continue; + } + /* If it is read and write than map bi-directional. */ if (range->pfns[i] & range->values[HMM_PFN_WRITE]) dir = DMA_BIDIRECTIONAL; @@ -1285,7 +1325,7 @@ long hmm_range_dma_unmap(struct hmm_range *range, dma_addr_t *daddrs, bool dirty) { - unsigned long i, npages, page_size; + unsigned long i, npages, page_size, addr; long cpages = 0; /* Sanity check. */ @@ -1298,7 +1338,7 @@ long hmm_range_dma_unmap(struct hmm_range *range, page_size = hmm_range_page_size(range); npages = (range->end - range->start) >> range->page_shift; - for (i = 0; i < npages; ++i) { + for (i = 0, addr = range->start; i < npages; ++i, addr += page_size) { enum dma_data_direction dir = DMA_FROM_DEVICE; struct page *page; @@ -1318,6 +1358,19 @@ long hmm_range_dma_unmap(struct hmm_range *range, set_page_dirty(page); } + if (is_device_private_page(page)) { + struct hmm_devmem *devmem = page->pgmap->data; + unsigned long ret; + + BUG_ON(!devmem->ops->p2p_unmap); + + ret = devmem->ops->p2p_unmap(devmem, range, device, + addr, &daddrs[i]); + BUG_ON(ret > npages); + i += ret; + continue; + } + /* Unmap and clear pfns/dma address */ dma_unmap_page(device, daddrs[i], page_size, dir); range->pfns[i] = range->values[HMM_PFN_NONE]; From patchwork Tue Jan 29 17:47:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10786687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 74A931390 for ; Tue, 29 Jan 2019 17:47:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 599CD2D095 for ; Tue, 29 Jan 2019 17:47:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4DB4B2D0C9; Tue, 29 Jan 2019 17:47:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 665F12D0C6 for ; Tue, 29 Jan 2019 17:47:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F59E8E0006; Tue, 29 Jan 2019 12:47:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0A4B38E0001; Tue, 29 Jan 2019 12:47:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFC6F8E0006; Tue, 29 Jan 2019 12:47:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id C06568E0001 for ; Tue, 29 Jan 2019 12:47:52 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id j125so22308501qke.12 for ; Tue, 29 Jan 2019 09:47:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9TXGHpPZf9+9Zhqq1k19W7GL0aExCP0ZWj0gy6NB/yI=; b=EImHIBwHvimic0st/OZqItSlH8L0/ccCF5d+MUKA7PpoEtUkiWNFmqp27b405PqWVS KpMvJ7zHdnBYo0EOZoz4yeWCH347b2u7m5jldNyjWxdS4LB7ydMFq2SlPHrajzitkjzB qeFyy2LDZmB31i1vYXaS6TTqK5tw6TL9kmrso7Uq76xxECNoU3pM029sx6zXHHHqHIG/ IVYB+Cramu+7+pipgvUMCu/4jEdztk86oXXUlhjNrk1J7lYPfUfUDNIJLlh1sYh5O6+X OwgHuXWuOXgFaht+EdHmZi7adlWvZtYVBkT84ERhpZOldNXE2+F5vaKJQDjleqKow+sY Y87A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukdfJMJ0zr4xODX5YrNfkKEDJsv4e42OcuzjPup2uNEAnwaAEDVW ju4dUmg1RV8uCaHvO0Mr2g0nk9pS/Ktp5wycMAE5zruhAOsYWIr62J0Ni9LN4FJD8xB7koCqgjx ou/BQ1yq6VfEdPKM6uT+RWFUxjmTk1ZNzUbLc1qWaqwCu5d3Y++ar0nu6Mgs43lFo4g== X-Received: by 2002:aed:3384:: with SMTP id v4mr26119303qtd.169.1548784072490; Tue, 29 Jan 2019 09:47:52 -0800 (PST) X-Google-Smtp-Source: ALg8bN5AbUomOyyDXIFIxTZ+omKpmBp9ZWAVQ/zp/B+h3ydvjGdAzSmkwIlyqAYtw0+/0gtsZPH2 X-Received: by 2002:aed:3384:: with SMTP id v4mr26119267qtd.169.1548784071870; Tue, 29 Jan 2019 09:47:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548784071; cv=none; d=google.com; s=arc-20160816; b=q2eBoQlHX5oq1jjFZHcw5hd+UGuuyx3+mOc3N+NOEIjPnjpj2ysg0/Yd4PB2U2F0EV TEGrv3aXHkThwVMS+wrWYU+G1tABi8NVHdLNDJ4SEFZ/EsHJ5oupn3YS1gEI3AD0fb8b ZiyfTl7eAQexCSNAmj5yi3Y1xv2kQYTj7AIlfR3B3Jc1rq9Uu+sqBgPzHlRw3nhU86Rj HJYYL4QU1A9MR0k61xL66wd2nCx2Yw6RDCbU8RC1Lq8tHfkQH6o7hXSBJQD3cKccf2oL Xbu6LQ3aj9uDyalnzKQUIH6kmYRmLwhu8qwePgiJO0aVu1CXvFQX3iI3JYkqT56Wlv2u /0DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=9TXGHpPZf9+9Zhqq1k19W7GL0aExCP0ZWj0gy6NB/yI=; b=QylEygcdUw0wwhxlSRloGzHgUz0IDIgcJQkGoWAGNbavJ6dC91vC72oHE+MwKTtBHR fOVPdekhQZng3mdPRPB45MXetPykVEJ7+yjN8c39QXYaopWMoRJ2PGdactZn6ojFxH+I +braNCppUJekfr2V/mbfGxP1FKnJSq2nWm6yWNPfXAZOAxYcYjKbRPd3G3vVtTtzzYrz y28ir+phwfgw7u/R8A3rjEQxooVqfgyQxRFdmvoQfKq2mZj+rB6z4LFK1NAmr837XwJ7 zjuxC6K4630QP0e8UILRoDnXiBFrQB7uVPAlcKnhU9lFREkn/5WxKdoEHJjberXgy9PK SfYw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id n127si230856qkf.230.2019.01.29.09.47.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jan 2019 09:47:51 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5443A7AE81; Tue, 29 Jan 2019 17:47:50 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-122-2.rdu2.redhat.com [10.10.122.2]) by smtp.corp.redhat.com (Postfix) with ESMTP id D994D5D97E; Tue, 29 Jan 2019 17:47:47 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Logan Gunthorpe , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , Jason Gunthorpe , linux-pci@vger.kernel.org, dri-devel@lists.freedesktop.org, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , iommu@lists.linux-foundation.org Subject: [RFC PATCH 5/5] mm/hmm: add support for peer to peer to special device vma Date: Tue, 29 Jan 2019 12:47:28 -0500 Message-Id: <20190129174728.6430-6-jglisse@redhat.com> In-Reply-To: <20190129174728.6430-1-jglisse@redhat.com> References: <20190129174728.6430-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 29 Jan 2019 17:47:51 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse Special device vma (mmap of a device file) can correspond to device driver object that some device driver might want to share with other device (giving access to). This add support for HMM to map those special device vma if the owning device (exporter) allows it. Signed-off-by: Jérôme Glisse Cc: Logan Gunthorpe Cc: Greg Kroah-Hartman Cc: Rafael J. Wysocki Cc: Bjorn Helgaas Cc: Christian Koenig Cc: Felix Kuehling Cc: Jason Gunthorpe Cc: linux-pci@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Joerg Roedel Cc: iommu@lists.linux-foundation.org --- include/linux/hmm.h | 6 ++ mm/hmm.c | 156 ++++++++++++++++++++++++++++++++++---------- 2 files changed, 128 insertions(+), 34 deletions(-) diff --git a/include/linux/hmm.h b/include/linux/hmm.h index 7a3ac182cc48..98ebe9f52432 100644 --- a/include/linux/hmm.h +++ b/include/linux/hmm.h @@ -137,6 +137,7 @@ enum hmm_pfn_flag_e { * result of vmf_insert_pfn() or vm_insert_page(). Therefore, it should not * be mirrored by a device, because the entry will never have HMM_PFN_VALID * set and the pfn value is undefined. + * HMM_PFN_P2P: this entry have been map as P2P ie the dma address is valid * * Driver provide entry value for none entry, error entry and special entry, * driver can alias (ie use same value for error and special for instance). It @@ -151,6 +152,7 @@ enum hmm_pfn_value_e { HMM_PFN_ERROR, HMM_PFN_NONE, HMM_PFN_SPECIAL, + HMM_PFN_P2P, HMM_PFN_VALUE_MAX }; @@ -250,6 +252,8 @@ static inline bool hmm_range_valid(struct hmm_range *range) static inline struct page *hmm_pfn_to_page(const struct hmm_range *range, uint64_t pfn) { + if (pfn == range->values[HMM_PFN_P2P]) + return NULL; if (pfn == range->values[HMM_PFN_NONE]) return NULL; if (pfn == range->values[HMM_PFN_ERROR]) @@ -270,6 +274,8 @@ static inline struct page *hmm_pfn_to_page(const struct hmm_range *range, static inline unsigned long hmm_pfn_to_pfn(const struct hmm_range *range, uint64_t pfn) { + if (pfn == range->values[HMM_PFN_P2P]) + return -1UL; if (pfn == range->values[HMM_PFN_NONE]) return -1UL; if (pfn == range->values[HMM_PFN_ERROR]) diff --git a/mm/hmm.c b/mm/hmm.c index fd49b1e116d0..621a4f831483 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -1058,37 +1058,36 @@ long hmm_range_snapshot(struct hmm_range *range) } EXPORT_SYMBOL(hmm_range_snapshot); -/* - * hmm_range_fault() - try to fault some address in a virtual address range - * @range: range being faulted - * @block: allow blocking on fault (if true it sleeps and do not drop mmap_sem) - * Returns: 0 on success ortherwise: - * -EINVAL: - * Invalid argument - * -ENOMEM: - * Out of memory. - * -EPERM: - * Invalid permission (for instance asking for write and range - * is read only). - * -EAGAIN: - * If you need to retry and mmap_sem was drop. This can only - * happens if block argument is false. - * -EBUSY: - * If the the range is being invalidated and you should wait for - * invalidation to finish. - * -EFAULT: - * Invalid (ie either no valid vma or it is illegal to access that - * range), number of valid pages in range->pfns[] (from range start - * address). - * - * This is similar to a regular CPU page fault except that it will not trigger - * any memory migration if the memory being faulted is not accessible by CPUs - * and caller does not ask for migration. - * - * On error, for one virtual address in the range, the function will mark the - * corresponding HMM pfn entry with an error flag. - */ -long hmm_range_fault(struct hmm_range *range, bool block) +static int hmm_vma_p2p_map(struct hmm_range *range, struct vm_area_struct *vma, + unsigned long start, unsigned long end, + struct device *device, dma_addr_t *pas) +{ + struct hmm_vma_walk hmm_vma_walk; + unsigned long npages, i; + bool fault, write; + uint64_t *pfns; + int ret; + + i = (start - range->start) >> PAGE_SHIFT; + npages = (end - start) >> PAGE_SHIFT; + pfns = &range->pfns[i]; + pas = &pas[i]; + + hmm_vma_walk.range = range; + hmm_vma_walk.fault = true; + hmm_range_need_fault(&hmm_vma_walk, pfns, npages, + 0, &fault, &write); + + ret = vma->vm_ops->p2p_map(vma, device, start, end, pas, write); + for (i = 0; i < npages; ++i) { + pfns[i] = ret ? range->values[HMM_PFN_ERROR] : + range->values[HMM_PFN_P2P]; + } + return ret; +} + +static long _hmm_range_fault(struct hmm_range *range, bool block, + struct device *device, dma_addr_t *pas) { const unsigned long device_vma = VM_IO | VM_PFNMAP | VM_MIXEDMAP; unsigned long start = range->start, end; @@ -1110,9 +1109,22 @@ long hmm_range_fault(struct hmm_range *range, bool block) } vma = find_vma(hmm->mm, start); - if (vma == NULL || (vma->vm_flags & device_vma)) + if (vma == NULL) return -EFAULT; + end = min(range->end, vma->vm_end); + if (vma->vm_flags & device_vma) { + if (!device || !pas || !vma->vm_ops->p2p_map) + return -EFAULT; + + ret = hmm_vma_p2p_map(range, vma, start, + end, device, pas); + if (ret) + return ret; + start = end; + continue; + } + if (is_vm_hugetlb_page(vma)) { struct hstate *h = hstate_vma(vma); @@ -1142,7 +1154,6 @@ long hmm_range_fault(struct hmm_range *range, bool block) hmm_vma_walk.block = block; hmm_vma_walk.range = range; mm_walk.private = &hmm_vma_walk; - end = min(range->end, vma->vm_end); mm_walk.vma = vma; mm_walk.mm = vma->vm_mm; @@ -1175,6 +1186,41 @@ long hmm_range_fault(struct hmm_range *range, bool block) return (hmm_vma_walk.last - range->start) >> PAGE_SHIFT; } + +/* + * hmm_range_fault() - try to fault some address in a virtual address range + * @range: range being faulted + * @block: allow blocking on fault (if true it sleeps and do not drop mmap_sem) + * Returns: 0 on success ortherwise: + * -EINVAL: + * Invalid argument + * -ENOMEM: + * Out of memory. + * -EPERM: + * Invalid permission (for instance asking for write and range + * is read only). + * -EAGAIN: + * If you need to retry and mmap_sem was drop. This can only + * happens if block argument is false. + * -EBUSY: + * If the the range is being invalidated and you should wait for + * invalidation to finish. + * -EFAULT: + * Invalid (ie either no valid vma or it is illegal to access that + * range), number of valid pages in range->pfns[] (from range start + * address). + * + * This is similar to a regular CPU page fault except that it will not trigger + * any memory migration if the memory being faulted is not accessible by CPUs + * and caller does not ask for migration. + * + * On error, for one virtual address in the range, the function will mark the + * corresponding HMM pfn entry with an error flag. + */ +long hmm_range_fault(struct hmm_range *range, bool block) +{ + return _hmm_range_fault(range, block, NULL, NULL); +} EXPORT_SYMBOL(hmm_range_fault); /* @@ -1197,7 +1243,7 @@ long hmm_range_dma_map(struct hmm_range *range, long ret; again: - ret = hmm_range_fault(range, block); + ret = _hmm_range_fault(range, block, device, daddrs); if (ret <= 0) return ret ? ret : -EBUSY; @@ -1209,6 +1255,11 @@ long hmm_range_dma_map(struct hmm_range *range, enum dma_data_direction dir = DMA_FROM_DEVICE; struct page *page; + if (range->pfns[i] == range->values[HMM_PFN_P2P]) { + mapped++; + continue; + } + /* * FIXME need to update DMA API to provide invalid DMA address * value instead of a function to test dma address value. This @@ -1274,6 +1325,11 @@ long hmm_range_dma_map(struct hmm_range *range, enum dma_data_direction dir = DMA_FROM_DEVICE; struct page *page; + if (range->pfns[i] == range->values[HMM_PFN_P2P]) { + mapped--; + continue; + } + page = hmm_pfn_to_page(range, range->pfns[i]); if (page == NULL) continue; @@ -1305,6 +1361,30 @@ long hmm_range_dma_map(struct hmm_range *range, } EXPORT_SYMBOL(hmm_range_dma_map); +static unsigned long hmm_vma_p2p_unmap(struct hmm_range *range, + struct vm_area_struct *vma, + unsigned long start, + struct device *device, + dma_addr_t *pas) +{ + unsigned long end; + + if (!vma) { + BUG(); + return 1; + } + + start &= PAGE_MASK; + if (start < vma->vm_start || start >= vma->vm_end) { + BUG(); + return 1; + } + + end = min(range->end, vma->vm_end); + vma->vm_ops->p2p_unmap(vma, device, start, end, pas); + return (end - start) >> PAGE_SHIFT; +} + /* * hmm_range_dma_unmap() - unmap range of that was map with hmm_range_dma_map() * @range: range being unmapped @@ -1342,6 +1422,14 @@ long hmm_range_dma_unmap(struct hmm_range *range, enum dma_data_direction dir = DMA_FROM_DEVICE; struct page *page; + if (range->pfns[i] == range->values[HMM_PFN_P2P]) { + BUG_ON(!vma); + cpages += hmm_vma_p2p_unmap(range, vma, addr, + device, &daddrs[i]); + i += cpages - 1; + continue; + } + page = hmm_pfn_to_page(range, range->pfns[i]); if (page == NULL) continue;