From patchwork Tue Dec 12 07:37:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13488634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68476C4332F for ; Tue, 12 Dec 2023 08:01:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ECF286B0297; Tue, 12 Dec 2023 03:01:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E57A96B0299; Tue, 12 Dec 2023 03:01:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C82DE6B029A; Tue, 12 Dec 2023 03:01:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AB31E6B0297 for ; Tue, 12 Dec 2023 03:01:33 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7A8C4C094B for ; Tue, 12 Dec 2023 08:01:33 +0000 (UTC) X-FDA: 81557421666.29.FFF8519 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf17.hostedemail.com (Postfix) with ESMTP id 56B8140023 for ; Tue, 12 Dec 2023 08:01:31 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TfsS5uFq; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf17.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702368091; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OFWxI1i+p4efvmikRdB46knSq10ZwkVaViiQ0SsvQMA=; b=TqxidEFtgI3NW5d/YBUZlyfU42qkeV59ysUFrDhe/5cPQiagFxawFpbFvxRQkLxBQWb5Oc DPZuoXb8VrSWCllJmTA/s7SBIb65jeCZjUq59AX98eiBN+l+bFKekFVJH4JLfwp6fZQyPQ dijbkVNwy19RcFBkhMMEbbPK2jWAjz8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TfsS5uFq; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf17.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702368091; a=rsa-sha256; cv=none; b=pdIB2Vfrsm9ZcBf+2sTlDhDoYg7UcYCXyTZbBgwcdMm9W8Cu6ANtZMYrQcO1La1JHLtUgD nFrdWIkqaaJS3cp2nnGCgSYpQyVhAbvy58RdkFG58XG3P+Zjh7TzrMvx25vwvvMRIbGmPV XPcEE13fEclXTWH6mOy4NEvEZMFFq5g= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702368091; x=1733904091; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U5Op4CjhDDAHfKtiDzpe6+9a1b7R8CllBxVTE/TesJE=; b=TfsS5uFq7tEyN3ePyQXjeD33QWotSFNT03n4HmKzCrd64N2QhMr6LiqJ qATkhdeY2M4feMFKmK/+Sn/tn6if5p5l9DV3/ipvIQWvWaPnSISkVOsng 3s3ZD32JA41RN6oiVargvsHV4pRlRC6PIRztnSlqJBevGFv1irSu966PY UTwaAZt5Z24rZA1W7CyQe9O9JNaz56MRB6cBr+bHwQOlEZei7vhlOSjEz yMBuoFo4epf6YRtiB02AdD6mOGsTbVnPDslfjklExBsm4rVJqANCnHtY5 +zrPidcr7ol70q+PM6b4EgigtoUoM1e3CJQ33nGb1UtEg2OTuviZqeID2 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="397553943" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="397553943" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="802376320" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="802376320" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:24 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Jason Gunthorpe , Gerd Hoffmann , Dongwon Kim , Junxiao Chang Subject: [PATCH v7 1/6] udmabuf: Use vmf_insert_pfn and VM_PFNMAP for handling mmap Date: Mon, 11 Dec 2023 23:37:58 -0800 Message-Id: <20231212073803.3233055-2-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231212073803.3233055-1-vivek.kasireddy@intel.com> References: <20231212073803.3233055-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 56B8140023 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: orh7eqw3be4s9hzm8rmz4gmahm8u4ye4 X-HE-Tag: 1702368091-262880 X-HE-Meta: U2FsdGVkX18UtgbGcwVXHOjyy+491c1/aQ69QABgEgPvfRd5OssRCIsf32qwiIjn5nj/sSCTNsnvl0DRPGSqXEnLQ371pKIuDD+u0D+jWeY3Hymi6RTv4xzvY2ASmI/eTjzQrNuzjs+9r31Qnq7l2aQAUhcLIaP13AQ6OUleR9ip5ueia3pgcgz5ONRMDK0WugTO/qCAotMqyZ0I03heXKnpTlOXlGX35WCwgL5/OqlLvUqcHcipCTRbmZAAJQOj4+fEv7LslPYI1efXkRAUktKyqVgbkavQlc+3qi5USYZf1+PhEml+VLYbMrjR1hPJjhU10+gZ8GH+boLVBuTU4Rauff2upi/gO4b6TSXUIcshkvXurBSZV1HOK4qr79D886mAo5hRdZ8SS4XKzkOohY/MCt/AgLHMOEH0ZNTaEKmJcSgW/VqKWPadQ5DmR6U20WaPC3YOxtt9jp1CYvJGHMnnH96WEuA1lQI1cREDm786+bYehNbxiYd5ePp6RxdCaiqglFtAlj9U1NOJ+jgdO1sYzfcbn0A7dvfzh+zXsExIw/LiAemH39ov6H9oCyOKhaUnjFpPpEVnX42XtWzBXpGD67oUntqfEcV2fJnyBTUTw+3XRs38wS+dVZppRJdfgc/5SZF5jCIxU//TrH/aftrwyNDmwdbxVwBd5dUgHp27KCjGVvZIT5dm8QYpuXe7TiMs1kMj4qfk7l7Sa8qWPRiBJtCo3VyS5boY4Uu8YOuI7/oL4Ds8yvEHTqpmUO8DqCAhMBs8vb+UEm+RPRHPL9XtNVT8pJMafVw5AeoPk1Y2msp1wUEEqEimTOZ1jNVk2F4sSJ0FIV5R6x97wpSnrxpR2ZAfx6dJrJFwXRrqg/kQlhDLR5c/B4q6HbOXvGXYbXHuJGCbKjQF60yMjyIQPPSwDkTMgyRrBN12uyHro0FNX4D1F1WxNn3OSvAjLh22evbEJFndOPBmWp3Sm0x k8cmHfEe 9qC/73W9zVodvpghNNfbf6I2HZApHA07AFo/rXiLTZVb758j6YD8hcJTxqAmtBQ8j47ApVq82wy6DxWxHrM1M/fGRSfBVZN0YFcFRfhp7MtH1QotPBCkpmZ5PZRFnYfZEp+AtB7d/rrF3vdppYY1/uv6/frcYHEBl9tDYL556nYuJeIaogDVcT3R44I097MNAFUFe7wkTCmUIt7r84dFnXh2eHzvxpU+mNt+qe//zIuQD9nsHFqNBUe+wTINXJ/3AsKItsrrfjrXNWNHBgMJALmnjVzOQ/6wvPlUX3TT3Rv3Bf3I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add VM_PFNMAP to vm_flags in the mmap handler to ensure that the mappings would be managed without using struct page. And, in the vm_fault handler, use vmf_insert_pfn to share the page's pfn to userspace instead of directly sharing the page (via struct page *). Cc: David Hildenbrand Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Jason Gunthorpe Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Suggested-by: David Hildenbrand Acked-by: David Hildenbrand Signed-off-by: Vivek Kasireddy --- drivers/dma-buf/udmabuf.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index c40645999648..820c993c8659 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -35,12 +35,13 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) struct vm_area_struct *vma = vmf->vma; struct udmabuf *ubuf = vma->vm_private_data; pgoff_t pgoff = vmf->pgoff; + unsigned long pfn; if (pgoff >= ubuf->pagecount) return VM_FAULT_SIGBUS; - vmf->page = ubuf->pages[pgoff]; - get_page(vmf->page); - return 0; + + pfn = page_to_pfn(ubuf->pages[pgoff]); + return vmf_insert_pfn(vma, vmf->address, pfn); } static const struct vm_operations_struct udmabuf_vm_ops = { @@ -56,6 +57,7 @@ static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma) vma->vm_ops = &udmabuf_vm_ops; vma->vm_private_data = ubuf; + vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); return 0; } From patchwork Tue Dec 12 07:37:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13488633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8AE6C4167D for ; Tue, 12 Dec 2023 08:01:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 61BA66B0295; Tue, 12 Dec 2023 03:01:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CACC6B0297; Tue, 12 Dec 2023 03:01:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41D536B0298; Tue, 12 Dec 2023 03:01:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2C0006B0295 for ; Tue, 12 Dec 2023 03:01:32 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 01FE01208C8 for ; Tue, 12 Dec 2023 08:01:31 +0000 (UTC) X-FDA: 81557421624.20.E34AC32 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf08.hostedemail.com (Postfix) with ESMTP id CC839160029 for ; Tue, 12 Dec 2023 08:01:28 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=m3E5X7xO; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702368089; a=rsa-sha256; cv=none; b=QcGrXcpUEBOmEzGbh1HkddPqAl8gW21r6OSfF7CsFnUO7E8rAIIrwG9s960MXSoc9uiYt2 5wqHPytoo6s0RvB89CH1wGaoAHizNd4qyMv2miFZjrpTCRDRveNNOnRXpqWAuyN/4Y8yQg SeI4eMkMrRX6T1FFzqAlGv55pV1W6Xo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=m3E5X7xO; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702368089; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gnFFCZ/4N7bQ8jwcsO7jGvxgDXP7BPVbDrHLX60MqX8=; b=do+e9LhcHwThZceF6OS4JFCoAu8/5nmuIYGjwAdTkW6NK6QsRis6QravktsioLhF4n6o8z V8D20RqiF8jQXKVwyovNJOBs2ULihc95Z3XZ0/2bL3m8NOUlDgheNpcQmofyLH7XBCIUhe vPtxqDE+KbL1JnesNTjf4TAsNcpSvc0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702368088; x=1733904088; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O1c3JIhGATN1dxcRTvUuRK9RhQSZEkJPgfU5c33zkAo=; b=m3E5X7xOFuAojGv6yUTvgAHqjbsJNu43vGm45Xj8nCvBAkC204b04fMa EIKuZcnZmmZghvAwRQRHG8CfKdbuiNOUDZn1frQJ/LwMdxR1U40FTxHWD VXffwVZv5cWM4oZmS5EGxOZnqIhTqVFy6mwFdvwHhqMnVijQV+egKe6Y4 ytk17c+GJQJN3KfVxm7j0lJLfF2dhvyy4AKxA2C40EBgRtqOvy5MO3r+Z +SSdOx9qwmOSxwO8eM6seNPLw7QOy7f4U3veib7uXLZn0GHoKcgg/Ke0l FmhACc7NoHHcKZxmJnZjZ3cA5o8Wf3qM7d7if0ZFUoJOLLR+qRg9ZP+Pn A==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="397553950" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="397553950" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="802376324" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="802376324" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:24 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Jason Gunthorpe , Gerd Hoffmann , Dongwon Kim , Junxiao Chang Subject: [PATCH v7 2/6] udmabuf: Add back support for mapping hugetlb pages (v6) Date: Mon, 11 Dec 2023 23:37:59 -0800 Message-Id: <20231212073803.3233055-3-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231212073803.3233055-1-vivek.kasireddy@intel.com> References: <20231212073803.3233055-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CC839160029 X-Stat-Signature: gn1pc6axcprgafmy1qbjddna4ek3xsxw X-HE-Tag: 1702368088-958027 X-HE-Meta: U2FsdGVkX1/Q874ulQfUf1zrMGOX1xw1WiPq4F9PuPTRyuVVjj/GW20ztQ75JovD4m1h1DJ1v8+GszFNICRf/4++2W3497ofyUrJdQ1zdh2FK9dks/uWWXt/KCtCwftH/q8xoOoqsgltet39/ni7rkyx6yPqTkn17rXiaxWcK49R8rHx7IPl3+wnUdGLN/Mhg9LCG/w4iZbMqXpAIahFwX26oNF6xoMb1D79Lstv2qVjnOUd1S1NrmO8KnFW/PmksfMzz/9g+TqL8myAS7bN07V+r5UKNokoGlhJxgE309nUsUUaol+yskO9mjW9fTDCfaLoB+iFjwezmwFevz9QUmvSfBv6qoMN0Iweo8zsQjXQvX6mrFlKvLXO2FaHJCj11zVW21oDJxmzrPO8R3V0jgmTt7e57Gdp4dgzmXXNrReaZC4MDgPDlkLX3URXa7e7BNrCn0m7tp+JVzETZdhNxvuE+pcBNfeh8DRsYLS9BvnCBNQLpxhSJhKe97sGSC8W+d74LEvniPKtxfg2CsyimRMpMndp73nvI45v82+DMtgT1susaixps8/c4Tt73HHzdcRUmMiiBLRdl/fQS8Vd8Y2H0IaaSkWoC3lJfjmcgUHfPr4CpJEFUh2z3GVPQSk8Pb1RT9i5orjAzLx16++iwuE9JgWsfJV+cpmwVbVjhFULmkoBmvT+zXkPLPmxC6hQOrrs2e/5MN1Ga4ZFtCXetGICH3/zX0FXU+Y+hbvqV6AZ1kRVDBxgU/xnSEQgiRC952WJD7KBvVo/jYXeTS+7MMU9t7asn0d9X4lwZei4FB47iMBEMrVXurO0mm18sEKA4sqTphJz4kocSZrsKWFmtFg8O55FQDj+KqW2YvU9DJ51DAStjDsnpXyfF1vsiBh2h1372+l0CdGrobEC0KqEFT/2qAwuvl6IQ4pxLo0P1Zgb5ezOk8xl2Mpnco2rZFVig1KW7HGK/guHXqV7qmF 784RPKTu gccA0RBpwiazdZUTB+cMJofL+uROFeeIhVn+2/LOCkq9DLXPXp9h1e3jyi2mQp8APiG9djXv3icdriuoMeqZVUrXIh9fBSYswLkZ3TeTZzoOMBoZZuJJ1hGAvIUuR/H6dMWNCGfR0LVAptWkRsaARjlm+AWtwP7WQSzaYFZxl7vyNR0K5yIqw04rSG8vCW50B4h9cy11PhbHWL4vxpDHGaEUZ5dATz9k+iq+RZHBvILTTv+LZkurCukaxWG5spZ5UQtll X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A user or admin can configure a VMM (Qemu) Guest's memory to be backed by hugetlb pages for various reasons. However, a Guest OS would still allocate (and pin) buffers that are backed by regular 4k sized pages. In order to map these buffers and create dma-bufs for them on the Host, we first need to find the hugetlb pages where the buffer allocations are located and then determine the offsets of individual chunks (within those pages) and use this information to eventually populate a scatterlist. Testcase: default_hugepagesz=2M hugepagesz=2M hugepages=2500 options were passed to the Host kernel and Qemu was launched with these relevant options: qemu-system-x86_64 -m 4096m.... -device virtio-gpu-pci,max_outputs=1,blob=true,xres=1920,yres=1080 -display gtk,gl=on -object memory-backend-memfd,hugetlb=on,id=mem1,size=4096M -machine memory-backend=mem1 Replacing -display gtk,gl=on with -display gtk,gl=off above would exercise the mmap handler. v2: Updated get_sg_table() to manually populate the scatterlist for both huge page and non-huge-page cases. v3: s/offsets/subpgoff/g s/hpoff/mapidx/g v4: Replaced find_get_page_flags() with __filemap_get_folio() to ensure that we only obtain head pages from the mapping v5: Fix the calculation of mapidx to ensure that it is a order-n page multiple v6: - Split the processing of hugetlb or shmem pages into helpers to simplify the code in udmabuf_create() (Christoph) - Move the creation of offsets array out of hugetlb context and into common code Cc: David Hildenbrand Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Jason Gunthorpe Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Acked-by: Mike Kravetz (v2) Signed-off-by: Vivek Kasireddy --- drivers/dma-buf/udmabuf.c | 122 +++++++++++++++++++++++++++++++------- 1 file changed, 101 insertions(+), 21 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index 820c993c8659..274defd3fa3e 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -28,6 +29,7 @@ struct udmabuf { struct page **pages; struct sg_table *sg; struct miscdevice *device; + pgoff_t *offsets; }; static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) @@ -41,6 +43,8 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) return VM_FAULT_SIGBUS; pfn = page_to_pfn(ubuf->pages[pgoff]); + pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT; + return vmf_insert_pfn(vma, vmf->address, pfn); } @@ -90,23 +94,29 @@ static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, { struct udmabuf *ubuf = buf->priv; struct sg_table *sg; + struct scatterlist *sgl; + unsigned int i = 0; int ret; sg = kzalloc(sizeof(*sg), GFP_KERNEL); if (!sg) return ERR_PTR(-ENOMEM); - ret = sg_alloc_table_from_pages(sg, ubuf->pages, ubuf->pagecount, - 0, ubuf->pagecount << PAGE_SHIFT, - GFP_KERNEL); + + ret = sg_alloc_table(sg, ubuf->pagecount, GFP_KERNEL); if (ret < 0) - goto err; + goto err_alloc; + + for_each_sg(sg->sgl, sgl, ubuf->pagecount, i) + sg_set_page(sgl, ubuf->pages[i], PAGE_SIZE, ubuf->offsets[i]); + ret = dma_map_sgtable(dev, sg, direction, 0); if (ret < 0) - goto err; + goto err_map; return sg; -err: +err_map: sg_free_table(sg); +err_alloc: kfree(sg); return ERR_PTR(ret); } @@ -143,6 +153,7 @@ static void release_udmabuf(struct dma_buf *buf) for (pg = 0; pg < ubuf->pagecount; pg++) put_page(ubuf->pages[pg]); + kfree(ubuf->offsets); kfree(ubuf->pages); kfree(ubuf); } @@ -196,17 +207,77 @@ static const struct dma_buf_ops udmabuf_ops = { #define SEALS_WANTED (F_SEAL_SHRINK) #define SEALS_DENIED (F_SEAL_WRITE) +static int handle_hugetlb_pages(struct udmabuf *ubuf, struct file *memfd, + pgoff_t offset, pgoff_t pgcnt, + pgoff_t *pgbuf) +{ + struct hstate *hpstate = hstate_file(memfd); + pgoff_t mapidx = offset >> huge_page_shift(hpstate); + pgoff_t subpgoff = (offset & ~huge_page_mask(hpstate)) >> PAGE_SHIFT; + pgoff_t maxsubpgs = huge_page_size(hpstate) >> PAGE_SHIFT; + struct page *hpage = NULL; + struct folio *folio; + pgoff_t pgidx; + + mapidx <<= huge_page_order(hpstate); + for (pgidx = 0; pgidx < pgcnt; pgidx++) { + if (!hpage) { + folio = __filemap_get_folio(memfd->f_mapping, + mapidx, + FGP_ACCESSED, 0); + if (IS_ERR(folio)) + return PTR_ERR(folio); + + hpage = &folio->page; + } + + get_page(hpage); + ubuf->pages[*pgbuf] = hpage; + ubuf->offsets[*pgbuf] = subpgoff << PAGE_SHIFT; + (*pgbuf)++; + if (++subpgoff == maxsubpgs) { + put_page(hpage); + hpage = NULL; + subpgoff = 0; + mapidx += pages_per_huge_page(hpstate); + } + } + + if (hpage) + put_page(hpage); + + return 0; +} + +static int handle_shmem_pages(struct udmabuf *ubuf, struct file *memfd, + pgoff_t offset, pgoff_t pgcnt, + pgoff_t *pgbuf) +{ + pgoff_t pgidx, pgoff = offset >> PAGE_SHIFT; + struct page *page; + + for (pgidx = 0; pgidx < pgcnt; pgidx++) { + page = shmem_read_mapping_page(memfd->f_mapping, + pgoff + pgidx); + if (IS_ERR(page)) + return PTR_ERR(page); + + ubuf->pages[*pgbuf] = page; + (*pgbuf)++; + } + + return 0; +} + static long udmabuf_create(struct miscdevice *device, struct udmabuf_create_list *head, struct udmabuf_create_item *list) { DEFINE_DMA_BUF_EXPORT_INFO(exp_info); struct file *memfd = NULL; - struct address_space *mapping = NULL; struct udmabuf *ubuf; struct dma_buf *buf; - pgoff_t pgoff, pgcnt, pgidx, pgbuf = 0, pglimit; - struct page *page; + pgoff_t pgcnt, pgbuf = 0, pglimit; int seals, ret = -EINVAL; u32 i, flags; @@ -234,6 +305,12 @@ static long udmabuf_create(struct miscdevice *device, ret = -ENOMEM; goto err; } + ubuf->offsets = kcalloc(ubuf->pagecount, sizeof(*ubuf->offsets), + GFP_KERNEL); + if (!ubuf->offsets) { + ret = -ENOMEM; + goto err; + } pgbuf = 0; for (i = 0; i < head->count; i++) { @@ -241,8 +318,7 @@ static long udmabuf_create(struct miscdevice *device, memfd = fget(list[i].memfd); if (!memfd) goto err; - mapping = memfd->f_mapping; - if (!shmem_mapping(mapping)) + if (!shmem_file(memfd) && !is_file_hugepages(memfd)) goto err; seals = memfd_fcntl(memfd, F_GET_SEALS, 0); if (seals == -EINVAL) @@ -251,16 +327,19 @@ static long udmabuf_create(struct miscdevice *device, if ((seals & SEALS_WANTED) != SEALS_WANTED || (seals & SEALS_DENIED) != 0) goto err; - pgoff = list[i].offset >> PAGE_SHIFT; - pgcnt = list[i].size >> PAGE_SHIFT; - for (pgidx = 0; pgidx < pgcnt; pgidx++) { - page = shmem_read_mapping_page(mapping, pgoff + pgidx); - if (IS_ERR(page)) { - ret = PTR_ERR(page); - goto err; - } - ubuf->pages[pgbuf++] = page; - } + + pgcnt = list[i].size >> PAGE_SHIFT; + if (is_file_hugepages(memfd)) + ret = handle_hugetlb_pages(ubuf, memfd, + list[i].offset, + pgcnt, &pgbuf); + else + ret = handle_shmem_pages(ubuf, memfd, + list[i].offset, + pgcnt, &pgbuf); + if (ret < 0) + goto err; + fput(memfd); memfd = NULL; } @@ -287,6 +366,7 @@ static long udmabuf_create(struct miscdevice *device, put_page(ubuf->pages[--pgbuf]); if (memfd) fput(memfd); + kfree(ubuf->offsets); kfree(ubuf->pages); kfree(ubuf); return ret; From patchwork Tue Dec 12 07:38:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13488635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F0CEC10F04 for ; Tue, 12 Dec 2023 08:01:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B510C6B0299; Tue, 12 Dec 2023 03:01:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ADF186B029D; Tue, 12 Dec 2023 03:01:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8516B029C; Tue, 12 Dec 2023 03:01:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7AA586B0299 for ; Tue, 12 Dec 2023 03:01:34 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 478D140919 for ; Tue, 12 Dec 2023 08:01:34 +0000 (UTC) X-FDA: 81557421708.11.02FBEF8 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf08.hostedemail.com (Postfix) with ESMTP id 11CD1160021 for ; Tue, 12 Dec 2023 08:01:31 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TdiL4rAm; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702368092; a=rsa-sha256; cv=none; b=WaYdPACQRwCz87/nlf9XyJ4Q6d0LxIj2vthxgkQpDsvw1MIilnmOrehTVujDE/ZrXUway+ tL5gze0xqCqeoA01NNR5ywdJoluxLCqeH2axoDYMu4/0kWmVGMJKV1thkTMiIiVcdBJXS1 ygJXqa21A/Q1oUZpv0luVT57i/lYPE4= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TdiL4rAm; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702368092; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QxYyvbRKfLRPUfzCWELaCg+BiNhYcz/y1k9RrEt1cXI=; b=lnroK7g7tek6SOJq7itPYCdHugiGa12LOu6yyA1ejw8KvB0VcsTpTyFmWffAZu7AaunaIe r+UDg6vDC/LlNrwhwW47Ejm2CaRZEU+HBb8YydQYJIKQtgRz/zYUKZEOMcccHuhP2Qi1Gt pim3L0EwqKT/HABBq/5HyC89jkRr/DM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702368092; x=1733904092; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YekEQ8DKML3h6GjGynAdhsl/FD/wBTW7cK6CjMcB3xU=; b=TdiL4rAmfrkBaHrKGCY5SHSjg2XJ7fT8EvsDdnqx+6J4A54puh03zy/2 K62GsFdhA8eeZcrI2dXyGYxjMGTPO3yEWO2diesbOphG4uJwIma+aEFSL GwbInmebYMVuDS2ADostE2E5P8E+MW11f6PKg6/k0tfcQuvBCAv9aFe2f NHcTFop7rn1A326HFFXCAWICA/KDVY9YfG6JB5wW/S4CYIqCnnjTkUM8M GoGzF32nKQ/gbszZA4gpQxPAdU9ddBEjUPf/chMcTqxXX/fEcbpoxWJUr 38E34NbZl4eV0Y1fii/ndJdb20brhzaodVduatsRGFXnVFc8qo/HX8W4J A==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="397553957" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="397553957" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="802376329" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="802376329" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:25 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Christoph Hellwig , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Gerd Hoffmann , Dongwon Kim , Junxiao Chang , Jason Gunthorpe , Christoph Hellwig Subject: [PATCH v7 3/6] mm/gup: Introduce memfd_pin_folios() for pinning memfd folios (v7) Date: Mon, 11 Dec 2023 23:38:00 -0800 Message-Id: <20231212073803.3233055-4-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231212073803.3233055-1-vivek.kasireddy@intel.com> References: <20231212073803.3233055-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 11CD1160021 X-Stat-Signature: ysqrs7deucsodxbnofgcb1d485mmtxu5 X-HE-Tag: 1702368091-425117 X-HE-Meta: U2FsdGVkX1+DdeQHwqdmOI3EVO0bgRcTHPKSCmF4jY+6t/2ZNh0VTGbsOWuxyv9CqmCKUNOnjg1Etg/544NUyGF3hdh958Kr/KSH+AszvHdtEDhl2SwPlVAirgztkZgXP9Fw1xlFGyuDSif2ULp3FOK+yfsahYTTyR6ZW6vjv8+GQVdcvtYjeu+JTWA+eRGMKB3mq+eLPgcNMdJl/wq2Od393rkUCpgEzieTs9gq6MaVPYueZ+9UnpPrV6L87cUoTfN3ZogDZFT5rhLb0vQNfek9PziAVEQi7oX2y3wke1zSgnWGMgcjnYepmRgnffKT2s3H94Zk2SfRXtjQEquRMXZs8BWfdAtEAYBEoTIT+DRnnM1gh4wqfyMpYzspJlTiH1UHOuxhbHFEuK+uJxadsp4fetySb5YcVDimEqLur5YISSpnwqKxi+j5X9gamd59r8IXuY1nOyxGrSEyQ0aJwc1Cm4yOC7uCTFWKLvP4DGAhHYzOi2Y+MqfhntYYqdXQYAB2y785lCqKJ4r07BWbCxrnqUdXPHvmgaq07LXMEpZPEosXQCqMKbSfDViP2Fk5kSwd6CmTrZeFFXjsHtpNikhFBLN3Tgqhs7WwWLatP3xLbHKFwaf6QAL/vfgyZxZuA7PSJjh16zysyKGxqOi8y/NPI8Vx2uhP9/hFsM8DUKi1SPEmx8UGq9rwRw2YAa73pZnO3/KxBSDHOBtoiC9Nmn21Q8P21k9KWXi6q7FdtC8+L8DSVl/MI1yVW0uwiPUQB7v8oXVHVGMmizgTjp0o+Z67OIFVFoRJL+DnTr+RPXd7ZozrbCr4iguZSFexeQmaiQMoWzCgXTqhGDDnMIH+g4ZyrMFiDb6Jmux+GWNAMgij4dBm419Ths7Mtpe9LkFE4HYi8SbO8Ic3rpVGndPiF+ch89NZ1ZQGjpn+5oGu08DuOsWF6zquY2qrR/hQ5krvt/Pz3UQzYXf3HQQMIik Nv5U23OK tnNIyFtUsQrwGzAjJ9nMLFMv9MqKkP64f7sIk999asdlQQLazV5RzvOao4srx/hOkCw+ZtiZhGtUhbKP+Ow2ovlFoK2ooMoyg0mTfh1Fv8ntWrPMpGyXdb8bR7MIaARzFjlOdpBSuEjTVKuUGh4w5n9vzhuGhepu+mx3N/JHOVi9CXztoXF/95VQwJv4A+9QzdD283Hw+xp4a97xpDgtW3P/PE2yujRRB00I8CH/mikqP5SpCzw09TQSOnzqtVd+o9OnDV5dINWdXtgvkdVUnID7+o2ANk+XifhO28xx7oRpqSZPRCyKgGPZqNu/IOlNtDmF/bxmkgzOtgGaOnzM8DGR1FY3IV/LBFe85JJ0u31iKt5A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For drivers that would like to longterm-pin the folios associated with a memfd, the memfd_pin_folios() API provides an option to not only pin the folios via FOLL_PIN but also to check and migrate them if they reside in movable zone or CMA block. This API currently works with memfds but it should work with any files that belong to either shmemfs or hugetlbfs. Files belonging to other filesystems are rejected for now. The folios need to be located first before pinning them via FOLL_PIN. If they are found in the page cache, they can be immediately pinned. Otherwise, they need to be allocated using the filesystem specific APIs and then pinned. v2: - Drop gup_flags and improve comments and commit message (David) - Allocate a page if we cannot find in page cache for the hugetlbfs case as well (David) - Don't unpin pages if there is a migration related failure (David) - Drop the unnecessary nr_pages <= 0 check (Jason) - Have the caller of the API pass in file * instead of fd (Jason) v3: (David) - Enclose the huge page allocation code with #ifdef CONFIG_HUGETLB_PAGE (Build error reported by kernel test robot ) - Don't forget memalloc_pin_restore() on non-migration related errors - Improve the readability of the cleanup code associated with non-migration related errors - Augment the comments by describing FOLL_LONGTERM like behavior - Include the R-b tag from Jason v4: - Remove the local variable "page" and instead use 3 return statements in alloc_file_page() (David) - Add the R-b tag from David v5: (David) - For hugetlb case, ensure that we only obtain head pages from the mapping by using __filemap_get_folio() instead of find_get_page_flags() - Handle -EEXIST when two or more potential users try to simultaneously add a huge page to the mapping by forcing them to retry on failure v6: (Christoph) - Rename this API to memfd_pin_user_pages() to make it clear that it is intended for memfds - Move the memfd page allocation helper from gup.c to memfd.c - Fix indentation errors in memfd_pin_user_pages() - For contiguous ranges of folios, use a helper such as filemap_get_folios_contig() to lookup the page cache in batches v7: - Rename this API to memfd_pin_folios() and make it return folios and offsets instead of pages (David) - Don't continue processing the folios in the batch returned by filemap_get_folios_contig() if they do not have correct next_idx - Add the R-b tag from Christoph Cc: David Hildenbrand Cc: Christoph Hellwig Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Suggested-by: Jason Gunthorpe Reviewed-by: Jason Gunthorpe (v2) Reviewed-by: David Hildenbrand (v3) Reviewed-by: Christoph Hellwig (v6) Signed-off-by: Vivek Kasireddy --- include/linux/memfd.h | 5 ++ include/linux/mm.h | 3 + mm/gup.c | 155 ++++++++++++++++++++++++++++++++++++++++++ mm/memfd.c | 34 +++++++++ 4 files changed, 197 insertions(+) diff --git a/include/linux/memfd.h b/include/linux/memfd.h index e7abf6fa4c52..b38fb35f4271 100644 --- a/include/linux/memfd.h +++ b/include/linux/memfd.h @@ -6,11 +6,16 @@ #ifdef CONFIG_MEMFD_CREATE extern long memfd_fcntl(struct file *file, unsigned int cmd, unsigned int arg); +extern struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx); #else static inline long memfd_fcntl(struct file *f, unsigned int c, unsigned int a) { return -EINVAL; } +static inline struct page *memfd_alloc_folio(struct file *memfd, pgoff_t idx) +{ + return ERR_PTR(-EINVAL); +} #endif #endif /* __LINUX_MEMFD_H */ diff --git a/include/linux/mm.h b/include/linux/mm.h index 418d26608ece..537f40989837 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2472,6 +2472,9 @@ long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); +long memfd_pin_folios(struct file *file, unsigned long start, + unsigned long nr_pages, struct folio **folios, + pgoff_t *offsets); int get_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages); diff --git a/mm/gup.c b/mm/gup.c index 231711efa390..e798cdbc6a11 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -5,6 +5,7 @@ #include #include +#include #include #include #include @@ -17,6 +18,7 @@ #include #include #include +#include #include #include @@ -3410,3 +3412,156 @@ long pin_user_pages_unlocked(unsigned long start, unsigned long nr_pages, &locked, gup_flags); } EXPORT_SYMBOL(pin_user_pages_unlocked); + +/** + * memfd_pin_folios() - pin folios associated with a memfd + * @memfd: the memfd whose folios are to be pinned + * @start: starting memfd offset + * @nr_pages: number of pages from start to pin + * @folios: array that receives pointers to the folios pinned. + * Should be at-least nr_pages long. + * @offsets: array that receives offsets of pages in their folios. + * Should be at-least nr_pages long. + * + * Attempt to pin folios associated with a memfd; given that a memfd is + * either backed by shmem or hugetlb, the folios can either be found in + * the page cache or need to be allocated if necessary. Once the folios + * are located, they are all pinned via FOLL_PIN and the @offsets array + * is populated with offsets of the pages in their respective folios. + * Therefore, for each page the caller requested, there will be a + * corresponding entry in both @folios and @offsets. And, eventually, + * these pinned folios need to be released either using unpin_user_pages() + * or unpin_user_page(). + * + * It must be noted that the folios may be pinned for an indefinite amount + * of time. And, in most cases, the duration of time they may stay pinned + * would be controlled by the userspace. This behavior is effectively the + * same as using FOLL_LONGTERM with other GUP APIs. + * + * Returns number of folios pinned. This would be equal to the number of + * pages requested. If no folios were pinned, it returns -errno. + */ +long memfd_pin_folios(struct file *memfd, unsigned long start, + unsigned long nr_pages, struct folio **folios, + pgoff_t *offsets) +{ + unsigned long end = start + (nr_pages << PAGE_SHIFT) - 1; + unsigned int max_pgs, pgoff, pgshift = PAGE_SHIFT; + pgoff_t start_idx, end_idx, next_idx; + unsigned int flags, nr_folios, i, j; + struct folio *folio = NULL; + struct folio_batch fbatch; + struct page **pages; + struct hstate *h; + long ret; + + if (!nr_pages) + return -EINVAL; + + if (!memfd) + return -EINVAL; + + if (!shmem_file(memfd) && !is_file_hugepages(memfd)) + return -EINVAL; + + pages = kmalloc_array(nr_pages, sizeof(*pages), GFP_KERNEL); + if (!pages) + return -ENOMEM; + + if (is_file_hugepages(memfd)) { + h = hstate_file(memfd); + pgshift = huge_page_shift(h); + } + + flags = memalloc_pin_save(); + do { + i = 0; + start_idx = start >> pgshift; + end_idx = end >> pgshift; + if (is_file_hugepages(memfd)) { + start_idx <<= huge_page_order(h); + end_idx <<= huge_page_order(h); + } + + folio_batch_init(&fbatch); + while (start_idx <= end_idx) { + /* + * In most cases, we should be able to find the folios + * in the page cache. If we cannot find them for some + * reason, we try to allocate them and add them to the + * page cache. + */ + nr_folios = filemap_get_folios_contig(memfd->f_mapping, + &start_idx, + end_idx, + &fbatch); + if (folio) { + folio_put(folio); + folio = NULL; + } + + next_idx = 0; + for (j = 0; j < nr_folios; j++) { + if (next_idx && + next_idx != folio_index(fbatch.folios[j])) + continue; + + folio = try_grab_folio(&fbatch.folios[j]->page, + 1, FOLL_PIN); + if (!folio) { + folio_batch_release(&fbatch); + kfree(pages); + goto err; + } + + max_pgs = folio_nr_pages(folio); + if (i == 0) { + pgoff = offset_in_folio(folio, start); + pgoff >>= PAGE_SHIFT; + } + + do { + folios[i] = folio; + offsets[i] = pgoff << PAGE_SHIFT; + pages[i] = folio_page(folio, 0); + folio_add_pin(folio); + + pgoff++; + i++; + } while (pgoff < max_pgs && i < nr_pages); + + pgoff = 0; + next_idx = folio_next_index(folio); + gup_put_folio(folio, 1, FOLL_PIN); + } + + folio = NULL; + folio_batch_release(&fbatch); + if (!nr_folios) { + folio = memfd_alloc_folio(memfd, start_idx); + if (IS_ERR(folio)) { + ret = PTR_ERR(folio); + if (ret != -EEXIST) { + kfree(pages); + goto err; + } + } + } + } + + ret = check_and_migrate_movable_pages(nr_pages, pages); + } while (ret == -EAGAIN); + + kfree(pages); + memalloc_pin_restore(flags); + return ret ? ret : nr_pages; +err: + memalloc_pin_restore(flags); + while (i-- > 0) + if (folios[i]) + gup_put_folio(folios[i], 1, FOLL_PIN); + + return ret; +} +EXPORT_SYMBOL_GPL(memfd_pin_folios); + diff --git a/mm/memfd.c b/mm/memfd.c index d3a1ba4208c9..36a75e8249f8 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -63,6 +63,40 @@ static void memfd_tag_pins(struct xa_state *xas) xas_unlock_irq(xas); } +/* + * This is a helper function used by memfd_pin_user_pages() in GUP (gup.c). + * It is mainly called to allocate a page in a memfd when the caller + * (memfd_pin_user_pages()) cannot find a page in the page cache at a given + * index in the mapping. + */ +struct folio *memfd_alloc_folio(struct file *memfd, pgoff_t idx) +{ +#ifdef CONFIG_HUGETLB_PAGE + struct folio *folio; + int err; + + if (is_file_hugepages(memfd)) { + folio = alloc_hugetlb_folio_nodemask(hstate_file(memfd), + NUMA_NO_NODE, + NULL, + GFP_USER); + if (folio && folio_try_get(folio)) { + err = hugetlb_add_to_page_cache(folio, + memfd->f_mapping, + idx); + if (err) { + folio_put(folio); + free_huge_folio(folio); + return ERR_PTR(err); + } + return folio; + } + return ERR_PTR(-ENOMEM); + } +#endif + return shmem_read_folio(memfd->f_mapping, idx); +} + /* * Setting SEAL_WRITE requires us to verify there's no pending writer. However, * via get_user_pages(), drivers might have some pending I/O without any active From patchwork Tue Dec 12 07:38:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13488636 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED5B2C4167D for ; Tue, 12 Dec 2023 08:01:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ECC986B029B; Tue, 12 Dec 2023 03:01:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E7CFE6B029C; Tue, 12 Dec 2023 03:01:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD51C6B029E; Tue, 12 Dec 2023 03:01:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AA0036B029B for ; Tue, 12 Dec 2023 03:01:34 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 69BA11A0902 for ; Tue, 12 Dec 2023 08:01:34 +0000 (UTC) X-FDA: 81557421708.13.94EBA2B Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf18.hostedemail.com (Postfix) with ESMTP id 47DBD1C001F for ; Tue, 12 Dec 2023 08:01:32 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=O9qM7cxz; spf=pass (imf18.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702368092; a=rsa-sha256; cv=none; b=CJCeoC6yYnfq+x0vwALnUbKc6+fwBm0Pfp9ZzevMJlWLLIeVUXNGrIFj7mNn9oFw46udEw ajA8SUJBU0gUNAknHOyGZPGYRVmhfJocU8Qyc8I0Ds3A5I9NqTNuhfRXzm/hAFwPND4DCg ppvXWfOxnht3RF90I7Eruve85I+zUBU= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=O9qM7cxz; spf=pass (imf18.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702368092; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NtGqQorwK6XjMKtiB6PpCEar5h8TkXplS2dOJZUVwIk=; b=R3AgMg/XcwCG6BWsA8wNM6JMSC6qnTRlO/xKNHjxfr797lt+eZk5+0HP3xKWm9oPmAW0Uv ojpyjVnpETAz4RSva0CKBcY0JHaMmkX52xkuVQYEwpDnM87lmbl83sk83/qsqE67Db1NFC wwe7UT3/ywBzMi1FNpT3N6QdUxIvF2A= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702368092; x=1733904092; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JRJidqYXQ7ISKysa3iQ61C2SFbl51me8hEe7o/dSDng=; b=O9qM7cxzTEcN7NIFsv600guEt4gA5M28FRTTrUdCvzRL6vaE1uSJ3mEn kxqQjtUhI6Xo0MAPxsWJWxZaZ+Q2fQULJ0UMH5IIvcSmzHPhPgHwEM1Tz gTo3fGIK5Kw6tGCUS8826QYByCkio95YrZ9VgrKAKDja6u6wA/hEvN70K OS0NO+pbTnOUzuDUtdebMvdgP6Fr2syGqxuUkRwiWLgaFahbM3rQ5Bhzy vEQmuli1tFkTINylKTpGEth5oMjkC4zpDrB8Z0GdqlAmsL1DAy7qSPxl0 8Nm/+oSjtaVc5wQGtY4v5cLjNgMqLna6YIecs93AlVRu2+p7Q9tilTIir g==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="397553969" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="397553969" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="802376332" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="802376332" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:25 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Jason Gunthorpe , Gerd Hoffmann , Dongwon Kim , Junxiao Chang Subject: [PATCH v7 4/6] udmabuf: Convert udmabuf driver to use folios Date: Mon, 11 Dec 2023 23:38:01 -0800 Message-Id: <20231212073803.3233055-5-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231212073803.3233055-1-vivek.kasireddy@intel.com> References: <20231212073803.3233055-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 47DBD1C001F X-Stat-Signature: x6jipfnrj7i3ra8n645qbg5z73w8tena X-Rspam-User: X-HE-Tag: 1702368092-386501 X-HE-Meta: U2FsdGVkX1+soPzW52leq0cZh6SqjFrXEZSG+shDgRhJQQHMkMZqlDdaCdfK8XUNhWYmm1q4EEFtaKJptpUExqcUnqgtGWpJMy9vMjSSri8w6D8QZS70rwIr6xdVXI1jcZv+l6AD9/EarvEd3w2g+lHyaI66plT0B8j4obPtuhe5t9tKHxAUP1W4X8BaJcieia/MwlV3vIRtheL7DNkd6FRJfrpRSc4aXU9f6X2VTHyG2iJK0F2lksUzT6AXcZ4Ax8iqtL8pryqgPPUcVSeTM55VoftbBCVkYwUrHtkaAL6QzLjL7tGMBglmMYA6/6nCL7TQP9JMSwQ8B54e1sitKLi6BK42FDGea/9xbQQ9Krr7/Rl4+GuQ1yxrMRERr7QG7EtbTRD0Z3ZDV27/7fdQ73dLYX6IlusGITEfgXq3h6K/pjgx5N08vKU7NeiMGpasQPsvXmVKYKdw7AAHvheUYsLKNyOeHHP00TBhrXcoS/KY861DLs9vtfbZxbK/sSa4DLJW5UfRnJooonpcdS1rGZ31aChuOHMud61aODjt4D8DeJdPNcKzFzHzcNTAH9P2Bck7O9ft92y24ennw5O+JGn3uPep5fOOitIF1EHdQeDcrnJuLW8eyLS7wueiiwbqrTvQk4wvYSfALlacMmu04/uD2VikID3wOd9U0nDkTKFMM7yaNeaqtOXKyECKVC3aNd6DrFMbtN4SoS5uXtMfHfPtLDd6OTe4MWmthbh7B1wCCG8qO9M0dEMvnOVWNXsLXeOHGaxVixiLMoRttkEF63Mxyce2ziueNW1YVZzi+gwADEVIsdyBjF+OW6Zs6bmacFO+tuwYLDjqBKffYypaWg8huIco1O/GYZ14PAUwYMkOstoVeCeb+Xidy0w47+BY1fZqc7VAU4ASzOBSKTjPu9eHYZWXP9hEZvClNMLchpdZxNkIqv7bmBDQKz4vMukHiZs3SsyVwTHIAvNauWf +HA0fHZO XhT8Fs77ZK+XVlbAU+2DMadb+Y9p/uW/BStQVM+Z9MabETMD2A2NjLaM3R1+DX9PL47IMx36bKbodXP2XoAC+Qsv0dMxWRsFmF+EVwo59kQiHbfUR7sFaUWsZz2oTp1YPHwZYXY6fFXSFnxW+YYkqsTi9h2mB31YC0mXB5y7KVmbaNzAvK2KP6nbuHD6HDuhfCx+6xhwGXGmby2lpJzBzgwlW5q4wTGFBAdlNIO8MiPWLmnfKhG6//YmZ6dk0yCiS6Bzn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is mainly a preparatory patch to use memfd_pin_folios() API for pinning folios. Using folios instead of pages makes sense as the udmabuf driver needs to handle both shmem and hugetlb cases. However, the function vmap_udmabuf() still needs a list of pages; so, we collect all the head pages into a local array in this case. Other changes in this patch include the addition of helpers for checking the memfd seals and exporting dmabuf. Moving code from udmabuf_create() into these helpers improves readability given that udmabuf_create() is a bit long. Cc: David Hildenbrand Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Jason Gunthorpe Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Signed-off-by: Vivek Kasireddy --- drivers/dma-buf/udmabuf.c | 139 +++++++++++++++++++++++--------------- 1 file changed, 83 insertions(+), 56 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index 274defd3fa3e..e1b8da3c9b2a 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -26,7 +26,7 @@ MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is struct udmabuf { pgoff_t pagecount; - struct page **pages; + struct folio **folios; struct sg_table *sg; struct miscdevice *device; pgoff_t *offsets; @@ -42,7 +42,7 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) if (pgoff >= ubuf->pagecount) return VM_FAULT_SIGBUS; - pfn = page_to_pfn(ubuf->pages[pgoff]); + pfn = page_to_pfn(&ubuf->folios[pgoff]->page); pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT; return vmf_insert_pfn(vma, vmf->address, pfn); @@ -68,11 +68,21 @@ static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma) static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map) { struct udmabuf *ubuf = buf->priv; + struct page **pages; void *vaddr; + pgoff_t pg; dma_resv_assert_held(buf->resv); - vaddr = vm_map_ram(ubuf->pages, ubuf->pagecount, -1); + pages = kmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL); + if (!pages) + return -ENOMEM; + + for (pg = 0; pg < ubuf->pagecount; pg++) + pages[pg] = &ubuf->folios[pg]->page; + + vaddr = vm_map_ram(pages, ubuf->pagecount, -1); + kfree(pages); if (!vaddr) return -EINVAL; @@ -107,7 +117,8 @@ static struct sg_table *get_sg_table(struct device *dev, struct dma_buf *buf, goto err_alloc; for_each_sg(sg->sgl, sgl, ubuf->pagecount, i) - sg_set_page(sgl, ubuf->pages[i], PAGE_SIZE, ubuf->offsets[i]); + sg_set_folio(sgl, ubuf->folios[i], PAGE_SIZE, + ubuf->offsets[i]); ret = dma_map_sgtable(dev, sg, direction, 0); if (ret < 0) @@ -152,9 +163,9 @@ static void release_udmabuf(struct dma_buf *buf) put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL); for (pg = 0; pg < ubuf->pagecount; pg++) - put_page(ubuf->pages[pg]); + folio_put(ubuf->folios[pg]); kfree(ubuf->offsets); - kfree(ubuf->pages); + kfree(ubuf->folios); kfree(ubuf); } @@ -215,36 +226,33 @@ static int handle_hugetlb_pages(struct udmabuf *ubuf, struct file *memfd, pgoff_t mapidx = offset >> huge_page_shift(hpstate); pgoff_t subpgoff = (offset & ~huge_page_mask(hpstate)) >> PAGE_SHIFT; pgoff_t maxsubpgs = huge_page_size(hpstate) >> PAGE_SHIFT; - struct page *hpage = NULL; - struct folio *folio; + struct folio *folio = NULL; pgoff_t pgidx; mapidx <<= huge_page_order(hpstate); for (pgidx = 0; pgidx < pgcnt; pgidx++) { - if (!hpage) { + if (!folio) { folio = __filemap_get_folio(memfd->f_mapping, mapidx, FGP_ACCESSED, 0); if (IS_ERR(folio)) return PTR_ERR(folio); - - hpage = &folio->page; } - get_page(hpage); - ubuf->pages[*pgbuf] = hpage; + folio_get(folio); + ubuf->folios[*pgbuf] = folio; ubuf->offsets[*pgbuf] = subpgoff << PAGE_SHIFT; (*pgbuf)++; if (++subpgoff == maxsubpgs) { - put_page(hpage); - hpage = NULL; + folio_put(folio); + folio = NULL; subpgoff = 0; mapidx += pages_per_huge_page(hpstate); } } - if (hpage) - put_page(hpage); + if (folio) + folio_put(folio); return 0; } @@ -254,31 +262,70 @@ static int handle_shmem_pages(struct udmabuf *ubuf, struct file *memfd, pgoff_t *pgbuf) { pgoff_t pgidx, pgoff = offset >> PAGE_SHIFT; - struct page *page; + struct folio *folio = NULL; for (pgidx = 0; pgidx < pgcnt; pgidx++) { - page = shmem_read_mapping_page(memfd->f_mapping, - pgoff + pgidx); - if (IS_ERR(page)) - return PTR_ERR(page); + folio = shmem_read_folio(memfd->f_mapping, + pgoff + pgidx); + if (IS_ERR(folio)) + return PTR_ERR(folio); - ubuf->pages[*pgbuf] = page; + ubuf->folios[*pgbuf] = folio; (*pgbuf)++; } return 0; } +static int check_memfd_seals(struct file *memfd) +{ + int seals; + + if (!memfd) + return -EBADFD; + + if (!shmem_file(memfd) && !is_file_hugepages(memfd)) + return -EBADFD; + + seals = memfd_fcntl(memfd, F_GET_SEALS, 0); + if (seals == -EINVAL) + return -EBADFD; + + if ((seals & SEALS_WANTED) != SEALS_WANTED || + (seals & SEALS_DENIED) != 0) + return -EINVAL; + + return 0; +} + +static int export_udmabuf(struct udmabuf *ubuf, + struct miscdevice *device, + u32 flags) +{ + DEFINE_DMA_BUF_EXPORT_INFO(exp_info); + struct dma_buf *buf; + + ubuf->device = device; + exp_info.ops = &udmabuf_ops; + exp_info.size = ubuf->pagecount << PAGE_SHIFT; + exp_info.priv = ubuf; + exp_info.flags = O_RDWR; + + buf = dma_buf_export(&exp_info); + if (IS_ERR(buf)) + return PTR_ERR(buf); + + return dma_buf_fd(buf, flags); +} + static long udmabuf_create(struct miscdevice *device, struct udmabuf_create_list *head, struct udmabuf_create_item *list) { - DEFINE_DMA_BUF_EXPORT_INFO(exp_info); + pgoff_t pgcnt, pgbuf = 0, pglimit; struct file *memfd = NULL; struct udmabuf *ubuf; - struct dma_buf *buf; - pgoff_t pgcnt, pgbuf = 0, pglimit; - int seals, ret = -EINVAL; + int ret = -EINVAL; u32 i, flags; ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL); @@ -299,9 +346,9 @@ static long udmabuf_create(struct miscdevice *device, if (!ubuf->pagecount) goto err; - ubuf->pages = kmalloc_array(ubuf->pagecount, sizeof(*ubuf->pages), + ubuf->folios = kmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios), GFP_KERNEL); - if (!ubuf->pages) { + if (!ubuf->folios) { ret = -ENOMEM; goto err; } @@ -314,18 +361,8 @@ static long udmabuf_create(struct miscdevice *device, pgbuf = 0; for (i = 0; i < head->count; i++) { - ret = -EBADFD; memfd = fget(list[i].memfd); - if (!memfd) - goto err; - if (!shmem_file(memfd) && !is_file_hugepages(memfd)) - goto err; - seals = memfd_fcntl(memfd, F_GET_SEALS, 0); - if (seals == -EINVAL) - goto err; - ret = -EINVAL; - if ((seals & SEALS_WANTED) != SEALS_WANTED || - (seals & SEALS_DENIED) != 0) + if (check_memfd_seals(memfd) < 0) goto err; pgcnt = list[i].size >> PAGE_SHIFT; @@ -344,30 +381,20 @@ static long udmabuf_create(struct miscdevice *device, memfd = NULL; } - exp_info.ops = &udmabuf_ops; - exp_info.size = ubuf->pagecount << PAGE_SHIFT; - exp_info.priv = ubuf; - exp_info.flags = O_RDWR; - - ubuf->device = device; - buf = dma_buf_export(&exp_info); - if (IS_ERR(buf)) { - ret = PTR_ERR(buf); + flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0; + ret = export_udmabuf(ubuf, device, flags); + if (ret < 0) goto err; - } - flags = 0; - if (head->flags & UDMABUF_FLAGS_CLOEXEC) - flags |= O_CLOEXEC; - return dma_buf_fd(buf, flags); + return ret; err: while (pgbuf > 0) - put_page(ubuf->pages[--pgbuf]); + folio_put(ubuf->folios[--pgbuf]); if (memfd) fput(memfd); kfree(ubuf->offsets); - kfree(ubuf->pages); + kfree(ubuf->folios); kfree(ubuf); return ret; } From patchwork Tue Dec 12 07:38:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13488637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FE69C4167B for ; Tue, 12 Dec 2023 08:01:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FCFA6B029C; Tue, 12 Dec 2023 03:01:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 137686B029E; Tue, 12 Dec 2023 03:01:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECDC46B02A0; Tue, 12 Dec 2023 03:01:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D88F36B029C for ; Tue, 12 Dec 2023 03:01:35 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9F5F64091D for ; Tue, 12 Dec 2023 08:01:35 +0000 (UTC) X-FDA: 81557421750.20.A81A6A4 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf17.hostedemail.com (Postfix) with ESMTP id 90D994000C for ; Tue, 12 Dec 2023 08:01:33 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HVRFRBbr; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf17.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702368093; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TmPdM8lTasRCij/S+x27V0M5JMNO4KjKoYtlWEm7eBs=; b=J6YEl5E5kH6Ui9clDke3p+WZtJgxXl4mgJflm+mVWUUTu8CTtkk3OBbny0lVc8ppReiaPo ToVxQ+zagUWPq7V0f0zIrzvXOzrfL6qVxNBwG9wyaC6fXfrv98QjkQy0EPRvQGxz2BRhJn QuTSjVPeh163sMbTzTtzywiaeFjsG+4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HVRFRBbr; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf17.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702368093; a=rsa-sha256; cv=none; b=liysAxT7b5etTTnU4x3EtF4aHQN/Fdf7AAU5ZE+lG6Kwe/H6jfd6IQ11LxikX4/9xDoU3v IhU8tzp5gTZdyuoEQOzBJqcGLVbeWE0gtXhKnsuy2Td4MlU8kSDREbcVmyrxug6JQyHny+ BFjtt9u7du0LnvcVk30BMrFkKcDj9BM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702368093; x=1733904093; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mAcJFIew1ykttdyyN6tBQbZzsTFAdhUHsqSaxgVF/4Y=; b=HVRFRBbrG28NIKto6gvk9lxKghty9u8eYqet6gmyECo9QC3O/kpXGSut L4dbWC6CdvC+E6gvhTJ4Mb2FIcnBMqV+i1XnaFdxh1uS5C9XU1q2D0Yya +1+mCJHBucjQ1JBotWD2bKu5YhBvuZ5hjSywWcy7peIF2VEwvrPRAEOy4 axVjufS40TwgreNjFiwG6Zx1MW7sV29E7j8/epaQNnGJ4N4eEXcG/GEJA YIdRfXppCkSX7be4RXA/ob+819FCki5DZyEzv9ZHok+tl0mC6fynPYa33 rskBdFXtIUbnWwnR76WvKoaMF6N7zvY1SJXm/Ll9GWZY/Nc6LBj6q619O A==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="397553983" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="397553983" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="802376335" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="802376335" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:25 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Jason Gunthorpe , Gerd Hoffmann , Dongwon Kim , Junxiao Chang Subject: [PATCH v7 5/6] udmabuf: Pin the pages using memfd_pin_folios() API (v5) Date: Mon, 11 Dec 2023 23:38:02 -0800 Message-Id: <20231212073803.3233055-6-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231212073803.3233055-1-vivek.kasireddy@intel.com> References: <20231212073803.3233055-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 90D994000C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 813jbpsenhgxpuya67daabhjy586a5um X-HE-Tag: 1702368093-968130 X-HE-Meta: U2FsdGVkX1/tbOkaYI8FRxe6druB1otnIi5Zsay2vMbhsiXOd3lCKJFSQlxS9NmHGl9BL7C37/cpNiBg7OZv1tUHkKgYeo/mJoF7iIpgW0Ceh+zF07kZZpVBdlQrbEB+YFJDm7Dle/aGlxp67qnm8C48VP7fXxSWC4YQAEwkT2m1eaLwSZ1pvzIZArX9skssK4JiXCYn5MdCY+bALnHcVmpitqbZNegqQ17rPm/6GrWOorAfuCX3uGLtQIQsETOcMcgyr7uoU2ceyvrrsjScaHPsTFzlCUR6FkPRiPPaZln2krcCmkwZa8iwLlS+8R7I6NYr6dCFEXXXSIeZDc8ruJhLeIP+fFAiaBotyFCu5y+5huv44DL2sN7LYdgnQ6ByFwVs7SWv1/9PN8ny5FjcIhv9ArZcjJinjkP5CXVcsqPeqzqjox4GINkdbB7D9MHl0/f0ju/k3OEk1PRwCtNQukEQPBJphgdcDjaPLr49rNAjtwb6jgiyCRcELbRDMoX4M9g/m+DbIfRwmuI31NU3gvx0arVTbYndEyymZBZyGqZZqNR11x2uZaqaqCExeRFS17Vc/ZQ0UFDe6OI3Ps9TjD56+HFsMsAbTz0n8yRw8WlSSu4/hZ575CCiJ3A5fApmLt1SndEC1AXEUiN1zeXKe7tZtGdu5IbzZhVyTUz7BTNnuX0XU67tmVTJMG8nN+tZ4aBMmGWWx+gGzHTC1eoEVXa0j0XypCoJocnuTKZd/wbygUwcLPPkxwgozHjK1aYVSHHFUL+QBTJ0JkMS/JBHDU+HGC6uR+OIUh2NDGMrzlk8YqNO8q2odDd/eQ1J3Ufb4yVws7bZjVFWFBqBFmaPNDrmS3IJb5XehCJvc9WZ9MRm73mU2VxG0N7DNdOJoPoSoHBhIe09ygL/FfaJrEVbz3+x+jn6VI1qdjGHiLbKecwfdX0sp5yamdZWxcLI+Mzv+0BK8/bG4vZca4H/T8g MMMrmDAK uJ+b74niHbPgPWjTwS0q96scgRmtwxXfMG0lc9fBuDz0U6LXDG3T77isNNy15Naw2eMmeKZTYZA6Vm/KXE5RU1jX/hWGetU46yKMt0g24KFd3AE5X8TwwncnOzDoKKVmsL8ZJh3KExec6TPoOs6SYmN4tPwki1VUrnaERacKcxV9ldZ3G1ZMGX03TSrDQMDeatY/bwQuVvwgFEiH5zlaaEXMx+visrQljMlOmLUH3M4JulLgOajPZtWB0ezuGlUhxRk0LRQeXPuG72TAmDVSlZ0/4RS4PnwGeW4SekuXRANX6Y34= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Using memfd_pin_folios() will ensure that the pages are pinned correctly using FOLL_PIN. And, this also ensures that we don't accidentally break features such as memory hotunplug as it would not allow pinning pages in the movable zone. Using this new API also simplifies the code as we no longer have to deal with extracting individual pages from their mappings or handle shmem and hugetlb cases separately. v2: - Adjust to the change in signature of pin_user_pages_fd() by passing in file * instead of fd. v3: - Limit the changes in this patch only to those that are required for using pin_user_pages_fd() - Slightly improve the commit message v4: - Adjust to the change in name of the API (memfd_pin_user_pages) v5: - Adjust to the changes in memfd_pin_folios which now populates a list of folios and offsets Cc: David Hildenbrand Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Jason Gunthorpe Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Signed-off-by: Vivek Kasireddy --- drivers/dma-buf/udmabuf.c | 85 ++++++--------------------------------- 1 file changed, 12 insertions(+), 73 deletions(-) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index e1b8da3c9b2a..a614e720837d 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -42,7 +42,7 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) if (pgoff >= ubuf->pagecount) return VM_FAULT_SIGBUS; - pfn = page_to_pfn(&ubuf->folios[pgoff]->page); + pfn = page_to_pfn(folio_page(ubuf->folios[pgoff], 0)); pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT; return vmf_insert_pfn(vma, vmf->address, pfn); @@ -79,7 +79,7 @@ static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map) return -ENOMEM; for (pg = 0; pg < ubuf->pagecount; pg++) - pages[pg] = &ubuf->folios[pg]->page; + pages[pg] = folio_page(ubuf->folios[pg], 0); vaddr = vm_map_ram(pages, ubuf->pagecount, -1); kfree(pages); @@ -163,7 +163,8 @@ static void release_udmabuf(struct dma_buf *buf) put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL); for (pg = 0; pg < ubuf->pagecount; pg++) - folio_put(ubuf->folios[pg]); + unpin_user_page(folio_page(ubuf->folios[pg], 0)); + kfree(ubuf->offsets); kfree(ubuf->folios); kfree(ubuf); @@ -218,65 +219,6 @@ static const struct dma_buf_ops udmabuf_ops = { #define SEALS_WANTED (F_SEAL_SHRINK) #define SEALS_DENIED (F_SEAL_WRITE) -static int handle_hugetlb_pages(struct udmabuf *ubuf, struct file *memfd, - pgoff_t offset, pgoff_t pgcnt, - pgoff_t *pgbuf) -{ - struct hstate *hpstate = hstate_file(memfd); - pgoff_t mapidx = offset >> huge_page_shift(hpstate); - pgoff_t subpgoff = (offset & ~huge_page_mask(hpstate)) >> PAGE_SHIFT; - pgoff_t maxsubpgs = huge_page_size(hpstate) >> PAGE_SHIFT; - struct folio *folio = NULL; - pgoff_t pgidx; - - mapidx <<= huge_page_order(hpstate); - for (pgidx = 0; pgidx < pgcnt; pgidx++) { - if (!folio) { - folio = __filemap_get_folio(memfd->f_mapping, - mapidx, - FGP_ACCESSED, 0); - if (IS_ERR(folio)) - return PTR_ERR(folio); - } - - folio_get(folio); - ubuf->folios[*pgbuf] = folio; - ubuf->offsets[*pgbuf] = subpgoff << PAGE_SHIFT; - (*pgbuf)++; - if (++subpgoff == maxsubpgs) { - folio_put(folio); - folio = NULL; - subpgoff = 0; - mapidx += pages_per_huge_page(hpstate); - } - } - - if (folio) - folio_put(folio); - - return 0; -} - -static int handle_shmem_pages(struct udmabuf *ubuf, struct file *memfd, - pgoff_t offset, pgoff_t pgcnt, - pgoff_t *pgbuf) -{ - pgoff_t pgidx, pgoff = offset >> PAGE_SHIFT; - struct folio *folio = NULL; - - for (pgidx = 0; pgidx < pgcnt; pgidx++) { - folio = shmem_read_folio(memfd->f_mapping, - pgoff + pgidx); - if (IS_ERR(folio)) - return PTR_ERR(folio); - - ubuf->folios[*pgbuf] = folio; - (*pgbuf)++; - } - - return 0; -} - static int check_memfd_seals(struct file *memfd) { int seals; @@ -325,7 +267,7 @@ static long udmabuf_create(struct miscdevice *device, pgoff_t pgcnt, pgbuf = 0, pglimit; struct file *memfd = NULL; struct udmabuf *ubuf; - int ret = -EINVAL; + long ret = -EINVAL; u32 i, flags; ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL); @@ -366,17 +308,13 @@ static long udmabuf_create(struct miscdevice *device, goto err; pgcnt = list[i].size >> PAGE_SHIFT; - if (is_file_hugepages(memfd)) - ret = handle_hugetlb_pages(ubuf, memfd, - list[i].offset, - pgcnt, &pgbuf); - else - ret = handle_shmem_pages(ubuf, memfd, - list[i].offset, - pgcnt, &pgbuf); + ret = memfd_pin_folios(memfd, list[i].offset, pgcnt, + ubuf->folios + pgbuf, + ubuf->offsets + pgbuf); if (ret < 0) goto err; + pgbuf += pgcnt; fput(memfd); memfd = NULL; } @@ -389,8 +327,9 @@ static long udmabuf_create(struct miscdevice *device, return ret; err: - while (pgbuf > 0) - folio_put(ubuf->folios[--pgbuf]); + while (pgbuf-- > 0) + if (ubuf->folios[pgbuf]) + unpin_user_page(folio_page(ubuf->folios[pgbuf], 0)); if (memfd) fput(memfd); kfree(ubuf->offsets); From patchwork Tue Dec 12 07:38:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13488638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6165DC4332F for ; Tue, 12 Dec 2023 08:01:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE4866B029E; Tue, 12 Dec 2023 03:01:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D92E96B02A1; Tue, 12 Dec 2023 03:01:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0CFC6B02A2; Tue, 12 Dec 2023 03:01:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id ABC876B029E for ; Tue, 12 Dec 2023 03:01:37 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 72D8EC0632 for ; Tue, 12 Dec 2023 08:01:37 +0000 (UTC) X-FDA: 81557421834.25.B52B41F Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by imf08.hostedemail.com (Postfix) with ESMTP id 43D5A160019 for ; Tue, 12 Dec 2023 08:01:35 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JnID1DBm; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702368095; a=rsa-sha256; cv=none; b=4z59VxNlkaG4XNpUiaWorbgZ/N80ykA30m0JFQE9KSm2EQmkQOiAzOpbBLs5BbqW7qxNp3 FMOXDChCGZAcZiYGzyVwT7aFQqcz0UPrceAD6MpjcugJRwXLgbBm9XNRIBTkFryQSJ8Sps BFlKWcTxt8HcCaXgGfQJWMMOx4vZ52Q= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JnID1DBm; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of vivek.kasireddy@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702368095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=h3emmYlObQz4YBWvxea89j9SxvY/uMNOIgA3BV8wvNU=; b=0D7+uQ+y6mXjseZODS65L7n51agJB38x37Dr7LteU2FCKpMWTxdvKh+NX25kRF8BKUqkzS 0gmfWy2iSQXZeEvcF+MwBknO1sRrFOR4LfG6C8UjNmydVf2i+PxOm2GxuuufCu6OdSAV8u yrqynm4yKg8oA2KlIW4KIlVK7wLTXV0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702368095; x=1733904095; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r39C9fURv/tYjjTgfRvVfOiacLkGBXJF65fG2x4l+b0=; b=JnID1DBmQ4tO/0pwvPmKXRv7PqMm1MJfmKvkfJcWiBA23f0aIzyLsZJq KwwnIlzYfwzcGPozT3wDoXwKQLACk9lqcl3dkMt6OpsIdu+psLG9lLvHF KnqSaLBJ38+8aH3hW+njWMcLeh1HTbI/nbVu+wOCbBU9+m4r9EwF5/GNe jiVzRmQRlSVNXiNXrJbFJax458hXXUHP7BLcrZ+9t4e45EYbVc8dHxqFm 2Q8G5u7VaG8N6WXF5JeMiE4F9EfZWy4d0yLKMwQQxewq/N5OxtnVsz+GC vM34ySPY1ViyQop5TvYXvIMRQ9CeMLdGyLB6hWiytizktM2+B5eCYxeOy g==; X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="397553996" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="397553996" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10921"; a="802376338" X-IronPort-AV: E=Sophos;i="6.04,269,1695711600"; d="scan'208";a="802376338" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 00:01:26 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , Shuah Khan , David Hildenbrand , Daniel Vetter , Mike Kravetz , Hugh Dickins , Peter Xu , Jason Gunthorpe , Gerd Hoffmann , Dongwon Kim , Junxiao Chang Subject: [PATCH v7 6/6] selftests/dma-buf/udmabuf: Add tests to verify data after page migration Date: Mon, 11 Dec 2023 23:38:03 -0800 Message-Id: <20231212073803.3233055-7-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231212073803.3233055-1-vivek.kasireddy@intel.com> References: <20231212073803.3233055-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 43D5A160019 X-Stat-Signature: pm4kpj1kjjrpbsz5437pniar5mu63faw X-HE-Tag: 1702368095-698139 X-HE-Meta: U2FsdGVkX19jkU/EN4epEokSAYy6hXjzvfYDEX9sc+9ET5HB+eoRDpDyIUJRIgtLyCjBCW4o2WY06d6EVVVhu3VtMaeOdzCOy/Vii3sU7InA2vZUPCkZNCLI9/n/D4gOl8ZOzVmbSBddd305tqpvwX//wgu4yFpSMwS5SitiM1mtB63cGc4xhUJKbouX9RFDSiEQMruR3aOz6E5gStuIOc1GpPpgeMgdW2bG1p/J7rre99iqFbcLwddRA6E+xYxcU60eRCwugIqRfJ+HKrE0lEF1xBOIRjTSvHW6viY4gy3jq1Pky0tBur86/gWshZ/Qse5FVeL+BovhXrATkEpe9GATpGa2lRidtdw9pEp8FzoNCU2KMaoX/uh4ccM8Qp9eNgnj/GguYffTcEDtZWx2A69cV4EnLO20BUYMSUE/1qManLBJt22QEtdn5jIqwpnCU81HpvGn399e5EpdvrUh1TWxz6yUDXxrXoj0NZmGaMr3eHiTMuF7bWEHEr3FeSUzdious+ttX6elxzZZffdSSKiAc0G2Mycylx/HWYOmKvdB+MmrFodoe7y+AfHF9rZnMsQg147tIPTybGN//FC18f6eNJi00kI+WYctpVo1yAV/XT50A2hyrvPDdmkBMAuJmf5sAVpGkAwjFg+XEMnML2mzHd6a6f4lTCdcmyfYSRcd2l8Y4sUujCq6rgfrNo6JO6TusedJ8pgaIJirUtWrRdPuj4MrCILoneqVUWtDmjBXS2cl7Au4BjevcC8KG9N6ZcxxG88n6jqUIBzN0/C/i4B+k3FLqoLPzcv+gIQh7cmKGH/HY1NRq3zYtykFyF9j2xL4j/k3TtnwqeAeUtoGz705+43tBR27jTFIBlOFG7qkk0OVJ/bqUZ874WHfXOH0j57bvvaouQo19/Glyv0HZMT/yPQrpdwvVOIgM56lL/4AEc2tMj6aBBMo/zaDrXBxMBX3KIEYJ84V2pd0Vu+ i0WOCL6o bra9cWT0ASggrzJkKKulndux7A8CVOhjEBOiQzxAMRbAVVvbvxDIs4E2LDGRcqtEwk5HspDkQwQIknQMPBXr2dWQGrBGYNjDD1JNcDNxz+LnnuPvtMz1Czu7xLFYfI0m+v+PmS1EMNNiuBJ6yXy0GPy9EWX7N2w5TQ4UxzgXddmrhhQZbPIGtC+r5mdXeWiVd3SFNRWuq8YsEAOkG1HGHhI88FWxpD43oJso2WbLHbwgFH6nteYIUpzDdEVgW99nmuWUvpwTLi/rMNrK3AmRQ6q4zriK3st98F5OG2kYvJdAq/sHCwudFx8EHFQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since the memfd pages associated with a udmabuf may be migrated as part of udmabuf create, we need to verify the data coherency after successful migration. The new tests added in this patch try to do just that using 4k sized pages and also 2 MB sized huge pages for the memfd. Successful completion of the tests would mean that there is no disconnect between the memfd pages and the ones associated with a udmabuf. And, these tests can also be augmented in the future to test newer udmabuf features (such as handling memfd hole punch). Cc: Shuah Khan Cc: David Hildenbrand Cc: Daniel Vetter Cc: Mike Kravetz Cc: Hugh Dickins Cc: Peter Xu Cc: Jason Gunthorpe Cc: Gerd Hoffmann Cc: Dongwon Kim Cc: Junxiao Chang Based-on-patch-by: Mike Kravetz Signed-off-by: Vivek Kasireddy --- .../selftests/drivers/dma-buf/udmabuf.c | 151 +++++++++++++++++- 1 file changed, 147 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/drivers/dma-buf/udmabuf.c b/tools/testing/selftests/drivers/dma-buf/udmabuf.c index c812080e304e..d76c813fe652 100644 --- a/tools/testing/selftests/drivers/dma-buf/udmabuf.c +++ b/tools/testing/selftests/drivers/dma-buf/udmabuf.c @@ -9,26 +9,132 @@ #include #include #include +#include #include #include +#include #include #include #define TEST_PREFIX "drivers/dma-buf/udmabuf" #define NUM_PAGES 4 +#define NUM_ENTRIES 4 +#define MEMFD_SIZE 1024 /* in pages */ -static int memfd_create(const char *name, unsigned int flags) +static unsigned int page_size; + +static int create_memfd_with_seals(off64_t size, bool hpage) +{ + int memfd, ret; + unsigned int flags = MFD_ALLOW_SEALING; + + if (hpage) + flags |= MFD_HUGETLB; + + memfd = memfd_create("udmabuf-test", flags); + if (memfd < 0) { + printf("%s: [skip,no-memfd]\n", TEST_PREFIX); + exit(77); + } + + ret = fcntl(memfd, F_ADD_SEALS, F_SEAL_SHRINK); + if (ret < 0) { + printf("%s: [skip,fcntl-add-seals]\n", TEST_PREFIX); + exit(77); + } + + ret = ftruncate(memfd, size); + if (ret == -1) { + printf("%s: [FAIL,memfd-truncate]\n", TEST_PREFIX); + exit(1); + } + + return memfd; +} + +static int create_udmabuf_list(int devfd, int memfd, off64_t memfd_size) +{ + struct udmabuf_create_list *list; + int ubuf_fd, i; + + list = malloc(sizeof(struct udmabuf_create_list) + + sizeof(struct udmabuf_create_item) * NUM_ENTRIES); + if (!list) { + printf("%s: [FAIL, udmabuf-malloc]\n", TEST_PREFIX); + exit(1); + } + + for (i = 0; i < NUM_ENTRIES; i++) { + list->list[i].memfd = memfd; + list->list[i].offset = i * (memfd_size / NUM_ENTRIES); + list->list[i].size = getpagesize() * NUM_PAGES; + } + + list->count = NUM_ENTRIES; + list->flags = UDMABUF_FLAGS_CLOEXEC; + ubuf_fd = ioctl(devfd, UDMABUF_CREATE_LIST, list); + free(list); + if (ubuf_fd < 0) { + printf("%s: [FAIL, udmabuf-create]\n", TEST_PREFIX); + exit(1); + } + + return ubuf_fd; +} + +static void write_to_memfd(void *addr, off64_t size, char chr) +{ + int i; + + for (i = 0; i < size / page_size; i++) { + *((char *)addr + (i * page_size)) = chr; + } +} + +static void *mmap_fd(int fd, off64_t size) { - return syscall(__NR_memfd_create, name, flags); + void *addr; + + addr = mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); + if (addr == MAP_FAILED) { + printf("%s: ubuf_fd mmap fail\n", TEST_PREFIX); + exit(1); + } + + return addr; +} + +static int compare_chunks(void *addr1, void *addr2, off64_t memfd_size) +{ + off64_t off; + int i = 0, j, k = 0, ret = 0; + char char1, char2; + + while (i < NUM_ENTRIES) { + off = i * (memfd_size / NUM_ENTRIES); + for (j = 0; j < NUM_PAGES; j++, k++) { + char1 = *((char *)addr1 + off + (j * getpagesize())); + char2 = *((char *)addr2 + (k * getpagesize())); + if (char1 != char2) { + ret = -1; + goto err; + } + } + i++; + } +err: + munmap(addr1, memfd_size); + munmap(addr2, NUM_ENTRIES * NUM_PAGES * getpagesize()); + return ret; } int main(int argc, char *argv[]) { struct udmabuf_create create; int devfd, memfd, buf, ret; - off_t size; - void *mem; + off64_t size; + void *addr1, *addr2; devfd = open("/dev/udmabuf", O_RDWR); if (devfd < 0) { @@ -90,6 +196,9 @@ int main(int argc, char *argv[]) } /* should work */ + page_size = getpagesize(); + addr1 = mmap_fd(memfd, size); + write_to_memfd(addr1, size, 'a'); create.memfd = memfd; create.offset = 0; create.size = size; @@ -98,6 +207,40 @@ int main(int argc, char *argv[]) printf("%s: [FAIL,test-4]\n", TEST_PREFIX); exit(1); } + munmap(addr1, size); + close(buf); + close(memfd); + + /* should work (migration of 4k size pages)*/ + size = MEMFD_SIZE * page_size; + memfd = create_memfd_with_seals(size, false); + addr1 = mmap_fd(memfd, size); + write_to_memfd(addr1, size, 'a'); + buf = create_udmabuf_list(devfd, memfd, size); + addr2 = mmap_fd(buf, NUM_PAGES * NUM_ENTRIES * getpagesize()); + write_to_memfd(addr1, size, 'b'); + ret = compare_chunks(addr1, addr2, size); + if (ret < 0) { + printf("%s: [FAIL,test-5]\n", TEST_PREFIX); + exit(1); + } + close(buf); + close(memfd); + + /* should work (migration of 2MB size huge pages)*/ + page_size = getpagesize() * 512; /* 2 MB */ + size = MEMFD_SIZE * page_size; + memfd = create_memfd_with_seals(size, true); + addr1 = mmap_fd(memfd, size); + write_to_memfd(addr1, size, 'a'); + buf = create_udmabuf_list(devfd, memfd, size); + addr2 = mmap_fd(buf, NUM_PAGES * NUM_ENTRIES * getpagesize()); + write_to_memfd(addr1, size, 'b'); + ret = compare_chunks(addr1, addr2, size); + if (ret < 0) { + printf("%s: [FAIL,test-6]\n", TEST_PREFIX); + exit(1); + } fprintf(stderr, "%s: ok\n", TEST_PREFIX); close(buf);