From patchwork Wed Jun 3 02:42:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Song Bao Hua (Barry Song)" X-Patchwork-Id: 11584909 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADD33618 for ; Wed, 3 Jun 2020 02:44:38 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 784082086A for ; Wed, 3 Jun 2020 02:44:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="um2bYcBF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 784082086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=MBUwdg8n4tv/moEHul6d2xSZFzaFlsWQXBSDeqiMdpo=; b=um2bYcBF+oxh88 cER259PQZ6Sfkyrfgbc42ShGAhQumNV50QmQrKbUfWIKiLtzCcMoVeeRAZgqueUEYog1SI7ANasRj UQ1ANewPtorW20gd0xSsmNQiIiVSuSfOpccE6E7I07lw1qA3DHi9FVMYSI9ImC+2Ltsv1E+GsC/N/ sE29VLJ8UA7Ihae+99+2OVisCLNN7/ru3x5rDWz8EAU0AlQous6caix3jH13vVwuMLCLW/j0igb2I jNnWjPrh055VYIufrzoSOlFvDyPMU2EOnjQPVgPMNx0Plwa2s5PsDMUsGC1kcUJ2ZMBTqJjxslglT 1Nm++v80Hb7vJy4mcRJA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgJOJ-00012n-2A; Wed, 03 Jun 2020 02:44:31 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jgJOG-00010R-45 for linux-arm-kernel@lists.infradead.org; Wed, 03 Jun 2020 02:44:29 +0000 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 9B98898D0D4831E0C2FB; Wed, 3 Jun 2020 10:44:13 +0800 (CST) Received: from SWX921481.china.huawei.com (10.126.201.193) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.487.0; Wed, 3 Jun 2020 10:44:04 +0800 From: Barry Song To: , , , Subject: [PATCH 0/3] support per-numa CMA for ARM server Date: Wed, 3 Jun 2020 14:42:28 +1200 Message-ID: <20200603024231.61748-1-song.bao.hua@hisilicon.com> X-Mailer: git-send-email 2.21.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.126.201.193] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200602_194428_328641_FD89FF79 X-CRM114-Status: UNSURE ( 7.63 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [45.249.212.191 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [45.249.212.191 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 AC_FROM_MANY_DOTS Multiple periods in From user name X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Barry Song , john.garry@huawei.com, linux-kernel@vger.kernel.org, linuxarm@huawei.com, iommu@lists.linux-foundation.org, prime.zeng@hisilicon.com, Jonathan.Cameron@huawei.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Right now, smmu is using dma_alloc_coherent() to get memory to save queues and tables. Typically, on ARM64 server, there is a default CMA located at node0, which could be far away from node2, node3 etc. Saving queues and tables remotely will increase the latency of ARM SMMU significantly. For example, when SMMU is at node2 and the default global CMA is at node0, after sending a CMD_SYNC in an empty command queue, we have to wait more than 550ns for the completion of the command CMD_SYNC. However, if we save them locally, we only need to wait for 240ns. with per-numa CMA, smmu will get memory from local numa node to save command queues and page tables. that means dma_unmap latency will be shrunk much. Meanwhile, when iommu.passthrough is on, device drivers which call dma_ alloc_coherent() will also get local memory and avoid the travel between numa nodes. Barry Song (3): dma-direct: provide the ability to reserve per-numa CMA arm64: mm: reserve hugetlb CMA after numa_init arm64: mm: reserve per-numa CMA after numa_init arch/arm64/mm/init.c | 12 ++++++---- include/linux/dma-contiguous.h | 4 ++++ kernel/dma/Kconfig | 10 ++++++++ kernel/dma/contiguous.c | 43 +++++++++++++++++++++++++++++++++- 4 files changed, 63 insertions(+), 6 deletions(-)