From patchwork Thu Aug 15 15:11:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13764903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35469C52D7C for ; Thu, 15 Aug 2024 15:12:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B80206B0132; Thu, 15 Aug 2024 11:12:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B2C476B0133; Thu, 15 Aug 2024 11:12:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 959B76B0134; Thu, 15 Aug 2024 11:12:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6DE606B0132 for ; Thu, 15 Aug 2024 11:12:14 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1BEB1120118 for ; Thu, 15 Aug 2024 15:12:14 +0000 (UTC) X-FDA: 82454820588.05.136F063 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2065.outbound.protection.outlook.com [40.107.94.65]) by imf21.hostedemail.com (Postfix) with ESMTP id 40B541C0010 for ; Thu, 15 Aug 2024 15:12:11 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=OUeZlj+Y; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf21.hostedemail.com: domain of jgg@nvidia.com designates 40.107.94.65 as permitted sender) smtp.mailfrom=jgg@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1723734677; a=rsa-sha256; cv=pass; b=gro0A4d/+SUtvJjD+g80ctI78Z//Q6FThc8RAAlfBAGUNrhHgIGnxVIbXLHvxNh+Qt9jHz Z+WA4HF7F3n8TCVpHwe+8X8wDkBLqql2okC+FjbX87syBTb/+2Dmc87poKNM0uU2ZqJFe2 m+kK+eoM08HIvfAJzE8/gET84cToFVg= ARC-Authentication-Results: i=2; imf21.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=OUeZlj+Y; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf21.hostedemail.com: domain of jgg@nvidia.com designates 40.107.94.65 as permitted sender) smtp.mailfrom=jgg@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723734677; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CseehZW1/tl+23sWMKn8i2i+3hNkbe8IlewYv7NWFro=; b=pGHb9pp3z7A/nQhP19rCM+l1sd708vbqudMjWhai6afzpSj1KKWrDxDElYL1WDKQODIAOf dCeaJ4pry6B4Z8xXHhNRKc96Y4BcEYy6k6HndBKMvUSQ7hZY2TOw09xVFndAQYy4PvYrXU HKNeUVuOKa70dh16QoqIxOdwmUE01Gc= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NDyvsAXFE+UImNepmrP2uZukyuK8T7itDLChP1mRMkcrbQ43lwz6TA+CrRlOfnPXwpAvhQQqT6l53dMSPH2Tq8dtedacFYR796g0bxfMmkeud0OrkFjrkYjb9oxdheCxpk1xQWgqWD6lWsfrI+oRY8F39/P8F/92S/LxVU8zmxgzo3zmspXkFsZnVTxra0JQsmPntSOzrfC9+gt334tPV4RJyfbQiudIuZcUom4ityT+ZsR/avPLTkKRHJm7nqyXKeB+vgC1lj+IXIj0OIEZ6D1Kkxvr/+NDJD7IaXsuh0HLNv8TOoUY18JpKOdJanROUWQGjLW1xgxkZWmGQClUXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CseehZW1/tl+23sWMKn8i2i+3hNkbe8IlewYv7NWFro=; b=wVdEfnpVXAs2f/XURwAs32xU8DOdkpmf/AQ7wfL4aOkZ3tSECWgn6Tkf2bIryCVkUKGUKtFT9RZU+DtpWfvfeW1VVgEADjGviqsOtzlXDA91N5YI2HyVvFGh/RD+LRJ8vwjMuyo+I6MSXU94d3527amhNjweu/dEKJZe9kYUd+6BfxzGhGIeVqrnL5112hxzj1jdlPEmrKXeEYhWaTtAm1OidPWSY+SnrlwbVBm4jb8UsEuVqhf638SR7WzmNBVLUPOVKqFglqhQfvNQAsk1dl6uqxrCQ0lHau90o9ulJnkr1QbWTm/IF495nJ2/OjCw9PlpuIjLCDitDJs4qMVjvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CseehZW1/tl+23sWMKn8i2i+3hNkbe8IlewYv7NWFro=; b=OUeZlj+YG12xvlNOQivw+1b81M1wuRxcqkvkc96QPg3kEas8yvqOc/uWPR2GeY5Tj7QqzRBFV7CecbE7NLUvtDG8erE+CfJhuLhFSgWhexByURzAindzUKB64IbONj+4p2hVlQyOtyruT57Idgiw7ajG28Kf0yph1QW4e2KPxTzplVABcwXv27Hk15qF6nZQCUCPCtVhJNQHd9oUx+nQZLZEAm9S8QlooK6IS8Xr5ZQCVtNGlo2JNu47q/cIkv7Mz/7usp8M9CBnm7R5bcJcCanwZLxEbugJiCGOPihSHQhvZkT1M6xPBlh5YNjcu/jz/xxQ5dC4W62TLTy9XNLIdg== Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by SN7PR12MB8146.namprd12.prod.outlook.com (2603:10b6:806:323::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.17; Thu, 15 Aug 2024 15:11:43 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.016; Thu, 15 Aug 2024 15:11:43 +0000 From: Jason Gunthorpe To: Cc: Alejandro Jimenez , Lu Baolu , David Hildenbrand , Christoph Hellwig , iommu@lists.linux.dev, Joao Martins , Kevin Tian , kvm@vger.kernel.org, linux-mm@kvack.org, Pasha Tatashin , Peter Xu , Ryan Roberts , Sean Christopherson , Tina Zhang Subject: [PATCH 03/16] iommupt: Add the basic structure of the iommu implementation Date: Thu, 15 Aug 2024 12:11:19 -0300 Message-ID: <3-v1-01fa10580981+1d-iommu_pt_jgg@nvidia.com> In-Reply-To: <0-v1-01fa10580981+1d-iommu_pt_jgg@nvidia.com> References: X-ClientProxiedBy: BL6PEPF00016416.NAMP222.PROD.OUTLOOK.COM (2603:10b6:22e:400:0:1004:0:4) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|SN7PR12MB8146:EE_ X-MS-Office365-Filtering-Correlation-Id: 30051ef4-32fb-4d1a-e569-08dcbd3c8ea3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: qXM0zrAo/03YIptDigjFnSwsxV//MoZCf3FiSjAHFUqxcpkjBBpkERLmf78bXJrtGX2HW2N9G3jK0eseHnVjYLqsc6DkpGrFz9/etxrM1JV4lezvBChDcX6q8uGW2i3lOlxxKYZBGqN6IW354vV0xf3b7/bkrFZtFpMdCsfnnmcokeSWRU0TFDMLFwFoUE0+rKFhi9SbjkDH49vwiuB8d1eHnLZu9e3z+oGlX//BukB/v7ILhDY7qIgWwZ2MX2AJHdREwran7+JPVrimSX2iXOidoz4xWkIss9SAz61q88JpeCpwCiChlzbsR9KCVeIIbyCbjxEyTqYGEPGybxW/qHWdKwhCkZwx3x0NP3VJnt8iGRXK6hr7fjjIZtn3wL5YH292rQlmiI7h5PmtmnOaKDzxpI270+ANwsEwaHZj/clroxpzz1OQxytcvHwbVIp80BfEXu+7ZedQGt1+ofDZlYK5H4vHEL7mnZw4ViqvSFGsHlAjtecf5a1tlBcoNrnUvEjcyzf0ez5k4xXWErvPQdnYR6AQxY8kG+p1PBARJuZ5WQWF6ASeJsUueKwF4YNcPbvfnl0MQpNxJtzPEAjMxGDrpge0owT1Cojh+dpUFkqh/mBcJjtSLkKoAJ1v65sVt1O9H+UhEsodCNNWxaemZnC1gh3BT5FKYs1nXk3WD70aiuZSW+GKGhLAqdZDJh0qFDiBLl0gf32cbJLGrtzVeTVdbp1VP835jhpdWr23dgR1949xI+I0NMQcS8nAPEF8fGToV8UaO4QPmVeiepugjQaEH2A08BjmAvnttKQDj3jSy8Mytg/Weqhi4+X+LAX7H2prJVWUokqzcZUKlLQTChir+fUvTOJlSi1QBxSFHmGQQoZ07zikjRBgf5CG1BdIbShZ6b0WhimQ9vzCZ8LPVrXlzhWcw/pI6YYQgsg8ou4BwquGt7jwrWyyb8SiuhOEwJ2QOSixkGOs82WjQRQoSVsT3fdRDS28QwUDAqPBBS8y5YLqLuHwBa+sBjUKqUUjPGwsEwXEa5WnnaU9r0Q1f/dqO+DMPYEQN8hGbbgLKCayET0KvEiEqbNVi5qSHcOFg1Cl7LqPGiXU9xk6jjtkOObbZfOJ0dLFg+nQR0qz/QrT0s8oSq+pQY/e7P/RCRic2r1CLCg4dq8LtHw1LIVO9qg3dc003Sp3HNsC/lZQL5vZcTvBVcfZwamqkzjaQxwVZlCohcVqutDo7RL4IBE15B7ScmCHeVXN9iLo0hLbn6vgvbhNmitEKRsZH/oR38xloBUjznF6gMgqI1x5oz3ChyE8/GNWJvutr5izVsqsB8ng3ZpCDMxjNRlGdydFp617AugGT0dd842sPbHL1uzTdg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8Cqrdyrva3gB/cdyHOmmvSrTNKLVV/3Lw/zM7EwP0lFgx+0/yyPMZ9xd1i7dkaxxRVGG30ryloEJg7fbzI/g41GsMsYqbfqnVakDpZpQnJW+c7wDdJZqMld1D1gySyHgSQQ4A4Y31GN6Ck/2Z7FSHE7tUrKdSX4QP8kVKVlGMsjAD6Qb3Izesi17ZoFGgPpX8OgsITO2P/sHQdud0leT2CV+0w5BCjipnsHmBgrzL/ni0uT49E9uur+hVMPPeSpkBE0UlGBVdd3oPkNktflyox5JyGoXPcmL1lRvvqDvRMshaIUXMPDjnAf+AQogbLHH2fgj3Q3oIOJINyaoe/KFGPcOIJOWU2HHzIYHrfv1I2STwjb6TMLunFePX1/IEtjUX+gB+1u5HhJmhU0qQK3lhVSn+calJ+hiX7/ty4YTKi11MyFCFlG/h7ikj9YtsEa0q53mT08bXuTSArDclNJiX02E35jU+KtGgTdjR2DLlaVfM22o/IA43L20oGwYB2HRp4aRmSMa4Nbg3SrY2bWHjQISHI5UZMZWxAAHVxuxkFtt5tvlm/lKmi+isYHkexGuuyn8oUhSg0ram3UNRikPUFbLCABv8I51dl1NpphF6n4doTiYTeJehrTxjSjSutNjkwOuF70VXVSqSJ+PEO4+IoPCOBMUAi7a94DkuaUANfbAI6C/xtyJf4vvfpEPzNYzBPnfHmqn7lW/iOpcp4Zfnekj0I7oQStLNvZ9G7M4ciZTUuv6qTAKj5oJVJKINngPAjpSuEdaHtTH6XfkC3pgHNQ/6vlp+crVRUPY9wd5G/5fPR6fJo6MqqPZTUgtKdR/rlleSRmT4yeedztEeCdOLIldQ87k2LgY2B3KhFECP7tUXQfR4ESQFllJlM9cu8xcu8OMcfglBltZ6tLNoXNvfUW/YkV8lhr4Z+vy1o5FpE2MdeYMRxINlG1KwuodwspyVnyLHcz7Zw86jkM+u51iU4xU6S1S3mLjlP+LF5QyeE5Wubr8AbaKwSbUNEZS/akCHeGzYZgoEJv5W6vgdFN5HtSmt4gfKy8L5lIzCfQ+Vb5a0EdPYAlM7WM+dIldidyINi8d93itpDYUYQAcgXtbT19vAUuExXaZHJDhA8g7oY6mNbUVzqyidmKY7kZWkKcK9RsaKqH2/pltMNUVpBM2CFgjwfSN/0/bzzmSN56FE0U3BPWkfln+53yPRYMX+yYoqMgQxUfXu3EDaWNZIHeip9IlMl4YqPU9HtFpLbTcynt6UAQCB/gpsAODO677SStEpOdWuYUJ1neOGFU4OSmaXmB9o51v+SYjGQczV32S9LAMMN4IVWv73OWnl2X+kKKEkt8w4WptYM36GyvjnBN8eQg9+c9qC6PPD4ru45w8+h65vY0wKe8UpwjplhV31PP/sF3OCbP278JsjM5LuHlo0k1xxiz/mmPMag7GSzN+LkXUJ7iy0eLSEJOm4laa5It6AIBcn0KVJ7YnFiirom7PbihBOAwf8e8YXNQ85FKUPy0ELLxrJX91PshEWYJv3vA7Tr7U0NmZwnGcYdSbo0AaiPvzE5UfbXqMK1x33eRafdc= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30051ef4-32fb-4d1a-e569-08dcbd3c8ea3 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 15:11:37.0548 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: corMjB76D/Ts3sHjDqGOF8ddwwZujGF4zS5cAVLl7JwYZrNLAukH0iFydcT2OxIz X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8146 X-Rspamd-Queue-Id: 40B541C0010 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7eds59k5uid53z1bzn9dj38hfqbaazip X-HE-Tag: 1723734731-488318 X-HE-Meta: U2FsdGVkX19RuTfuzDh17egSuODbtDaOs4jfkCSRBMKac/TsvOXbuXRL979G7amnJa3CiuOqJsT8RHW8tAcXf1fxVhGV6SZvyhz1sKUvnqN4gKQVg44oyeiyI5yhxx1fI0a5SfjNJSEfpRq8s3vwWFp0cB1XpIygCC9UT66Vn7vWCmwZLvotz4IrkUyFtH8CluuRBoNCuoBQuSCRHyUXdYVTkgb2omz4jFzIrShkVHcJBCJcE8i4dvwpaTObvWUzQKI24Q7UKIFaK56kkOhZ3LMvHj/2k4vf8+K77fwy2suECzcmqNQYWFGYX1ffBt2JzMFZ72Oc7GkcyF6/l5C6qOCV7Dh7wKx/EaNXpH7KPLX38rNATjcX7nhk8/JmvW8dEv+2RpyXnvBLWGyb6lWdRcoE+iZf5AzkIbzOk1tH0koc0maQ5rzOrEvT5GMRjzXstxNrJ55uD0uMFZ8Jcljh9OKOC2RyaYW2+CqWF5JEFV9ujbCgZiTJAbrjgKFIIERvI9+aTW7ipYrMcLrSSJ00KUdezBzKrqFau/NS4HR7T9ggthaZqUMc789ss0KDaAsPGwicA8hQ4/HjcGddfVn1CZPebspPFpzejAX68ZdO4MVVI9iPpunoTaB5x+KpvuJOnb5noFnJTZGhnnOV8KZOaKfRZ3I80B7eNvgOP1iiFh/Sh5cy+pk5hLNgiPcaOVBKNn1gYl7DowMPgA925V15WLHdX/2rJg/ovagz2ho4sbPYfI+We11n6LQhrH/coi0dxdSfknCHDMqODhSvq7LRyKgyj04yR/9lgZjTSOr48seBFCfS1a+IqoRqoEcRfIh4c/IyJkfGVZsCH7gql5JXyF3/O0b5fF4KI5zNIu9ltkwI/A2ysDIHh7yuoUpShiCcozj1+emVEg/0H9QeAoTwicxwnCamMkom6HSa8+Ncxb7wVCBy55gIhvPuS+eHNHPYxsG49htQODTHhmQk/s7 RF1HrVcn 2Phfpw/Grm1b6I5orBhyMcIrvLCrz54KjETdi3xvuSXIUdHcTpvpQp7KjX3zKV1wMGEMilk6Y/mwSbMnN3MJlsEyffiJ2KOavC2a9sk8UgJUWYsblu8nMKv2v8rxw0Wx72Bzd2B3WJJYaNupWe7Uh93IdyJw5qzTDkxaeYPL/Cobbb42+1gjULkXVcGgWcdilZW+He1J4oD3ZL1/pULulXqTIXzh6MW/dffTbTr0nIVGQLSzmVPtDH8mKKF3TkwujOhEA+sQjns+H58tsLVDXrVWW/qkxrD/OJHrsObnK6MbJ1UYM1sjReQ2RYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The iommu implementation is a single version of the iommu domain operations, iova_to_phys, map, unmap, read_and_clear_dirty and flushing. It is intended to be a near drop in replacement for existing iopt users. By using the Generic Page Table mechanism it is a single algorithmic implementation that operates all the different page table formats with consistent characteristics. Implement the basic starting point: alloc(), get_info() and deinit(). Signed-off-by: Jason Gunthorpe --- drivers/iommu/generic_pt/fmt/iommu_template.h | 37 ++++ drivers/iommu/generic_pt/iommu_pt.h | 166 ++++++++++++++++++ include/linux/generic_pt/iommu.h | 87 +++++++++ 3 files changed, 290 insertions(+) create mode 100644 drivers/iommu/generic_pt/fmt/iommu_template.h create mode 100644 drivers/iommu/generic_pt/iommu_pt.h create mode 100644 include/linux/generic_pt/iommu.h diff --git a/drivers/iommu/generic_pt/fmt/iommu_template.h b/drivers/iommu/generic_pt/fmt/iommu_template.h new file mode 100644 index 00000000000000..d6ca1582e11ca4 --- /dev/null +++ b/drivers/iommu/generic_pt/fmt/iommu_template.h @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + * + * Template to build the iommu module and kunit from the format and + * implementation headers. + * + * The format should have: + * #define PT_FMT + * #define PT_SUPPORTED_FEATURES (BIT(PT_FEAT_xx) | BIT(PT_FEAT_yy)) + * And optionally: + * #define PT_FORCE_ENABLED_FEATURES .. + * #define PT_FMT_VARIANT + */ +#include +#include + +#ifdef PT_FMT_VARIANT +#define PTPFX \ + CONCATENATE(CONCATENATE(PT_FMT, _), CONCATENATE(PT_FMT_VARIANT, _)) +#else +#define PTPFX CONCATENATE(PT_FMT, _) +#endif + +#define _PT_FMT_H PT_FMT.h +#define PT_FMT_H __stringify(_PT_FMT_H) + +#define _PT_DEFS_H CONCATENATE(defs_, _PT_FMT_H) +#define PT_DEFS_H __stringify(_PT_DEFS_H) + +#include +#include PT_DEFS_H +#include "../pt_defs.h" +#include PT_FMT_H +#include "../pt_common.h" + +#include "../iommu_pt.h" diff --git a/drivers/iommu/generic_pt/iommu_pt.h b/drivers/iommu/generic_pt/iommu_pt.h new file mode 100644 index 00000000000000..708beaf5d812f7 --- /dev/null +++ b/drivers/iommu/generic_pt/iommu_pt.h @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + * + * "Templated C code" for implementing the iommu operations for page tables. + * This is compiled multiple times, over all the page table formats to pick up + * the per-format definitions. + */ +#ifndef __GENERIC_PT_IOMMU_PT_H +#define __GENERIC_PT_IOMMU_PT_H + +#include "pt_iter.h" +#include "pt_alloc.h" + +#include +#include + +struct pt_iommu_collect_args { + struct pt_radix_list_head free_list; + u8 ignore_mapped : 1; +}; + +static int __collect_tables(struct pt_range *range, void *arg, + unsigned int level, struct pt_table_p *table) +{ + struct pt_state pts = pt_init(range, level, table); + struct pt_iommu_collect_args *collect = arg; + int ret; + + if (collect->ignore_mapped && !pt_can_have_table(&pts)) + return 0; + + for_each_pt_level_item(&pts) { + if (pts.type == PT_ENTRY_TABLE) { + pt_radix_add_list(&collect->free_list, pts.table_lower); + ret = pt_descend(&pts, arg, __collect_tables); + if (ret) + return ret; + continue; + } + if (pts.type == PT_ENTRY_OA && !collect->ignore_mapped) + return -EADDRINUSE; + } + return 0; +} + +static void NS(get_info)(struct pt_iommu *iommu_table, + struct pt_iommu_info *info) +{ + struct pt_common *common = common_from_iommu(iommu_table); + struct pt_range range = pt_top_range(common); + struct pt_state pts = pt_init_top(&range); + pt_vaddr_t pgsize_bitmap = 0; + + if (pt_feature(common, PT_FEAT_DYNAMIC_TOP)) { + for (pts.level = 0; pts.level <= PT_MAX_TOP_LEVEL; + pts.level++) { + if (pt_table_item_lg2sz(&pts) >= common->max_vasz_lg2) + break; + pgsize_bitmap |= pt_possible_sizes(&pts); + } + } else { + for (pts.level = 0; pts.level <= range.top_level; pts.level++) + pgsize_bitmap |= pt_possible_sizes(&pts); + } + + /* Hide page sizes larger than the maximum OA */ + info->pgsize_bitmap = oalog2_mod(pgsize_bitmap, common->max_oasz_lg2); +} + +static void NS(deinit)(struct pt_iommu *iommu_table) +{ + struct pt_common *common = common_from_iommu(iommu_table); + struct pt_range range = pt_top_range(common); + struct pt_iommu_collect_args collect = { + .ignore_mapped = true, + }; + + pt_radix_add_list(&collect.free_list, range.top_table); + pt_walk_range(&range, __collect_tables, &collect); + if (pt_feature(common, PT_FEAT_DMA_INCOHERENT)) + pt_radix_stop_incoherent_list(&collect.free_list, + iommu_table->iommu_device); + pt_radix_free_list(&collect.free_list); +} + +static const struct pt_iommu_ops NS(ops) = { + .iova_to_phys = NS(iova_to_phys), + .get_info = NS(get_info), + .deinit = NS(deinit), +}; + +static int pt_init_common(struct pt_common *common) +{ + struct pt_range top_range = pt_top_range(common); + + if (PT_WARN_ON(top_range.top_level > PT_MAX_TOP_LEVEL)) + return -EINVAL; + + if (top_range.top_level == PT_MAX_TOP_LEVEL || + common->max_vasz_lg2 == top_range.max_vasz_lg2) + common->features &= ~BIT(PT_FEAT_DYNAMIC_TOP); + + if (!pt_feature(common, PT_FEAT_DYNAMIC_TOP)) + common->max_vasz_lg2 = top_range.max_vasz_lg2; + + if (top_range.max_vasz_lg2 == PT_VADDR_MAX_LG2) + common->features |= BIT(PT_FEAT_FULL_VA); + + /* Requested features must match features compiled into this format */ + if ((common->features & ~(unsigned int)PT_SUPPORTED_FEATURES) || + (common->features & PT_FORCE_ENABLED_FEATURES) != + PT_FORCE_ENABLED_FEATURES) + return -EOPNOTSUPP; + + /* FIXME generalize the oa/va maximums from HW better in the cfg */ + if (common->max_oasz_lg2 == 0) + common->max_oasz_lg2 = pt_max_output_address_lg2(common); + else + common->max_oasz_lg2 = min(common->max_oasz_lg2, + pt_max_output_address_lg2(common)); + return 0; +} + +#define pt_iommu_table_cfg CONCATENATE(pt_iommu_table, _cfg) +#define pt_iommu_init CONCATENATE(CONCATENATE(pt_iommu_, PTPFX), init) +int pt_iommu_init(struct pt_iommu_table *fmt_table, + struct pt_iommu_table_cfg *cfg, gfp_t gfp) +{ + struct pt_iommu *iommu_table = &fmt_table->iommu; + struct pt_common *common = common_from_iommu(iommu_table); + struct pt_table_p *table_mem; + int ret; + + memset(fmt_table, 0, sizeof(*fmt_table)); + spin_lock_init(&iommu_table->table_lock); + common->features = cfg->features; + common->max_vasz_lg2 = PT_MAX_VA_ADDRESS_LG2; + iommu_table->iommu_device = cfg->iommu_device; + iommu_table->nid = dev_to_node(cfg->iommu_device); + + ret = pt_iommu_fmt_init(fmt_table, cfg); + if (ret) + return ret; + + ret = pt_init_common(common); + if (ret) + return ret; + + table_mem = table_alloc_top(common, common->top_of_table, gfp, false); + if (IS_ERR(table_mem)) + return PTR_ERR(table_mem); +#ifdef PT_FIXED_TOP_LEVEL + pt_top_set(common, table_mem, PT_FIXED_TOP_LEVEL); +#else + pt_top_set(common, table_mem, pt_top_get_level(common)); +#endif + iommu_table->ops = &NS(ops); + return 0; +} +EXPORT_SYMBOL_NS_GPL(pt_iommu_init, GENERIC_PT_IOMMU); + +MODULE_LICENSE("GPL"); +MODULE_IMPORT_NS(GENERIC_PT); + +#endif diff --git a/include/linux/generic_pt/iommu.h b/include/linux/generic_pt/iommu.h new file mode 100644 index 00000000000000..d9d3da49dc0fe2 --- /dev/null +++ b/include/linux/generic_pt/iommu.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES + */ +#ifndef __GENERIC_PT_IOMMU_H +#define __GENERIC_PT_IOMMU_H + +#include +#include + +struct pt_iommu_ops; + +/** + * DOC: IOMMU Radix Page Table + * + * The iommu implementation of the Generic Page Table provides an ops struct + * that is useful to go with an iommu_domain to serve the DMA API, IOMMUFD and + * the generic map/unmap interface. + * + * This interface uses a caller provided locking approach. The caller must have + * a VA range lock concept that prevents concurrent threads from calling ops on + * the same VA. Generally the range lock must be at least as large as a single + * map call. + */ + +/** + * struct pt_iommu - Base structure for iommu page tables + * + * The format specific struct will include this as the first member. + */ +struct pt_iommu { + /** + * @ops: Function pointers to access the API + */ + const struct pt_iommu_ops *ops; + /** + * @nid: Node ID to use for table memory allocations. This defaults to + * dev_to_node(iommu_device). The iommu driver may want to set the NID + * to the device's NID, if there are multiple table walkers. + */ + int nid; + /* private: */ + /* Write lock for pt_common top_of_table */ + spinlock_t table_lock; + struct device *iommu_device; +}; + +/** + * struct pt_iommu_info - Details about the iommu page table + * + * Returned from pt_iommu_ops->get_info() + */ +struct pt_iommu_info { + /** + * @pgsize_bitmap: A bitmask where each set bit indicates + * a page size that can be natively stored in the page table. + */ + u64 pgsize_bitmap; +}; + +/* See the function comments in iommu_pt.c for kdocs */ +struct pt_iommu_ops { + /** + * get_info() - Return the pt_iommu_info structure + * @iommu_table: Table to query + * + * Return some basic static information about the page table. + */ + void (*get_info)(struct pt_iommu *iommu_table, + struct pt_iommu_info *info); + + /** + * deinit() - Undo a format specific init operation + * @iommu_table: Table to destroy + * + * Release all of the memory. The caller must have already removed the + * table from all HW access and all caches. + */ + void (*deinit)(struct pt_iommu *iommu_table); +}; + +static inline void pt_iommu_deinit(struct pt_iommu *iommu_table) +{ + iommu_table->ops->deinit(iommu_table); +} + +#endif