From patchwork Fri Nov 22 21:11:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Johannesmeyer X-Patchwork-Id: 13883635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9A37E69195 for ; Fri, 22 Nov 2024 21:11:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16E016B0089; Fri, 22 Nov 2024 16:11:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F6976B008A; Fri, 22 Nov 2024 16:11:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8BED6B008C; Fri, 22 Nov 2024 16:11:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CBCF46B0089 for ; Fri, 22 Nov 2024 16:11:52 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 568F9C198B for ; Fri, 22 Nov 2024 21:11:52 +0000 (UTC) X-FDA: 82814976384.30.895795B Received: from mail-lj1-f180.google.com (mail-lj1-f180.google.com [209.85.208.180]) by imf15.hostedemail.com (Postfix) with ESMTP id 96AF9A0016 for ; Fri, 22 Nov 2024 21:10:54 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HBp+qe4R; spf=pass (imf15.hostedemail.com: domain of bjohannesmeyer@gmail.com designates 209.85.208.180 as permitted sender) smtp.mailfrom=bjohannesmeyer@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732309847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ceOYRg+db2e1ipikZCFVwiN54lElh7G8dmSvY9RkE2I=; b=2zJ2PFCPn9/tY9aKKrn4MyCPHIFrXodTn9Io+sfhoz+bJmcOs5zQcwB7mm8xOIMulrxyNf FKFZ+Tb6QVPu57XcRF5lZeeV32RQc3EQ0CTMU6DaCAuN3MWTXoWHfgI1OjNiwlcxWbbxaT B8sTLIvMADfNfqXNdrxPPnzKmMtcU0Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732309847; a=rsa-sha256; cv=none; b=ICjWCjfrOCPZwOigx2xiXpIHVfCUWDNYFWClXfeYRzbJA/Pkb3qo5sQNjEI91rQcYxTpqc 8oucG2b/TYzOxnM+Op9UaTp6gEKynhiMHshxmIYsZmmeJkbou02PoHt7b+yCD8fAStyKuX 1iSaJoCUReJ2htEkP7BMb45Su0f8JyI= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HBp+qe4R; spf=pass (imf15.hostedemail.com: domain of bjohannesmeyer@gmail.com designates 209.85.208.180 as permitted sender) smtp.mailfrom=bjohannesmeyer@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lj1-f180.google.com with SMTP id 38308e7fff4ca-2fb561f273eso30448581fa.2 for ; Fri, 22 Nov 2024 13:11:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732309908; x=1732914708; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ceOYRg+db2e1ipikZCFVwiN54lElh7G8dmSvY9RkE2I=; b=HBp+qe4RqGmjlKLkEFJMEORkYiLqlZZl335/29zQ8rP9PNiV4YqibaLhkyUpPgqABb 8RCD9j6SyVww33hwsmBpTSiYV0lGzXG+X0Rjy9L4QERvMwyIyCy4gG/HVaHr7Bi3jwIS 9fMYvkCcP79FhMl/0zkgVhjqRJqZ7pqjmGcyceciT6noHO1T2XjaDf6kt0dMZRndPwwf s2TrVZERSqFhKfMNYclpZgLUHuWPiu+J5Qd3Hym28d509OKRxLaeOvpcw2FRftaBtG67 M6dPB1cNe4yKf9egw3gHnUTgH3Jkn7LglKTTUTRaqsBkfKyq0RxY+HtxKhikP67W4j7A 1XRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732309908; x=1732914708; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ceOYRg+db2e1ipikZCFVwiN54lElh7G8dmSvY9RkE2I=; b=dJvabl8HNnI1nDUvf4u05+ALhK8/kXWdA+eX7IHnqFR6QmL5F1b08LFqFpPYulOu/G Y8odcR08ukxJ2P7C3u2qvk20KRRt8dj3MZthhBp2Tca+QwFloyNdYULeEhyrTyjk4DSK aKCzsXr58Imz1W7qvyXG/mBQdfNbtBdjh6XJ8KQQLhg7qYqYyduZavPYhVygd49N0i0l uwKO/u4rgJJ7ZZ9Wg9/hY3ovoWCQpXaqajCyi/4Rl6q/03RVVBYFSDCZwFizBLmzBcE5 ms5haXtzSyFmPcscKhRDJwKC2Mm9Bu+as4XOBth3YnriflI8Sd8zJxZsrT/2aEPN6ieb tDrQ== X-Forwarded-Encrypted: i=1; AJvYcCUa5tMcyH1I+kDEkRdBlAR/WMEwshEM1tC+svESoOXjsgoCiU2AxHGpB8TaOefgOxdJe4A7uhJk2Q==@kvack.org X-Gm-Message-State: AOJu0YwR6gyPnRdan7bK7UljAB4/DEUVlV2oKs9005UC4lI8muPTyYxY pNUJtSs+AOakzhz0wFBbCrtz5DJwgplNcuzAntLvr8aJlp/jHGPB X-Gm-Gg: ASbGncuiwoqHjfZJDdpq3IivnYPnQ6hC3hph/JH50hcka86WvrLaUGO8pU1NzszCuQn uA9QCb47KA+x/uZuBAr0tnbnIUeyR8raIfRLW+vu+UMhuvjNRLe+b/BLpfp8A/TVs7RspTkWEp2 hbDhnc+EwgGG177aDUkK3Bk2OZev3758+UlXN3z3QNQUOtVW8eQDKSJOhMU8CSzNNQ84VavSQSN 0gFCCQYl7Td+CMON0JEUCOuxFUwWGYi8gpOUnAbTLzcU7edlOa4X7vfNkQIo3hQPMaBQMUTLnoY fQ0= X-Google-Smtp-Source: AGHT+IHflXg42uO4arPQnjfIO/UofICtO5/JXuDET1ub2S+twpsY6z11Bye48LygEoeC+l1hYpSXag== X-Received: by 2002:a2e:9811:0:b0:2ff:a89b:4348 with SMTP id 38308e7fff4ca-2ffa89b4584mr20119431fa.1.1732309908285; Fri, 22 Nov 2024 13:11:48 -0800 (PST) Received: from rex.hwlab.vusec.net (lab-4.lab.cs.vu.nl. [192.33.36.4]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5d01d3a3d77sm1276636a12.7.2024.11.22.13.11.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Nov 2024 13:11:47 -0800 (PST) From: Brian Johannesmeyer To: Keith Busch , Christoph Hellwig , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Cc: Brian Johannesmeyer , Raphael Isemann , Cristiano Giuffrida , Herbert Bos , Greg KH Subject: [PATCH v3 1/3] dmapool: Move pool metadata into non-DMA memory Date: Fri, 22 Nov 2024 22:11:43 +0100 Message-Id: <20241122211144.4186080-3-bjohannesmeyer@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241122211144.4186080-1-bjohannesmeyer@gmail.com> References: <20241122211144.4186080-1-bjohannesmeyer@gmail.com> MIME-Version: 1.0 X-Stat-Signature: jcfitrqfb74x8kc4kbgsqi5nh4d3w6py X-Rspam-User: X-Rspamd-Queue-Id: 96AF9A0016 X-Rspamd-Server: rspam02 X-HE-Tag: 1732309854-375430 X-HE-Meta: U2FsdGVkX19T8h/Hb1NBjT6YqrYjW6U+CvfLOHdpnuiquAs5G2QvkAXxd8flOHAqdRyasw0++v9WmU6Kgw6sofEs+ck/BkkvK4lnZ5f8xFZzqeh2PKp37JrugElkAIGJOTV9qAnhCU74tKcmgjVt0N9Z0Ny7e37WZ3YCwv0Jda2yryoAQAC6NF4eqUzSgfeGrSlkT2vnG2vnN1bbAGCtAvFp2KJ1TarnkOzxNl4ow0xJR/pcKr6zMFoKORLX+dDBknghF9fOBBI5TEcOHuGME5DQnVUxGPZ1WbmJXYR03kb90W/1PK7FWcaMOEQfcqmBJCaItCDlsWi/fD15+hAidog6x6/P3ylEgc3ZhoNTnTIZ3w2JyDHC+gtQwVUbbewVKj5teK4kKn7eeLobrHCObmbthtFSZISZJgYJ2jfF9yvO7b6mHiX/NX8GC0degKYMwTwUdXeCI4j74sV9oR36tK/cUc64gTHwzfPwwxv9axDpIFgJoD8rSMNAjJq9Lea65FrZ5vzipLO88G259IF58zg0bR1O4OZobYLBL4Shm2mPlJQHxBGxFOXzT52VuJrLcL2tESDVmEdsS/auJFVtFYcg+mzKx3n/C9i8JqbFnVWhySDjwxhPJ0we/GUblT1xtEWPP8FUY/1feoTt1PQDkK1JG9QZ+r1aS1h7vl/eLRXDedFmi/xayRGwJtTFx42c0VIUDneMf/ak5yeh5MSrHuMh7R+EPGFGbRwNK/0CrgavhT5UUEf1/Aq4mJSqZAQjZutgi6gsMyf8NQQ8j3+ObCXGheaPaN26ScgZEcs867UhK4GVs/o9zH/tIb3P0JmgyJhC65lTUAfSqsfF1Q+udbx1mHnJ/65FkvqsyupB0675iDfnWjQ3AKPSSFBf3aFObKnijgoeJJzqkSLA+3jogdYqGkI+rN4qlfAvqmG4BNGup+UNZTznePjZ+JcuZy3338egajx841kPRqcyaUY rplvT1M/ 8Qp+wkRiorxnzHga2RG3DGwcOAVQH8ll6geArgXVVwJdEUwtZHMwKPquuwFdVV+oK7lCKtJv/1wKC+IEj1s2ThTFRn/Mke12RAEYU55b8e7J6Yvws9GdMf94AjQ8ZGmsYEx4N1jeHrtNrs4aNTZiirZ8TN+L6F8H5lXy4brbIV5LziPkmAFBf4BrkcRy99O/P/pkv620zC6NsEH9L8KcmeEWIONidlxmeHLgZHktN8z6IMn0aS9iGZOpDCYyvxq3xxaNVdRbI+bHyJSHyn1UDVHgR+5cecUjyzeuk5NKFGPY0TeWg+O+GEG7MWwWM5kylZzSBHHhxXSLP333KKnK8vB1c0ZKeX8U/09HovC8kdnaW4GyNEWuRA5GBcA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a `struct dma_block` object resides in DMA memory, a malicious peripheral device can corrupt its metadata --- specifically, its `next_block` pointer, which links blocks in a DMA pool. By corrupting these pointers, an attacker can manipulate `dma_pool_alloc()` into returning attacker-controllable pointers, which can lead to kernel memory corruption from a driver that calls it. To prevent this, move the `struct dma_block` metadata into non-DMA memory, ensuring that devices cannot tamper with the internal pointers of the DMA pool allocator. Specifically: - Add a `vaddr` field to `struct dma_block` to point to the actual DMA-accessible block. - Maintain an array of `struct dma_block` objects in `struct dma_page` to track the metadata of each block within an allocated page. This change secures the DMA pool allocator by keeping its metadata in kernel memory, inaccessible to peripheral devices, thereby preventing potential attacks that could corrupt kernel memory through DMA operations. **Performance Impact** Unfortunately, performance results from the `DMAPOOL_TEST` test show this negatively affects performance. Before the patch: ``` dmapool test: size:16 align:16 blocks:8192 time:11860 dmapool test: size:64 align:64 blocks:8192 time:11951 dmapool test: size:256 align:256 blocks:8192 time:12287 dmapool test: size:1024 align:1024 blocks:2048 time:3134 dmapool test: size:4096 align:4096 blocks:1024 time:1686 dmapool test: size:68 align:32 blocks:8192 time:12050 ``` After the patch: ``` dmapool test: size:16 align:16 blocks:8192 time:34432 dmapool test: size:64 align:64 blocks:8192 time:62262 dmapool test: size:256 align:256 blocks:8192 time:238137 dmapool test: size:1024 align:1024 blocks:2048 time:61386 dmapool test: size:4096 align:4096 blocks:1024 time:75342 dmapool test: size:68 align:32 blocks:8192 time:88243 ``` While the performance impact is significant, this patch provides protection against malicious devices tampering with DMA pool metadata. A subsequent patch in this series introduces an optimization to mitigate the runtime overhead. Co-developed-by: Raphael Isemann Signed-off-by: Raphael Isemann Signed-off-by: Brian Johannesmeyer --- mm/dmapool.c | 62 +++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 52 insertions(+), 10 deletions(-) diff --git a/mm/dmapool.c b/mm/dmapool.c index f0bfc6c490f4..3790ca4a631d 100644 --- a/mm/dmapool.c +++ b/mm/dmapool.c @@ -43,6 +43,7 @@ struct dma_block { struct dma_block *next_block; dma_addr_t dma; + void *vaddr; }; struct dma_pool { /* the pool */ @@ -64,6 +65,8 @@ struct dma_page { /* cacheable header for 'allocation' bytes */ struct list_head page_list; void *vaddr; dma_addr_t dma; + struct dma_block *blocks; + size_t blocks_per_page; }; static DEFINE_MUTEX(pools_lock); @@ -91,14 +94,35 @@ static ssize_t pools_show(struct device *dev, struct device_attribute *attr, cha static DEVICE_ATTR_RO(pools); +static struct dma_block *pool_find_block(struct dma_pool *pool, void *vaddr) +{ + struct dma_page *page; + size_t offset, index; + + list_for_each_entry(page, &pool->page_list, page_list) { + if (vaddr < page->vaddr) + continue; + offset = vaddr - page->vaddr; + if (offset >= pool->allocation) + continue; + + index = offset / pool->size; + if (index >= page->blocks_per_page) + return NULL; + + return &page->blocks[index]; + } + return NULL; +} + #ifdef DMAPOOL_DEBUG static void pool_check_block(struct dma_pool *pool, struct dma_block *block, gfp_t mem_flags) { - u8 *data = (void *)block; + u8 *data = (void *)block->vaddr; int i; - for (i = sizeof(struct dma_block); i < pool->size; i++) { + for (i = 0; i < pool->size; i++) { if (data[i] == POOL_POISON_FREED) continue; dev_err(pool->dev, "%s %s, %p (corrupted)\n", __func__, @@ -114,7 +138,7 @@ static void pool_check_block(struct dma_pool *pool, struct dma_block *block, } if (!want_init_on_alloc(mem_flags)) - memset(block, POOL_POISON_ALLOCATED, pool->size); + memset(block->vaddr, POOL_POISON_ALLOCATED, pool->size); } static struct dma_page *pool_find_page(struct dma_pool *pool, dma_addr_t dma) @@ -143,7 +167,7 @@ static bool pool_block_err(struct dma_pool *pool, void *vaddr, dma_addr_t dma) } while (block) { - if (block != vaddr) { + if (block->vaddr != vaddr) { block = block->next_block; continue; } @@ -238,8 +262,6 @@ struct dma_pool *dma_pool_create(const char *name, struct device *dev, if (size == 0 || size > INT_MAX) return NULL; - if (size < sizeof(struct dma_block)) - size = sizeof(struct dma_block); size = ALIGN(size, align); allocation = max_t(size_t, size, PAGE_SIZE); @@ -301,6 +323,7 @@ static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) { unsigned int next_boundary = pool->boundary, offset = 0; struct dma_block *block, *first = NULL, *last = NULL; + size_t i = 0; pool_init_page(pool, page); while (offset + pool->size <= pool->allocation) { @@ -310,7 +333,8 @@ static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) continue; } - block = page->vaddr + offset; + block = &page->blocks[i]; + block->vaddr = page->vaddr + offset; block->dma = page->dma + offset; block->next_block = NULL; @@ -322,6 +346,7 @@ static void pool_initialise_page(struct dma_pool *pool, struct dma_page *page) offset += pool->size; pool->nr_blocks++; + i++; } last->next_block = pool->next_block; @@ -339,9 +364,18 @@ static struct dma_page *pool_alloc_page(struct dma_pool *pool, gfp_t mem_flags) if (!page) return NULL; + page->blocks_per_page = pool->allocation / pool->size; + page->blocks = kmalloc_array(page->blocks_per_page, + sizeof(struct dma_block), GFP_KERNEL); + if (!page->blocks) { + kfree(page); + return NULL; + } + page->vaddr = dma_alloc_coherent(pool->dev, pool->allocation, &page->dma, mem_flags); if (!page->vaddr) { + kfree(page->blocks); kfree(page); return NULL; } @@ -383,6 +417,7 @@ void dma_pool_destroy(struct dma_pool *pool) if (!busy) dma_free_coherent(pool->dev, pool->allocation, page->vaddr, page->dma); + kfree(page->blocks); list_del(&page->page_list); kfree(page); } @@ -432,9 +467,9 @@ void *dma_pool_alloc(struct dma_pool *pool, gfp_t mem_flags, *handle = block->dma; pool_check_block(pool, block, mem_flags); if (want_init_on_alloc(mem_flags)) - memset(block, 0, pool->size); + memset(block->vaddr, 0, pool->size); - return block; + return block->vaddr; } EXPORT_SYMBOL(dma_pool_alloc); @@ -449,9 +484,16 @@ EXPORT_SYMBOL(dma_pool_alloc); */ void dma_pool_free(struct dma_pool *pool, void *vaddr, dma_addr_t dma) { - struct dma_block *block = vaddr; + struct dma_block *block; unsigned long flags; + block = pool_find_block(pool, vaddr); + if (!block) { + dev_err(pool->dev, "%s %s, invalid vaddr %p\n", + __func__, pool->name, vaddr); + return; + } + spin_lock_irqsave(&pool->lock, flags); if (!pool_block_err(pool, vaddr, dma)) { pool_block_push(pool, block, dma);