From patchwork Thu Apr 27 00:08:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 13225034 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B32CAC77B7C for ; Thu, 27 Apr 2023 00:09:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1841B6B0078; Wed, 26 Apr 2023 20:09:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 134866B007D; Wed, 26 Apr 2023 20:09:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F16366B007E; Wed, 26 Apr 2023 20:09:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E01C36B0078 for ; Wed, 26 Apr 2023 20:09:49 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id ACB88120408 for ; Thu, 27 Apr 2023 00:09:49 +0000 (UTC) X-FDA: 80725237698.13.8E3FBA3 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf06.hostedemail.com (Postfix) with ESMTP id C9E11180010 for ; Thu, 27 Apr 2023 00:09:47 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=GCWhljQN; spf=pass (imf06.hostedemail.com: domain of anthony.yznaga@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=anthony.yznaga@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682554187; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=MfX2UCA2WLfnR/ast08TW20ISh9rczjU8Ar+dRs48HA=; b=8gZ4J6rdxNU8XG0Otb5tvoOQAWXdjhZvx3RUb/K5a7e7N8GCbYJlF5PAQRqXHg+KJV2UtO 0RkqcTNpjbFV09GDHyDF0K70xDsSk730MIYsv0u4gsjvhiIxoiVRh9/p8diZtaZdL3BrLb iHNfhTNi4aCR+HqaAC/lgxhhF3yaZjY= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=GCWhljQN; spf=pass (imf06.hostedemail.com: domain of anthony.yznaga@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=anthony.yznaga@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682554187; a=rsa-sha256; cv=none; b=SoEkPi1ENhfZ3pUM3vIAxF1IMILF8d7+uyFc84M3TGB5r1Xe84WMWaUtu98VEnujimNww2 +VE0BoYxmhKo+AlaHho7WhnLNXgA+vn1tBje1V7aAabNqIGHSvEWJMIs0fL7HhM39S26yq 5zd0PNRah+CjjAC1GnGwHhABK2uBXU4= Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33QGx0jh015505; Thu, 27 Apr 2023 00:09:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2023-03-30; bh=MfX2UCA2WLfnR/ast08TW20ISh9rczjU8Ar+dRs48HA=; b=GCWhljQNjE5AjIClKhqzHeKVW3z5kr5UBJP7HEDWHVPjD+xiaPJ2/5t9JZYlCLAGl/i7 yvTMTaC3FGePZ3YrF4pTNY5ulz6nyTG1wx4O2xEl56aT+LOheMQ72F2eqB71yGtxnX33 o/hrmAtnO3q1uFdVKVmX1jK+/8S2wrSpWAvtkAkE9A4H6AT9NupoiSiA9BPoU69PpeL1 VoaJtJ2XB0hsW0yKsZDTdd1p+zgv2sLxe8Xm5EP2E3B7UpH96KL1y7cW/00QTYOxaYPR XXZ0vDADFP4dLfYAb/Ef7z3nTdE2ZfvilGGHVUfiEBjCMvZFLCCUdIorty/kdB5tRZZL RQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3q460dampk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 27 Apr 2023 00:09:13 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 33QNBFC2007555; Thu, 27 Apr 2023 00:09:12 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3q4618mpcy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 27 Apr 2023 00:09:12 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 33R0938W013888; Thu, 27 Apr 2023 00:09:11 GMT Received: from ca-qasparc-x86-2.us.oracle.com (ca-qasparc-x86-2.us.oracle.com [10.147.24.103]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3q4618mp42-6; Thu, 27 Apr 2023 00:09:11 +0000 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@kernel.org, akpm@linux-foundation.org, ebiederm@xmission.com, keescook@chromium.org, graf@amazon.com, jason.zeng@intel.com, lei.l.li@intel.com, steven.sistare@oracle.com, fam.zheng@bytedance.com, mgalaxy@akamai.com, kexec@lists.infradead.org Subject: [RFC v3 05/21] mm: PKRAM: implement byte stream operations Date: Wed, 26 Apr 2023 17:08:41 -0700 Message-Id: <1682554137-13938-6-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.9.4 In-Reply-To: <1682554137-13938-1-git-send-email-anthony.yznaga@oracle.com> References: <1682554137-13938-1-git-send-email-anthony.yznaga@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-26_10,2023-04-26_03,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 spamscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304270000 X-Proofpoint-GUID: oHWiWle6IkzwQxWZ4r-eqosGLrXgVc0t X-Proofpoint-ORIG-GUID: oHWiWle6IkzwQxWZ4r-eqosGLrXgVc0t X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C9E11180010 X-Rspam-User: X-Stat-Signature: a7sxcq3dq66ezyn6ckayr79yxtego6hu X-HE-Tag: 1682554187-843949 X-HE-Meta: U2FsdGVkX1/VdRQrbG+JkkptjZN/9CXw78xwBB7/G/69RoLnsycOyJ0e+2ABXxPkZnL5xdYPOtfAuqJPDZMaDrWD3+J0J4LVmBBcBWGqoOmPtbYM+cCImNrpOA+kkGAf6U9BlFDTRm0LtzMqw/hpPlsi7upVfd9GcQ5wilci3f7ySkX30S5EckSfcKTMbldOCxoADckxSAJGxaRB2EQfi+/3Y8QLyhMYyQQBXrl4IfoRP1n8i3IwjsK1M1+T+7LcBRpp7i9cibIuB0PoGDjajI4c/hqe4A/DTWf0F4ej3e0IjioUwYM971/01ej+33P0Bvw/cgzqpLeTXPh2dyh7XwbW3h37S/BIjdF+CiTjcbuJKtT8uB6Ni9rzNhFtFUw6t+BH6sj6OXYSi3pVdgxRtGZHYghSkwyWWM2kEG20MRd8OK8VniAPpaVZnOsvbRGjt+XPolsvt8sqiTjAPZPvC//QeSSLn3s5SdJFqzTs9ZJPj4IfYtNdjkMWKTxHTufFqzWBcuyjR5u5H9Kbxb4sr545BR0B5t+q5hHfIXzxlTeFzV8OgrQp6qVtJP45+EuwWnPWMKYeJpwMIICbV2mSCsCNTuJyOh0Uj29Qt/jZT38h77Eov6DkUUbb/cvXyUhU5t3oyvWbWR5cFLCco3TImjVcdq03mDTuSvJY/lM4PKoToZ3GKz2GQ89iU/LWY6UgwI48ulJvz7H17ss0ldAOXwxKxD+upPFQJVz8rN3hBFFFhmH5FLe7I/4+m+MHo/GXpe/l1k9zx239E3cEMgaahRdisVsjV0VzLdFFTGAQKjbskE/tE3zdQj4vVvVogxXoZta25UAp7Q/4cxjVewRpwNf6mVRcQp1g6QRIHCRU/e0oSgqfAKWvSug5EioL6AiARdMfXhIevKzlx4Msmo1toUASNeWW9bYaU7o5r8fPENCC7418uASIbGRADpqMsCxyh7xvfSet8nQiikUVhpr Jqyx68r6 /eaLVZIYzI0SCtcm3J3bALQJ3CBWiXo61c17IQsM3nFk3uB+46CTAD73SeIu1tvPHzdZqn11XJ9LGHuFwNMVrLrNXxEQKVqABwiEK9FcHb5efl7ThH9CrTgNDC3ssvzJ5G43soZEnwRs/3xkRSA4hEpRpRWXstA1YNzT1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch adds the ability to save an arbitrary byte streams to a a PKRAM object using pkram_write() to be restored later using pkram_read(). Originally-by: Vladimir Davydov Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 11 +++++ mm/pkram.c | 123 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 130 insertions(+), 4 deletions(-) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index 130ab5c2d94a..b614e9059bba 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -14,10 +14,12 @@ * enum pkram_data_flags - definition of data types contained in a pkram obj * @PKRAM_DATA_none: No data types configured * @PKRAM_DATA_folios: obj contains folio data + * @PKRAM_DATA_bytes: obj contains byte data */ enum pkram_data_flags { PKRAM_DATA_none = 0x0, /* No data types configured */ PKRAM_DATA_folios = 0x1, /* Contains folio data */ + PKRAM_DATA_bytes = 0x2, /* Contains byte data */ }; struct pkram_data_stream { @@ -36,18 +38,27 @@ struct pkram_stream { __u64 *folios_head_link_pfnp; __u64 *folios_tail_link_pfnp; + + __u64 *bytes_head_link_pfnp; + __u64 *bytes_tail_link_pfnp; }; struct pkram_folios_access { unsigned long next_index; }; +struct pkram_bytes_access { + struct page *data_page; /* current page */ + unsigned int data_offset; /* offset into current page */ +}; + struct pkram_access { enum pkram_data_flags dtype; struct pkram_stream *ps; struct pkram_data_stream pds; struct pkram_folios_access folios; + struct pkram_bytes_access bytes; }; #define PKRAM_NAME_MAX 256 /* including nul */ diff --git a/mm/pkram.c b/mm/pkram.c index 610ff7a88c98..eac8cf6b0cdf 100644 --- a/mm/pkram.c +++ b/mm/pkram.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include #include #include #include @@ -44,6 +45,9 @@ struct pkram_link { struct pkram_obj { __u64 folios_head_link_pfn; /* the first folios link of the object */ __u64 folios_tail_link_pfn; /* the last folios link of the object */ + __u64 bytes_head_link_pfn; /* the first bytes link of the object */ + __u64 bytes_tail_link_pfn; /* the last bytes link of the object */ + __u64 data_len; /* byte data size */ __u64 obj_pfn; /* points to the next object in the list */ }; @@ -138,6 +142,11 @@ static void pkram_truncate_obj(struct pkram_obj *obj) pkram_truncate_links(obj->folios_head_link_pfn); obj->folios_head_link_pfn = 0; obj->folios_tail_link_pfn = 0; + + pkram_truncate_links(obj->bytes_head_link_pfn); + obj->bytes_head_link_pfn = 0; + obj->bytes_tail_link_pfn = 0; + obj->data_len = 0; } static void pkram_truncate_node(struct pkram_node *node) @@ -310,7 +319,7 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); - if (flags & ~PKRAM_DATA_folios) + if (flags & ~(PKRAM_DATA_folios | PKRAM_DATA_bytes)) return -EINVAL; page = pkram_alloc_page(ps->gfp_mask | __GFP_ZERO); @@ -326,6 +335,10 @@ int pkram_prepare_save_obj(struct pkram_stream *ps, enum pkram_data_flags flags) ps->folios_head_link_pfnp = &obj->folios_head_link_pfn; ps->folios_tail_link_pfnp = &obj->folios_tail_link_pfn; } + if (flags & PKRAM_DATA_bytes) { + ps->bytes_head_link_pfnp = &obj->bytes_head_link_pfn; + ps->bytes_tail_link_pfnp = &obj->bytes_tail_link_pfn; + } ps->obj = obj; return 0; } @@ -432,7 +445,7 @@ int pkram_prepare_load_obj(struct pkram_stream *ps) return -ENODATA; obj = pfn_to_kaddr(node->obj_pfn); - if (!obj->folios_head_link_pfn) { + if (!obj->folios_head_link_pfn && !obj->bytes_head_link_pfn) { WARN_ON(1); return -EINVAL; } @@ -443,6 +456,10 @@ int pkram_prepare_load_obj(struct pkram_stream *ps) ps->folios_head_link_pfnp = &obj->folios_head_link_pfn; ps->folios_tail_link_pfnp = &obj->folios_tail_link_pfn; } + if (obj->bytes_head_link_pfn) { + ps->bytes_head_link_pfnp = &obj->bytes_head_link_pfn; + ps->bytes_tail_link_pfnp = &obj->bytes_tail_link_pfn; + } ps->obj = obj; return 0; } @@ -493,6 +510,9 @@ void pkram_finish_access(struct pkram_access *pa, bool status_ok) if (pa->pds.link) pkram_truncate_link(pa->pds.link); + + if ((pa->dtype == PKRAM_DATA_bytes) && (pa->bytes.data_page)) + pkram_free_page(page_address(pa->bytes.data_page)); } /* @@ -547,6 +567,22 @@ int pkram_save_folio(struct pkram_access *pa, struct folio *folio) return __pkram_save_page(pa, page, page->index); } +static int __pkram_bytes_save_page(struct pkram_access *pa, struct page *page) +{ + struct pkram_data_stream *pds = &pa->pds; + struct pkram_link *link = pds->link; + + if (!link || pds->entry_idx >= PKRAM_LINK_ENTRIES_MAX) { + link = pkram_new_link(pds, pa->ps->gfp_mask); + if (!link) + return -ENOMEM; + } + + pkram_add_link_entry(pds, page); + + return 0; +} + static struct page *__pkram_prep_load_page(pkram_entry_t p) { struct page *page; @@ -662,10 +698,53 @@ struct folio *pkram_load_folio(struct pkram_access *pa, unsigned long *index) * * On success, returns the number of bytes written, which is always equal to * @count. On failure, -errno is returned. + * + * Error values: + * %ENOMEM: insufficient amount of memory available */ ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count) { - return -EINVAL; + struct pkram_node *node = pa->ps->node; + struct pkram_obj *obj = pa->ps->obj; + size_t copy_count, write_count = 0; + void *addr; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_SAVE); + + while (count > 0) { + if (!pa->bytes.data_page) { + gfp_t gfp_mask = pa->ps->gfp_mask; + struct page *page; + int err; + + page = pkram_alloc_page((gfp_mask & GFP_RECLAIM_MASK) | + __GFP_HIGHMEM | __GFP_ZERO); + if (!page) + return -ENOMEM; + err = __pkram_bytes_save_page(pa, page); + if (err) { + pkram_free_page(page_address(page)); + return err; + } + pa->bytes.data_page = page; + pa->bytes.data_offset = 0; + } + + copy_count = min_t(size_t, count, PAGE_SIZE - pa->bytes.data_offset); + addr = kmap_local_page(pa->bytes.data_page); + memcpy(addr + pa->bytes.data_offset, buf, copy_count); + kunmap_local(addr); + + buf += copy_count; + obj->data_len += copy_count; + pa->bytes.data_offset += copy_count; + if (pa->bytes.data_offset >= PAGE_SIZE) + pa->bytes.data_page = NULL; + + write_count += copy_count; + count -= copy_count; + } + return write_count; } /** @@ -679,5 +758,41 @@ ssize_t pkram_write(struct pkram_access *pa, const void *buf, size_t count) */ size_t pkram_read(struct pkram_access *pa, void *buf, size_t count) { - return 0; + struct pkram_node *node = pa->ps->node; + struct pkram_obj *obj = pa->ps->obj; + size_t copy_count, read_count = 0; + char *addr; + + BUG_ON((node->flags & PKRAM_ACCMODE_MASK) != PKRAM_LOAD); + + while (count > 0 && obj->data_len > 0) { + if (!pa->bytes.data_page) { + struct page *page; + + page = __pkram_load_page(pa, NULL); + if (IS_ERR_OR_NULL(page)) + break; + pa->bytes.data_page = page; + pa->bytes.data_offset = 0; + } + + copy_count = min_t(size_t, count, PAGE_SIZE - pa->bytes.data_offset); + if (copy_count > obj->data_len) + copy_count = obj->data_len; + addr = kmap_local_page(pa->bytes.data_page); + memcpy(buf, addr + pa->bytes.data_offset, copy_count); + kunmap_local(addr); + + buf += copy_count; + obj->data_len -= copy_count; + pa->bytes.data_offset += copy_count; + if (pa->bytes.data_offset >= PAGE_SIZE || !obj->data_len) { + put_page(pa->bytes.data_page); + pa->bytes.data_page = NULL; + } + + read_count += copy_count; + count -= copy_count; + } + return read_count; }