From patchwork Tue Sep 10 23:43:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ackerley Tng X-Patchwork-Id: 13799487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7C31EE01F1 for ; Tue, 10 Sep 2024 23:45:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 139448D00D9; Tue, 10 Sep 2024 19:45:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04F558D0002; Tue, 10 Sep 2024 19:44:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBC998D00D9; Tue, 10 Sep 2024 19:44:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B30168D0002 for ; Tue, 10 Sep 2024 19:44:59 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7249040C20 for ; Tue, 10 Sep 2024 23:44:59 +0000 (UTC) X-FDA: 82550461518.16.BEED4D0 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf12.hostedemail.com (Postfix) with ESMTP id 91B2A4000C for ; Tue, 10 Sep 2024 23:44:57 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=b7sLJDsj; spf=pass (imf12.hostedemail.com: domain of 3-NngZgsKCG0LNVPcWPjeYRRZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--ackerleytng.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3-NngZgsKCG0LNVPcWPjeYRRZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--ackerleytng.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726011845; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9Gc31H97UbE78QiSNq4CUIpVlCNlnI2SeIjwGOQZSc0=; b=qINsxTU7QtUTxevIRmf99WIUDclSuSbTnxPBdV3eRha9X3RKmW1U1lOjyv0cB+vchNS8uC AKN2CgY2p4EC8rg5tt9uuEgVMMxhWyBaUsnuVtKbdQU3dCk+Ngf0iyI5nYoGZyds5eKL6X EyY7OvHJPj8DplM3VAWB2XvJJEffk1c= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=b7sLJDsj; spf=pass (imf12.hostedemail.com: domain of 3-NngZgsKCG0LNVPcWPjeYRRZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--ackerleytng.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3-NngZgsKCG0LNVPcWPjeYRRZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--ackerleytng.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726011845; a=rsa-sha256; cv=none; b=uPBJWiVnPE5RLk+P9ktivTdT9dYiMiOfT3Kv+VvlE8d+emwKncQT2pDQtEjRih9r8Cpf8q So0dABfep6Y55bh3B5eLgV0xfUOMRVO8oGcrP4NZvXk0NQaMpqZ+EsSXA3QZU2gUfNvS3p IvMCcHulaeQdT/JOAJLgn/KDhsb4nZA= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2d8859d6e9dso1163423a91.1 for ; Tue, 10 Sep 2024 16:44:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1726011896; x=1726616696; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9Gc31H97UbE78QiSNq4CUIpVlCNlnI2SeIjwGOQZSc0=; b=b7sLJDsjOlcbnyTHujKClBCyeAb4b3XJKw5/IIPodD3AWW7SMufAz1pcweAMX5iAA4 5vogATQj5AJ+UhojwhtRFkGGH0Si/7ho3c/u0BbB1ub8WBxVNBu1jN9S4jc/+yiTHOUm c04EZAeYhIp45yBG7wiJV5jrKf0jI1/KyoSJhvgwkdxwJS2lKrDGsL0zWuMlam9s3SOY BdzEu7WsKpVnnm3ctphrRmju24e8LOmv/eFl9rVN1FtflHf/JoBfmDnRD8IOUv/PmFat /mx05OlTJev3qlQl28WioGJVEkGd8rdRXMs+4EDqG4SZw7I70bZeuM03INtIp4+BR7Jt uJ0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726011896; x=1726616696; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9Gc31H97UbE78QiSNq4CUIpVlCNlnI2SeIjwGOQZSc0=; b=s1P5fX8v2wpOC9cwOz2pQ8nc6pgkbj7KdnEAWheZOQQ+ZovoNKAaZK98aATrIhgpnu REc0z5wj7krsxhpg11T0xtD3Tmnn/fjjE0pViUCQh8f4aa4v/BO1dl9ZcMrOQhkfgeDN lg/5O/9O52vLWYeKfh55ymeZrnNMR+fTnMr3wRXAIekO3wPgePLgAVtvJHXRDf/CEr35 USBrfEBXHowqKbe5kyShECSP34kUk189zGOloU/npnIa1C84UbA0GVVQwZRuvLyiYur8 ZXS9NSX6KjNYXHMw5DycRPZtodUYYqMdme1qpjpS4MCuww6Hu7Xf1W4h5jWqbvl6gJGh fEsw== X-Forwarded-Encrypted: i=1; AJvYcCV199jSxzRtrfsMAwfmZXZE/0k/d13wrJQrmkIbjrSaVqRNsCmuqd69GciIoqvMnsoi8YpIiHpF3w==@kvack.org X-Gm-Message-State: AOJu0YwYvjb5ByyKJOOXU3zh2/WTC8xLTwNCTT+a43G3ezW3pMuPKQK8 87JDEykltJaZi9oSm4ozPMMXtqAoHSdvKRim8SFkTdEq6VW54jxex5tWY3PSUE8uhQ0ADHiWQR1 mQvNux7zqL+kazTsixdfTrw== X-Google-Smtp-Source: AGHT+IGTAqnZcaBNVO3Nl4Ef5dtGerZRypTfLYB/jrgUVkEZWYdnxrwGBGsysGpj4YxZly3jtQUA0VOLee5RXF37ZQ== X-Received: from ackerleytng-ctop.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:13f8]) (user=ackerleytng job=sendgmr) by 2002:a17:902:ce91:b0:206:928c:bfd9 with SMTP id d9443c01a7336-20752208a62mr470995ad.6.1726011896097; Tue, 10 Sep 2024 16:44:56 -0700 (PDT) Date: Tue, 10 Sep 2024 23:43:45 +0000 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.46.0.598.g6f2099f65c-goog Message-ID: <3fec11d8a007505405eadcf2b3e10ec9051cf6bf.1726009989.git.ackerleytng@google.com> Subject: [RFC PATCH 14/39] KVM: guest_memfd: hugetlb: initialization and cleanup From: Ackerley Tng To: tabba@google.com, quic_eberman@quicinc.com, roypat@amazon.co.uk, jgg@nvidia.com, peterx@redhat.com, david@redhat.com, rientjes@google.com, fvdl@google.com, jthoughton@google.com, seanjc@google.com, pbonzini@redhat.com, zhiquan1.li@intel.com, fan.du@intel.com, jun.miao@intel.com, isaku.yamahata@intel.com, muchun.song@linux.dev, mike.kravetz@oracle.com Cc: erdemaktas@google.com, vannapurve@google.com, ackerleytng@google.com, qperret@google.com, jhubbard@nvidia.com, willy@infradead.org, shuah@kernel.org, brauner@kernel.org, bfoster@redhat.com, kent.overstreet@linux.dev, pvorel@suse.cz, rppt@kernel.org, richard.weiyang@gmail.com, anup@brainfault.org, haibo1.xu@intel.com, ajones@ventanamicro.com, vkuznets@redhat.com, maciej.wieczor-retman@intel.com, pgonda@google.com, oliver.upton@linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kvm@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-fsdevel@kvack.org X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 91B2A4000C X-Stat-Signature: ns7narsky44mamfp8rauxg1ftomw3oe6 X-HE-Tag: 1726011897-862784 X-HE-Meta: U2FsdGVkX1/IJIzdU0XYOtrUla+J5MLwPcZ4fZndQkz1AvZzcpxKareZfIHyB48ciaDLwlW60xfzVBX9rA8LVP4HD1cKPgsW1HRPg8sKmHz/clVB/djP9alCHjjpQkSzLSTXtl5ng1Yde7YX3JWUj+CV3od/p87w/idHfc9oA8+HF/8v2g89d6+NYPRJFAjRhWYG3zRSZKU5c8fLEinXo1h6/CFmfUaTyvHL0jUH3YSWCeBUOtxgx3+j9tXJbtMWudUGe7yr8OCwMmpBxNByw4bNIPdyvnDpH/FVHmbZ3Jh2b1wi0nL0M4fFuJy9t2XCciCkjXRWoFjlM5bTtsPHVEPfwSY1rLp6d1Rdcx17GhSK2qLgUjYMUzX8kB3ediKgt9fidkVzXy7D6zx/LZHTGb1BvCbSLeR7eu6nl5NM76uFYtVrwPIop1osXTEMVGbbWK4i4b3WdSe0pyMeRIf732H+k/c9XWftu16LyN4Hwg5T58MkGltxK4TqaeYhwIGmgEMGFyHCONyn1EDOU6CACz4e2UnJiDS+SWvCu/M/+6RaKdxjVsozwsrJmtDvn2gfL4H3hxEx8wynVwuEUN8A7YnLiTwvqDawywQ9JaVbdHEsT5N+qQPm3AzbLbhuXEFrk3/m8oI8VvEA6sG0G3oi55ZAHVhsCW11vxfxL93fEz2nl28LP+micMierTuCDQV+d003fykuXh9KTX3mR9szWOieshXT2t3CBt3xNcGkvakP8oEG46VMOFLu+OlX+r8Wn7CIkoxvo5hFCqmOkCjP7AzZSke7VsfV3OFXWLnudjOGS5ihMfVCVbu3nWl0VtpBpJqbS/dbpXqrDC5Yinb1VbhZ0qN+Ai1iZd3U7TklF3Z7NsOYLIAvi7/Uwn03C9UOdnGGdO/1lppDT1wm2Dc27QYctI2VwbiLg/JeJoxb5lIgydRu2LIRFCTDRp1+R6Eurfe9LTQvO8EzTrJeRlJ ue5oFCJL zi5QjOkHXVrEyrtj6rqRfG9e/aZrDIvkD8OW/8fdfjR8qfJo4+L7m4xQmr21XWwxG6q/Je86jD5xs5tAY+L4oxdVna54fB1FtB5h1e4wyXNw9ittSDGRJH/dDAM7J7Q+8d6m0RqSq5zmotzMktTPZJyHQrvEVUt5NhKwIN2t8//RGjGN2nsarUNQz4fWWHrRTaOvzpGeKnBn+btvBmn5Vvc16po+9v0BCytJToCv8sfjxkd+G1Y3ZJ+R6naMQ+Z27SL9UmFfF1Wpt2ofpQtNrSVPxBVWOreCATmyQJIZxvB18vexbRNHC2N/H4OPVf0zdspjOQqBQ8vbMprXS2/VGjJF8Ewr7l9kLVWHhA4Vzbrp3yop3J5T3KdKpjAd/zwUXbIoh2tbgGMKnYB1LqMpYvIDi2kOXei84tiQRWerXWRhsYF0DAJ8v8HqRBVxJJS2Bwbfk9CGv4uCtVlRaPidAe7+CPT+fnzOaO9Tyk9CDWg4u9KS4lPoq5SJx2ONDbGJjW/bLzeTLaNA/QMcRkaqpWxcT4ZKBtQS12q9ku+6ROhtW+bF92AH7kKV9KRscqpKMfmHDYG+DDW3s9hXbaPr8gHXA81MHGVFkEVZNxHLlJsQA2Rr4jnmeW+cyN1kLJqFLLhrvCs/eDdl+UYkB6j8YQtWG4Gk13YowXAS411zPoxq63GBDQm2cXo/uGjAGeXjTo8B7EXr5Kl4sWGE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: First stage of hugetlb support: add initialization and cleanup routines. After guest_mem was massaged to use guest_mem inodes instead of anonymous inodes in an earlier patch, the .evict_inode handler can now be overridden to do hugetlb metadata cleanup. Signed-off-by: Ackerley Tng --- include/uapi/linux/kvm.h | 26 ++++++ virt/kvm/guest_memfd.c | 177 +++++++++++++++++++++++++++++++++++++-- 2 files changed, 197 insertions(+), 6 deletions(-) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 637efc055145..77de7c4432f6 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -13,6 +13,7 @@ #include #include #include +#include #define KVM_API_VERSION 12 @@ -1558,6 +1559,31 @@ struct kvm_memory_attributes { #define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd) +#define KVM_GUEST_MEMFD_HUGETLB (1ULL << 1) + +/* + * Huge page size encoding when KVM_GUEST_MEMFD_HUGETLB is specified, and a huge + * page size other than the default is desired. See hugetlb_encode.h. All + * known huge page size encodings are provided here. It is the responsibility + * of the application to know which sizes are supported on the running system. + * See mmap(2) man page for details. + */ +#define KVM_GUEST_MEMFD_HUGE_SHIFT HUGETLB_FLAG_ENCODE_SHIFT +#define KVM_GUEST_MEMFD_HUGE_MASK HUGETLB_FLAG_ENCODE_MASK + +#define KVM_GUEST_MEMFD_HUGE_64KB HUGETLB_FLAG_ENCODE_64KB +#define KVM_GUEST_MEMFD_HUGE_512KB HUGETLB_FLAG_ENCODE_512KB +#define KVM_GUEST_MEMFD_HUGE_1MB HUGETLB_FLAG_ENCODE_1MB +#define KVM_GUEST_MEMFD_HUGE_2MB HUGETLB_FLAG_ENCODE_2MB +#define KVM_GUEST_MEMFD_HUGE_8MB HUGETLB_FLAG_ENCODE_8MB +#define KVM_GUEST_MEMFD_HUGE_16MB HUGETLB_FLAG_ENCODE_16MB +#define KVM_GUEST_MEMFD_HUGE_32MB HUGETLB_FLAG_ENCODE_32MB +#define KVM_GUEST_MEMFD_HUGE_256MB HUGETLB_FLAG_ENCODE_256MB +#define KVM_GUEST_MEMFD_HUGE_512MB HUGETLB_FLAG_ENCODE_512MB +#define KVM_GUEST_MEMFD_HUGE_1GB HUGETLB_FLAG_ENCODE_1GB +#define KVM_GUEST_MEMFD_HUGE_2GB HUGETLB_FLAG_ENCODE_2GB +#define KVM_GUEST_MEMFD_HUGE_16GB HUGETLB_FLAG_ENCODE_16GB + struct kvm_create_guest_memfd { __u64 size; __u64 flags; diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 5d7fd1f708a6..31e1115273e1 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -18,6 +19,16 @@ struct kvm_gmem { struct list_head entry; }; +struct kvm_gmem_hugetlb { + struct hstate *h; + struct hugepage_subpool *spool; +}; + +static struct kvm_gmem_hugetlb *kvm_gmem_hgmem(struct inode *inode) +{ + return inode->i_mapping->i_private_data; +} + /** * folio_file_pfn - like folio_file_page, but return a pfn. * @folio: The folio which contains this index. @@ -154,6 +165,82 @@ static void kvm_gmem_invalidate_end(struct kvm_gmem *gmem, pgoff_t start, } } +static inline void kvm_gmem_hugetlb_filemap_remove_folio(struct folio *folio) +{ + folio_lock(folio); + + folio_clear_dirty(folio); + folio_clear_uptodate(folio); + filemap_remove_folio(folio); + + folio_unlock(folio); +} + +/** + * Removes folios in range [@lstart, @lend) from page cache/filemap (@mapping), + * returning the number of pages freed. + */ +static int kvm_gmem_hugetlb_filemap_remove_folios(struct address_space *mapping, + struct hstate *h, + loff_t lstart, loff_t lend) +{ + const pgoff_t end = lend >> PAGE_SHIFT; + pgoff_t next = lstart >> PAGE_SHIFT; + struct folio_batch fbatch; + int num_freed = 0; + + folio_batch_init(&fbatch); + while (filemap_get_folios(mapping, &next, end - 1, &fbatch)) { + int i; + for (i = 0; i < folio_batch_count(&fbatch); ++i) { + struct folio *folio; + pgoff_t hindex; + u32 hash; + + folio = fbatch.folios[i]; + hindex = folio->index >> huge_page_order(h); + hash = hugetlb_fault_mutex_hash(mapping, hindex); + + mutex_lock(&hugetlb_fault_mutex_table[hash]); + kvm_gmem_hugetlb_filemap_remove_folio(folio); + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + + num_freed++; + } + folio_batch_release(&fbatch); + cond_resched(); + } + + return num_freed; +} + +/** + * Removes folios in range [@lstart, @lend) from page cache of inode, updates + * inode metadata and hugetlb reservations. + */ +static void kvm_gmem_hugetlb_truncate_folios_range(struct inode *inode, + loff_t lstart, loff_t lend) +{ + struct kvm_gmem_hugetlb *hgmem; + struct hstate *h; + int gbl_reserve; + int num_freed; + + hgmem = kvm_gmem_hgmem(inode); + h = hgmem->h; + + num_freed = kvm_gmem_hugetlb_filemap_remove_folios(inode->i_mapping, + h, lstart, lend); + + gbl_reserve = hugepage_subpool_put_pages(hgmem->spool, num_freed); + hugetlb_acct_memory(h, -gbl_reserve); + + spin_lock(&inode->i_lock); + inode->i_blocks -= blocks_per_huge_page(h) * num_freed; + spin_unlock(&inode->i_lock); +} + + static long kvm_gmem_punch_hole(struct inode *inode, loff_t offset, loff_t len) { struct list_head *gmem_list = &inode->i_mapping->i_private_list; @@ -307,8 +394,33 @@ static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot) return get_file_active(&slot->gmem.file); } +static void kvm_gmem_hugetlb_teardown(struct inode *inode) +{ + struct kvm_gmem_hugetlb *hgmem; + + truncate_inode_pages_final_prepare(inode->i_mapping); + kvm_gmem_hugetlb_truncate_folios_range(inode, 0, LLONG_MAX); + + hgmem = kvm_gmem_hgmem(inode); + hugepage_put_subpool(hgmem->spool); + kfree(hgmem); +} + +static void kvm_gmem_evict_inode(struct inode *inode) +{ + u64 flags = (u64)inode->i_private; + + if (flags & KVM_GUEST_MEMFD_HUGETLB) + kvm_gmem_hugetlb_teardown(inode); + else + truncate_inode_pages_final(inode->i_mapping); + + clear_inode(inode); +} + static const struct super_operations kvm_gmem_super_operations = { .statfs = simple_statfs, + .evict_inode = kvm_gmem_evict_inode, }; static int kvm_gmem_init_fs_context(struct fs_context *fc) @@ -431,6 +543,42 @@ static const struct inode_operations kvm_gmem_iops = { .setattr = kvm_gmem_setattr, }; +static int kvm_gmem_hugetlb_setup(struct inode *inode, loff_t size, u64 flags) +{ + struct kvm_gmem_hugetlb *hgmem; + struct hugepage_subpool *spool; + int page_size_log; + struct hstate *h; + long hpages; + + page_size_log = (flags >> KVM_GUEST_MEMFD_HUGE_SHIFT) & KVM_GUEST_MEMFD_HUGE_MASK; + h = hstate_sizelog(page_size_log); + + /* Round up to accommodate size requests that don't align with huge pages */ + hpages = round_up(size, huge_page_size(h)) >> huge_page_shift(h); + + spool = hugepage_new_subpool(h, hpages, hpages, false); + if (!spool) + goto err; + + hgmem = kzalloc(sizeof(*hgmem), GFP_KERNEL); + if (!hgmem) + goto err_subpool; + + inode->i_blkbits = huge_page_shift(h); + + hgmem->h = h; + hgmem->spool = spool; + inode->i_mapping->i_private_data = hgmem; + + return 0; + +err_subpool: + kfree(spool); +err: + return -ENOMEM; +} + static struct inode *kvm_gmem_inode_make_secure_inode(const char *name, loff_t size, u64 flags) { @@ -443,9 +591,13 @@ static struct inode *kvm_gmem_inode_make_secure_inode(const char *name, return inode; err = security_inode_init_security_anon(inode, &qname, NULL); - if (err) { - iput(inode); - return ERR_PTR(err); + if (err) + goto out; + + if (flags & KVM_GUEST_MEMFD_HUGETLB) { + err = kvm_gmem_hugetlb_setup(inode, size, flags); + if (err) + goto out; } inode->i_private = (void *)(unsigned long)flags; @@ -459,6 +611,11 @@ static struct inode *kvm_gmem_inode_make_secure_inode(const char *name, WARN_ON_ONCE(!mapping_unevictable(inode->i_mapping)); return inode; + +out: + iput(inode); + + return ERR_PTR(err); } static struct file *kvm_gmem_inode_create_getfile(void *priv, loff_t size, @@ -526,14 +683,22 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags) return err; } +#define KVM_GUEST_MEMFD_ALL_FLAGS KVM_GUEST_MEMFD_HUGETLB + int kvm_gmem_create(struct kvm *kvm, struct kvm_create_guest_memfd *args) { loff_t size = args->size; u64 flags = args->flags; - u64 valid_flags = 0; - if (flags & ~valid_flags) - return -EINVAL; + if (flags & KVM_GUEST_MEMFD_HUGETLB) { + /* Allow huge page size encoding in flags */ + if (flags & ~(KVM_GUEST_MEMFD_ALL_FLAGS | + (KVM_GUEST_MEMFD_HUGE_MASK << KVM_GUEST_MEMFD_HUGE_SHIFT))) + return -EINVAL; + } else { + if (flags & ~KVM_GUEST_MEMFD_ALL_FLAGS) + return -EINVAL; + } if (size <= 0 || !PAGE_ALIGNED(size)) return -EINVAL;