From patchwork Mon May 4 14:57:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 11526477 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD8341392 for ; Mon, 4 May 2020 15:02:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 78E7B20735 for ; Mon, 4 May 2020 15:02:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="aiDhQg48" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78E7B20735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 82A7E8E0036; Mon, 4 May 2020 11:02:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7DA748E0024; Mon, 4 May 2020 11:02:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6EFBC8E0036; Mon, 4 May 2020 11:02:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id 529818E0024 for ; Mon, 4 May 2020 11:02:03 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 185F9181AEF31 for ; Mon, 4 May 2020 15:02:03 +0000 (UTC) X-FDA: 76779351726.04.skate92_4c08823562e2a X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,alexandre.chartre@oracle.com,,RULES_HIT:30003:30054:30064:30070,0,RBL:156.151.31.86:@oracle.com:.lbl8.mailshell.net-64.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:0,LUA_SUMMARY:none X-HE-Tag: skate92_4c08823562e2a X-Filterd-Recvd-Size: 8788 Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Mon, 4 May 2020 15:02:02 +0000 (UTC) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 044ElbML116466; Mon, 4 May 2020 15:01:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=mpXXd7uEdJ6fmVl0iUxBjNXdKzD8MJRJwDBOk9pZnT0=; b=aiDhQg48J1nTnS19xiB7JOTozw6IIJ6RDVQKmmdGkIqRuLpW8s76/frddUZtuAuBUjPy ElA+W64NLBZsxkRyNsTHCo/mD+yeVv56uxVANbu6S2URk2CukVo65oH/WwumHiwC9XLc QLGhAmMgjaDqXIMK2tNJ7Vprz/i/aInfcGOScTAJSLJvVJE6wJFVgf9hUJd/N0KarHKf iKzeHffR/dJ89EaaBaWr3bOJ5r6XSVp4fiR+K8C2nCHgNcngD6TILIBhQQQ8MlZsG6z+ pXc71NSWiGFr7xuMH6ygsa1m/aagaunI0X0rkwg6+vL6vOEyqgIYgPdHioAuVPhoN8i8 aw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 30s09qyg76-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 May 2020 15:01:42 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 044ElbCv053434; Mon, 4 May 2020 14:59:42 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 30t1r2esd5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 May 2020 14:59:42 +0000 Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 044ExeBx022128; Mon, 4 May 2020 14:59:40 GMT Received: from linux-1.home.com (/10.175.9.166) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 May 2020 07:59:40 -0700 From: Alexandre Chartre To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: pbonzini@redhat.com, konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, junaids@google.com, graf@amazon.de, rppt@linux.vnet.ibm.com, kuzuno@gmail.com, mgross@linux.intel.com, alexandre.chartre@oracle.com Subject: [RFC v4][PATCH part-2 02/13] mm/dpt: Track buffers allocated for a decorated page-table Date: Mon, 4 May 2020 16:57:59 +0200 Message-Id: <20200504145810.11882-3-alexandre.chartre@oracle.com> X-Mailer: git-send-email 2.18.2 In-Reply-To: <20200504145810.11882-1-alexandre.chartre@oracle.com> References: <20200504145810.11882-1-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9610 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 adultscore=0 suspectscore=2 spamscore=0 mlxlogscore=999 malwarescore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2005040123 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9610 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 clxscore=1015 suspectscore=2 priorityscore=1501 malwarescore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2005040123 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add functions to track buffers allocated for a decorated page-table. A page-table can have direct references to the kernel page table, at different levels (PGD, P4D, PUD, PMD). When freeing a page-table, we should make sure that we free parts actually allocated for the decorated page-table, and not parts of the kernel page table referenced from the page-table. To do so, we will keep track of buffers when building the page-table. Signed-off-by: Alexandre Chartre --- arch/x86/include/asm/dpt.h | 21 ++++++++++ arch/x86/mm/dpt.c | 82 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+) diff --git a/arch/x86/include/asm/dpt.h b/arch/x86/include/asm/dpt.h index 1da4d43d5e94..b9cba051ebf2 100644 --- a/arch/x86/include/asm/dpt.h +++ b/arch/x86/include/asm/dpt.h @@ -3,9 +3,18 @@ #define ARCH_X86_MM_DPT_H #include +#include #include +enum page_table_level { + PGT_LEVEL_PTE, + PGT_LEVEL_PMD, + PGT_LEVEL_PUD, + PGT_LEVEL_P4D, + PGT_LEVEL_PGD +}; + /* * A decorated page-table (dpt) encapsulates a native page-table (e.g. * a PGD) and maintain additional attributes related to this page-table. @@ -15,6 +24,18 @@ struct dpt { pgd_t *pagetable; /* the actual page-table */ unsigned int alignment; /* page-table alignment */ + /* + * A page-table can have direct references to another page-table, + * at different levels (PGD, P4D, PUD, PMD). When freeing or + * modifying a page-table, we should make sure that we free/modify + * parts effectively allocated to the actual page-table, and not + * parts of another page-table referenced from this page-table. + * + * To do so, the backend_pages XArray is used to keep track of pages + * used for this page-table. + */ + struct xarray backend_pages; /* page-table pages */ + unsigned long backend_pages_count; /* pages count */ }; extern struct dpt *dpt_create(unsigned int pgt_alignment); diff --git a/arch/x86/mm/dpt.c b/arch/x86/mm/dpt.c index 333e259c5b7f..6df2d4fde8ec 100644 --- a/arch/x86/mm/dpt.c +++ b/arch/x86/mm/dpt.c @@ -8,6 +8,80 @@ #include +/* + * Get the pointer to the beginning of a page table directory from a page + * table directory entry. + */ +#define DPT_BACKEND_PAGE_ALIGN(entry) \ + ((typeof(entry))(((unsigned long)(entry)) & PAGE_MASK)) + +/* + * Pages used to build a page-table are stored in the backend_pages XArray. + * Each entry in the array is a logical OR of the page address and the page + * table level (PTE, PMD, PUD, P4D) this page is used for in the page-table. + * + * As a page address is aligned with PAGE_SIZE, we have plenty of space + * for storing the page table level (which is a value between 0 and 4) in + * the low bits of the page address. + * + */ + +#define DPT_BACKEND_PAGE_ENTRY(addr, level) \ + ((typeof(addr))(((unsigned long)(addr)) | ((unsigned long)(level)))) +#define DPT_BACKEND_PAGE_ADDR(entry) \ + ((void *)(((unsigned long)(entry)) & PAGE_MASK)) +#define DPT_BACKEND_PAGE_LEVEL(entry) \ + ((enum page_table_level)(((unsigned long)(entry)) & ~PAGE_MASK)) + +static int dpt_add_backend_page(struct dpt *dpt, void *addr, + enum page_table_level level) +{ + unsigned long index; + void *old_entry; + + if ((!addr) || ((unsigned long)addr) & ~PAGE_MASK) + return -EINVAL; + + lockdep_assert_held(&dpt->lock); + index = dpt->backend_pages_count; + + old_entry = xa_store(&dpt->backend_pages, index, + DPT_BACKEND_PAGE_ENTRY(addr, level), + GFP_KERNEL); + if (xa_is_err(old_entry)) + return xa_err(old_entry); + if (old_entry) + return -EBUSY; + + dpt->backend_pages_count++; + + return 0; +} + +/* + * Check if an offset in the page-table is valid, i.e. check that the + * offset is on a page effectively belonging to the page-table. + */ +static bool dpt_valid_offset(struct dpt *dpt, void *offset) +{ + unsigned long index; + void *addr, *entry; + bool valid; + + addr = DPT_BACKEND_PAGE_ALIGN(offset); + valid = false; + + lockdep_assert_held(&dpt->lock); + xa_for_each(&dpt->backend_pages, index, entry) { + if (DPT_BACKEND_PAGE_ADDR(entry) == addr) { + valid = true; + break; + } + } + + return valid; +} + /* * dpt_create - allocate a page-table and create a corresponding * decorated page-table. The page-table is allocated and aligned @@ -41,6 +115,7 @@ struct dpt *dpt_create(unsigned int pgt_alignment) dpt->alignment = pgt_alignment; spin_lock_init(&dpt->lock); + xa_init(&dpt->backend_pages); return dpt; } @@ -50,10 +125,17 @@ void dpt_destroy(struct dpt *dpt) { unsigned int pgt_alignment; unsigned int alloc_order; + unsigned long index; + void *entry; if (!dpt) return; + if (dpt->backend_pages_count) { + xa_for_each(&dpt->backend_pages, index, entry) + free_page((unsigned long)DPT_BACKEND_PAGE_ADDR(entry)); + } + if (dpt->pagetable) { pgt_alignment = dpt->alignment; alloc_order = round_up(PAGE_SIZE + pgt_alignment,