From patchwork Thu Jun 9 10:18:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12875171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6CB3C433EF for ; Thu, 9 Jun 2022 10:18:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.344895.570514 (Exim 4.92) (envelope-from ) id 1nzFFZ-0002qj-7k; Thu, 09 Jun 2022 10:18:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 344895.570514; Thu, 09 Jun 2022 10:18:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nzFFZ-0002qc-40; Thu, 09 Jun 2022 10:18:49 +0000 Received: by outflank-mailman (input) for mailman id 344895; Thu, 09 Jun 2022 10:18:48 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nzFFX-000264-OF for xen-devel@lists.xenproject.org; Thu, 09 Jun 2022 10:18:47 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on0625.outbound.protection.outlook.com [2a01:111:f400:fe0d::625]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8c3db64c-e7dd-11ec-b605-df0040e90b76; Thu, 09 Jun 2022 12:18:46 +0200 (CEST) Received: from VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) by DU2PR04MB8790.eurprd04.prod.outlook.com (2603:10a6:10:2e1::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.13; Thu, 9 Jun 2022 10:18:45 +0000 Received: from VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b]) by VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b%7]) with mapi id 15.20.5332.013; Thu, 9 Jun 2022 10:18:45 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8c3db64c-e7dd-11ec-b605-df0040e90b76 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KzbRGGGfZwiRpIwszqMeYG8MPArqTM5DUedH7Qsip/Lvz9fntrSo5qK6j3Y3+iNMm7flGN8k5VZWUcmNjAVHOMpFjYdxJNH9a3sfksECBRoGZzNOOUDvichbN0SdFfgcht3B+Jb3xVA8dM1h5C4ECSW9rKmJatknbn2g/uIsalNo65OSOQvSxPYcUpcRU7Eik3kEIJJEDzt12bgHFWCmg8J7hcIHy+z9VQvBWJgx7aHlcxedXqpuBj7D+Fnems3v20HzUtx5tOWT8t3ClO4s8zxuFlzLnRuPpkh6yySlTS0LHnIszx+TYIil4c7zpP8/DA+PB/MARAVrmIxAdxC1hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nzH6EFMW+e3+Xs7xettJZg30tav9rjlAdUzEqGTjlFU=; b=WZq5xNk8k/aZ3WAC/pYR7TnmZ9xZMUSFIMmIaPf57q9RAoDl1q3HjOqTpa8QBgos0DCAKwF1OsEHzHpNIdDh1bDsLbJiAVmZMOUox8USRh5GUdERceYfuXJUbOxUss4dtE2OLJF0NVBHkO3jGlma14Kl9WmfNgNzUn4flyvGSc2bPhBYVA+H710l6I1boKDcJT9MctbhldV3gbiPyAkxWw9YgYLczhfZlQjKPHrJ0q5e80l55A32363DYcp9p0IjfUlZOpuSmuoQW4LpqLPXUqvGUSuPItzbeKdOU3qzm4CpMqF9XHfMLafTLzwC6eAL2eaHYyREMLyLLj9Tvcbpww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nzH6EFMW+e3+Xs7xettJZg30tav9rjlAdUzEqGTjlFU=; b=4ZIAI3IbTXjtYrTEodr/B8hdhaiWqSTIbRjV4Z4RitEQjM5oW1qIzcdvimWGLFdUreD28VPzgpq8m6BSTjTgUY9VlYtFkny9QuQ1zQ+pwvWQagjc0LagpCDTP2l/CSJDnJcYPiR2R4Wlo4pZHblefDpvKMwJBNwCDH64AMOvJe6kWFJoRptRvz+cVo7XDMI3DM1XPStpgfFP3EQ3SJLsiOKDPP0GqVZHjeJvDd2+sCgo2KsL48r02hJvzFP3OgTt9JIZzCJ4Tmhvn29yYVyCpVEvZqCIl41QjWP683KmclSEmq2vdajNomjcnm41+Rcjmu9h2XNJ8xOFbAkopo0zAw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <4932f8c0-26e5-4289-ef7b-6bba27a9216c@suse.com> Date: Thu, 9 Jun 2022 12:18:43 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v6 05/12] x86: introduce helper for recording degree of contiguity in page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM7PR02CA0029.eurprd02.prod.outlook.com (2603:10a6:20b:100::39) To VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e503443c-0bb0-4455-9fcf-08da4a016f84 X-MS-TrafficTypeDiagnostic: DU2PR04MB8790:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5MoHlvozFHMcq9CkQ4I7r6cMveOku/ksD7Vy+IoIccFLbt5RgtEZOyFAIACS3pQsuqyHUHsqLcYnguZC8Q228BrumWB3OX0jEMAtmJcRBoFVhhVKtpQR8jb0fM9Z1VNEfFwD4S7SpJl77AFzCn1LdO++RmURUKfd2qRv+ZZ4Nx3Jv8WJlWWw86c2XyB3wpwoExRRSPxkLwhbnarTqnO7Jh8jB5y6wZRA/CjnvY7leO3Hlm9cSI2J3boSpiyL5dbUILq8mG6gC7mzcu6Lx0hOk2aPlb1GtsMzRfG68wK9FDL6+IB6s+EOlgswN+9KD1Xacn5i9IlGDPvFO2R9MDWpptJw/6ZkoRtBO6W3hCBMmMy2FRG4M/N6TVT6lNBn5j8ya/HaH/SKm03MVTLqpkYb0AkvisBUps0xbfKyXb29yypNn4Fx3V/1fEjcobY35M+Nk4ztZx4cI/YD9VwSUAkBuqjx0snTk7MUPhFzcVv/s+Na08gj27xLVkK9sVyr4I7GKtXsPt3EGGtvcZ3DLder9JnPgmrNLTAEvOXcU17XbdggLvpdSt/UwR6VQzExtRotxQGc6rxtcW7k4nKheYkkIgNvFPdt19QXqEy5HrFNgj0oV0DdWkD/1uxw4tTk6xoOXogaIkFeH4Ne6xIlAqa527jIkVVvxUrqlfIyuH3dzIsamFbsFQKkYIFavbQhEj42yN7xGNEXvRTvr6c2/N/haCUVhEw6c5qi6+k3cznW74XQO5gRfWLqomA6DQ+ggSGB X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR04MB6560.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(26005)(6512007)(31696002)(6486002)(6506007)(5660300002)(186003)(66946007)(83380400001)(2906002)(86362001)(508600001)(8936002)(36756003)(2616005)(38100700002)(66476007)(66556008)(8676002)(4326008)(6916009)(316002)(54906003)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?HLZFygntkrlgxJJ3SJ75oeBUZwxm?= =?utf-8?q?93Y+/F/2N1IOXUgk8fh8/04xyT9J2c6s6jl21FDnAm4OJ9oyln6HI3wTCMTWDKUYW?= =?utf-8?q?8GV8lP56M2x4weKwgVzAOTJ1phNuGk+ef/TR3kL15VGaTZx7+PhDfTPMi5lE0qrYm?= =?utf-8?q?0Ugdwtq+Peh6nZYFu3luavYRqOicNYYBCjGfhBcmulu1NIYiicNWzCyqlYnOtZTOx?= =?utf-8?q?aalspUVymN9WhRtYt/v/CdKy6fpYxNtbBeCy3ng+v2pzhMAQ4T7bJNsnbpjEVzIgI?= =?utf-8?q?JGhekBhRLqfZyO6U4khrj8pj8eFQ4XSBn4i5uDdyoiDNAEvY4XBA/l4VYbV26rjfp?= =?utf-8?q?x0try9TXK6DDQMyncdW9lXOm2oxbNrh5WQaYJ2b7awEE3spuEv547yS5Vg82yYJXO?= =?utf-8?q?qH0S1JGc3zfr3IBnwJl7edW2WReCOMAKyIRooFkexhDZnHI3RPYutl825JBkifLsT?= =?utf-8?q?cMvyKk6DriAipumZ/p2EmCO/P7J6HhjUZj+Kb5yY11zQIMg0ZlRaYPlnS8aZMrxVW?= =?utf-8?q?4ik5Wm0yLe37gLOYy+1XfNeGU1//0vO8dzbk1d6QlzSyt/cazQZAk9D+icQZpjYOT?= =?utf-8?q?8bRPduays1Kj242ViQ2/qxSAu0nwF3FJJC51b8BhOEKnExhGM9EzAm1+1ShI1Qj2V?= =?utf-8?q?i5AP6jF8OuGEjksN1/aAi5LPEevaRGD+cYQkereJ0OmtHimoF/WCAQvwEx5ETfsy/?= =?utf-8?q?DBgmRaIILXgTDKw/M5vFgEeOYthJjr9uEwHB2YF9cR+ujnjEcdVSVyUO8ETs34J4v?= =?utf-8?q?7HK7f5A1JMIyNKTob7dYiJtBlBhzjtpnUT1HCHSei5dEvo4fQrNIEgjOxlGkXOgs8?= =?utf-8?q?l+V6CfuqkmVsouTQ1GQyKzsV4eXD7daUK85Wp72OnFGuCzTLwJbR+6mOLCFtUWRLo?= =?utf-8?q?fENE65+ym+TuMblyQhuxilUOstMmFQWxXvn7b9n+SNo7bkEHGNSRI/6y9aVUvw3ck?= =?utf-8?q?yZyBunDGJOlCJiY/5gsFH0hw0hJro7bhejU+lgd88IrF/Sq2NmfLVW8W2nyP6PzH0?= =?utf-8?q?THen3rT1vq6ss98SuhmMk6RzgTejuqlOMakK/+mkxSFkfadwGBm9yhlR+CJDFFnSU?= =?utf-8?q?symsbMDKY9sk1QBCnzXUvKvFiF5PijxNMCGHoCYGA0ccCBeKoV74Jho77CtYfUNVa?= =?utf-8?q?lJA/3Quw9+Nc6kmdOzS7Fb/gyl1Z3MG0BMjN9Wgx//VVH+D+C+JHvN6T3Uh0Z79tX?= =?utf-8?q?crrfC9URYmF1uXAxy22l6pzCLzo4HEqw4bk8DBTigAX0vS+jaqfPqd2JWmLd2hA9U?= =?utf-8?q?pTePmgS8YnMrsAV6YXyYXHY+nrn/LPojjxRM4YkX66B9myg3NV/p8z7foeRbh2Kl1?= =?utf-8?q?9Wk6veaK1T38J093PpjrMe0o6ahEMill/si7nFtdvBW0bhwGDhndKeUvR/DcTyWOQ?= =?utf-8?q?yS1dir8Zw1ksVWm29xsnKzDCxF7lrzyls1G0HDWSwDxJ1Gqmf0X8AILVVUS0HdV4Y?= =?utf-8?q?Au/cSawAmqIZgzyx+cwFxj0t6PvFt8iANezfuZcn29dnHgI1/3sZBCILzA4vP9fPC?= =?utf-8?q?xm4Uq4qL5pwV/xoVNbyUkbuWh8g7yUAp2+InMMD8lKEVwBgjKvy5gY3CU5WmMyRTp?= =?utf-8?q?23tHAx8BBbfzRpSq7SNqXY63mCc3x7bomxnakT1Eyyc7t8tXb12kMkwTiZ0gIFLn3?= =?utf-8?q?70I5g47svnfRhjZW6+U9d8kZD5t6NpjA=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: e503443c-0bb0-4455-9fcf-08da4a016f84 X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2022 10:18:45.3981 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ozcmZ5vgmGKPwYSYcnY6JQ3eJ9ICOyayjUBhirOlZEyIgVol2GxN6TZ+rkZN8oXwO911m/h/2HNJzkn2AlrXqQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8790 This is a re-usable helper (kind of a template) which gets introduced without users so that the individual subsequent patches introducing such users can get committed independently of one another. See the comment at the top of the new file. To demonstrate the effect, if a page table had just 16 entries, this would be the set of markers for a page table with fully contiguous mappings: index 0 1 2 3 4 5 6 7 8 9 A B C D E F marker 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 "Contiguous" here means not only present entries with successively increasing MFNs, each one suitably aligned for its slot, but also a respective number of all non-present entries. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v5: Bail early from step 1 if possible. Arrange for consumers who are just after CONTIG_{LEVEL_SHIFT,NR}. Extend comment. v3: Rename function and header. Introduce IS_CONTIG(). v2: New. --- /dev/null +++ b/xen/arch/x86/include/asm/pt-contig-markers.h @@ -0,0 +1,110 @@ +#ifndef __ASM_X86_PT_CONTIG_MARKERS_H +#define __ASM_X86_PT_CONTIG_MARKERS_H + +/* + * Short of having function templates in C, the function defined below is + * intended to be used by multiple parties interested in recording the + * degree of contiguity in mappings by a single page table. + * + * Scheme: Every entry records the order of contiguous successive entries, + * up to the maximum order covered by that entry (which is the number of + * clear low bits in its index, with entry 0 being the exception using + * the base-2 logarithm of the number of entries in a single page table). + * While a few entries need touching upon update, knowing whether the + * table is fully contiguous (and can hence be replaced by a higher level + * leaf entry) is then possible by simply looking at entry 0's marker. + * + * Prereqs: + * - CONTIG_MASK needs to be #define-d, to a value having at least 4 + * contiguous bits (ignored by hardware), before including this file (or + * else only CONTIG_LEVEL_SHIFT and CONTIG_NR will become available), + * - page tables to be passed to the helper need to be initialized with + * correct markers, + * - not-present entries need to be entirely clear except for the marker. + */ + +/* This is the same for all anticipated users, so doesn't need passing in. */ +#define CONTIG_LEVEL_SHIFT 9 +#define CONTIG_NR (1 << CONTIG_LEVEL_SHIFT) + +#ifdef CONTIG_MASK + +#include +#include +#include + +#define GET_MARKER(e) MASK_EXTR(e, CONTIG_MASK) +#define SET_MARKER(e, m) \ + ((void)((e) = ((e) & ~CONTIG_MASK) | MASK_INSR(m, CONTIG_MASK))) + +#define IS_CONTIG(kind, pt, i, idx, shift, b) \ + ((kind) == PTE_kind_leaf \ + ? (((pt)[i] ^ (pt)[idx]) & ~CONTIG_MASK) == (1ULL << ((b) + (shift))) \ + : !((pt)[i] & ~CONTIG_MASK)) + +enum PTE_kind { + PTE_kind_null, + PTE_kind_leaf, + PTE_kind_table, +}; + +static bool pt_update_contig_markers(uint64_t *pt, unsigned int idx, + unsigned int level, enum PTE_kind kind) +{ + unsigned int b, i = idx; + unsigned int shift = (level - 1) * CONTIG_LEVEL_SHIFT + PAGE_SHIFT; + + ASSERT(idx < CONTIG_NR); + ASSERT(!(pt[idx] & CONTIG_MASK)); + + /* Step 1: Reduce markers in lower numbered entries. */ + while ( i ) + { + b = find_first_set_bit(i); + i &= ~(1U << b); + if ( GET_MARKER(pt[i]) <= b ) + break; + SET_MARKER(pt[i], b); + } + + /* An intermediate table is never contiguous with anything. */ + if ( kind == PTE_kind_table ) + return false; + + /* + * Present entries need in-sync index and address to be a candidate + * for being contiguous: What we're after is whether ultimately the + * intermediate table can be replaced by a superpage. + */ + if ( kind != PTE_kind_null && + idx != ((pt[idx] >> shift) & (CONTIG_NR - 1)) ) + return false; + + /* Step 2: Check higher numbered entries for contiguity. */ + for ( b = 0; b < CONTIG_LEVEL_SHIFT && !(idx & (1U << b)); ++b ) + { + i = idx | (1U << b); + if ( !IS_CONTIG(kind, pt, i, idx, shift, b) || GET_MARKER(pt[i]) != b ) + break; + } + + /* Step 3: Update markers in this and lower numbered entries. */ + for ( ; SET_MARKER(pt[idx], b), b < CONTIG_LEVEL_SHIFT; ++b ) + { + i = idx ^ (1U << b); + if ( !IS_CONTIG(kind, pt, i, idx, shift, b) || GET_MARKER(pt[i]) != b ) + break; + idx &= ~(1U << b); + } + + return b == CONTIG_LEVEL_SHIFT; +} + +#undef IS_CONTIG +#undef SET_MARKER +#undef GET_MARKER +#undef CONTIG_MASK + +#endif /* CONTIG_MASK */ + +#endif /* __ASM_X86_PT_CONTIG_MARKERS_H */