From patchwork Fri Jul 14 13:39:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79945C04FE0 for ; Fri, 14 Jul 2023 14:16:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5EF68D0003; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AC5B08D0007; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C4238D0003; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 720048D0001 for ; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4483514028E for ; Fri, 14 Jul 2023 14:16:33 +0000 (UTC) X-FDA: 81010417866.07.DD5AFEE Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf02.hostedemail.com (Postfix) with ESMTP id 1F97B80030 for ; Fri, 14 Jul 2023 14:16:30 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=keXvIPEO; dmarc=none; spf=none (imf02.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=QqRA1A/aeG7Vp+frAisWOweSXHDCgPVqvl/Ov07GdYA=; b=PaMpMKtHTd8QSc5fiQb3b42w4j9GoRPv+uh8sjBVaN4ClnLC3KfCSC0V0mgZdkESCccHw1 vXv/m3SJuZcxt2cssVEUR2KF5J1RKmz9NTrxkcsVqwXH0RvbbSEhhi05+yBEAqPHssZsGj T2Zj8Jhwcg51CL7zXKI+PIOPANF8smw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=keXvIPEO; dmarc=none; spf=none (imf02.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344191; a=rsa-sha256; cv=none; b=OuTGEjlyMRx4c++icnhwC5x5bqwevkU8RBmQyASLyGjuLUtTOJ8qdsXOTuv2bROwsWEpCX yeONIglCmsSNCZ4wsJ7Fc+fwMD2GVE4wj2WHsK2Cgwu7+AQprtGGVMpdmo6n0ik6+WTHFn +/UFuumkQuR2QE8kLzuGERb2SgzWGy4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=QqRA1A/aeG7Vp+frAisWOweSXHDCgPVqvl/Ov07GdYA=; b=keXvIPEO22MCTYT4gVabzMzK4E 7zZ4B7VY3kbDqlVMOrYGQfJiEOSPC6TYRJwwLI4z77Mas0c6YAf8GNXbwWcnhfyKRjasvonlAqnBW +mJC9B+tXoPm19KZVSUqRxMF7xg73W6zJrsVfUwVqdzrEHWjFQnujsu33OkbGmCFyJwnz5GZzDle7 H9739utpS0sI3pEmgP1kgng0pbxnX6+lYHnPrWYSOAAt7it9tZSJbJHoOuUD8hx0g3LLH0uPTvNvE LhxvjVH8fIzku5xDR10iHOMR6yvT3U+fPYGSL56BOtfdNaCpQgl9mBurFiJ50dh/4e4oPwLfZQc2k fcIiaoxw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qKJah-006Iiv-21; Fri, 14 Jul 2023 14:16:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id A22C4300513; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 5EA7E200D83A3; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141218.679746713@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:00 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 01/10] futex: Clarify FUTEX2 flags References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1F97B80030 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: cjwzgwccc3phcn9so56qqpjw3o64ye45 X-HE-Tag: 1689344190-337648 X-HE-Meta: U2FsdGVkX1+62P+AGjPEX11a8DrV+MhN7dZ7eO+KoDgNr2+32AWcDSWGdl8gHIh20NZzfH8x/2toq6pY6tVq6tY9VXEJJyfq2jhTufzyX13cQjLItnYy4SGR9U/d7sRVvGxH0BfT/me2xZ+GNA8ZgO8sNPtusZWAJR921xJHfhDMfl/e62OG0C21gG72jt3upOnMFFQLniAClwewO4s9hmiLxAPSBnu795+PiWVkkR27zmBSEi2dK2W5ljUVNIGhUrjUqArA1JgBcL8Mp7W4E1K6/MqLaUBxkdy0bWNLkx6vdepez5Xm8dF3wNIvbdj8J8z3Bc2qhfpuwZR8ieH20PW4P/6bZvg3sDPNC4pmNhyMMxNu3L5lrqWEgfZqy9JbWSkLTN2IeVCmvbePraIcZ3DvtZh9ssU26+alEjAHpi/l76NPppHUXwl/4co7JVZg7J7iUUBtbFmHsnY0UrzNTOKLAM4WC34grjgWOyMUwOHABv3IJQKaI7uPoAgh3FrEM6y5JltGReVOj9LIOZGZso4ylqm3pk1yMnLrPSaUQWXGqE7HCc4PcnfL9cd+vRtTOsCgCAYasmECuyF+a+HJuQfNhKiQUvRrkSyAPeVm3/UEPz+UvzaBZXWm3e2WGz0oIu0HkoFkxfYoJuZnJha6gw345iL3L3ULnBpg6z6CfQMT5M+cYv+IqQcyId27LuJKy2AHY6Ts/4XLLCyo1EUwrVIMW47jCsTvfkpqJDOBwaWelkV7nR8LTPh7XhGZXEeLYAw8fok1hbzJ9buPxPEAyXoW4zU4NMynUV1lOoqw0MDMMDv+JlM6br4S2ssQmY6Lm6t479Enh7NdrdwS19YEFDTpSgijPs0wv6cOzaThKafS+d3HmUWF/RrDw7/PhAnGqnsmXoNwsj9IzWFwgh5jkIT7jAgOeFfRZJ58ex0C260t/trj/UsRB8HLjiKb2/bKdN15tfbLtJLiHDIfiKd Qk01H+0t K561mgNDK2TRSsrTfRh1/hlRskzNSxDOehmOho9DhdHcEpxCT7bNJ3xUNEv0HEOQwRXh+IAXiVy3EUe5qO8wBeyGHa8X/KFPxqWXfA0pCP8xGCWfljV8eWO5JqkZUH1+mwvrTulffCWkahna18PW6QoABXZmo1aj7OFp8qNV8RhbWMsc53FDgfx6VuLxJy9sVCd/JsOODTJ1pBnI1xY/uZ7u7iTNEu1p3Ci9MfoN7dAPXoUd4K+nbLZ0p7Z8FiKX4dcqsvDr7FZMnK61BAcWGI5XtdA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: sys_futex_waitv() is part of the futex2 series (the first and only so far) of syscalls and has a flags field per futex (as opposed to flags being encoded in the futex op). This new flags field has a new namespace, which unfortunately isn't super explicit. Notably it currently takes FUTEX_32 and FUTEX_PRIVATE_FLAG. Introduce the FUTEX2 namespace to clarify this Signed-off-by: Peter Zijlstra (Intel) --- include/uapi/linux/futex.h | 16 +++++++++++++--- kernel/futex/syscalls.c | 7 +++---- 2 files changed, 16 insertions(+), 7 deletions(-) --- a/include/uapi/linux/futex.h +++ b/include/uapi/linux/futex.h @@ -44,10 +44,20 @@ FUTEX_PRIVATE_FLAG) /* - * Flags to specify the bit length of the futex word for futex2 syscalls. - * Currently, only 32 is supported. + * Flags for futex2 syscalls. */ -#define FUTEX_32 2 + /* 0x00 */ + /* 0x01 */ +#define FUTEX2_32 0x02 + /* 0x04 */ + /* 0x08 */ + /* 0x10 */ + /* 0x20 */ + /* 0x40 */ +#define FUTEX2_PRIVATE FUTEX_PRIVATE_FLAG + +/* do not use */ +#define FUTEX_32 FUTEX2_32 /* historical accident :-( */ /* * Max numbers of elements in a futex_waitv array --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -183,8 +183,7 @@ SYSCALL_DEFINE6(futex, u32 __user *, uad return do_futex(uaddr, op, val, tp, uaddr2, (unsigned long)utime, val3); } -/* Mask of available flags for each futex in futex_waitv list */ -#define FUTEXV_WAITER_MASK (FUTEX_32 | FUTEX_PRIVATE_FLAG) +#define FUTEX2_MASK (FUTEX2_32 | FUTEX2_PRIVATE) /** * futex_parse_waitv - Parse a waitv array from userspace @@ -205,10 +204,10 @@ static int futex_parse_waitv(struct fute if (copy_from_user(&aux, &uwaitv[i], sizeof(aux))) return -EFAULT; - if ((aux.flags & ~FUTEXV_WAITER_MASK) || aux.__reserved) + if ((aux.flags & ~FUTEX2_MASK) || aux.__reserved) return -EINVAL; - if (!(aux.flags & FUTEX_32)) + if (!(aux.flags & FUTEX2_32)) return -EINVAL; futexv[i].w.flags = aux.flags; From patchwork Fri Jul 14 13:39:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FEB4C001B0 for ; Fri, 14 Jul 2023 14:16:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C73A6B0075; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84EB48D0001; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 715A06B007B; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 621396B0075 for ; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D2C8C8029B for ; Fri, 14 Jul 2023 14:16:28 +0000 (UTC) X-FDA: 81010417656.16.C77E01F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id C39771C0036 for ; Fri, 14 Jul 2023 14:16:26 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=svWkhtHF; dmarc=none; spf=none (imf21.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344186; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=JH41pDo7+Jk4qffVzaO2p6sCO6Nd3BbMmWUDDzgZdtk=; b=8pW4GYtYVjTyt8kEnSgcJMNJBL5yEOC2/OfdOOhyYcbd+Ta9Va/MTgVJLCBBn65m3/Uk5Z 41yGpAcfVv+AU/OQQrBdz7+9qVygma2vCvkx4cIlEClLzZvxTpB/HXm1hAy6uRN/ukpe6H FzwmvUPy/KuVEFVZRvxEApmYy34vHbo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=svWkhtHF; dmarc=none; spf=none (imf21.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344186; a=rsa-sha256; cv=none; b=3g0wf0+K3ukhk0hpIOjPcqc9zYq+HetadWh3rdmgHryqSf8ZD2Q49hpXQmbqcesBBdjaZ+ dLr0h1alnq+KPXovJYAErwP8IbzamYyyhetsIv+Sj9HPWMvAPDP54OPyJbJzoJq05xI7ZG dvTxs0k9SLl4dcgmaAbRNIjcF441LNI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=JH41pDo7+Jk4qffVzaO2p6sCO6Nd3BbMmWUDDzgZdtk=; b=svWkhtHF2Xpvm5S2/jG0ESgpOR TrBT7MXzx1VRkZJl6Z6JFy0pTLtfdLkAEIKsOMTFwQdkIcyy7ECVjFqE40I2iJe1vrTASfja+qtM8 WDZzYWTGTFQE6iD8ojO/zPWX2v5OoL4npt+I6MnajImZXMT3MMTG8Etf217dclTq1BWGZGQyPj8vI cckrPl4lyNHyA0T5FPLyqSrU5OeSoDbhmhEhc3Q+AzAlgex4F5uq1gNKB6i32P1QVfvRGn5o+/Qo9 SmMRrS031NkgiHkFCkG/Lg0t6+kVO2fNi2Q9P/DCCL+60sQ/32+jDCeaEmgLQ+HU5c+7wxd2+P8im H4FxABfQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qKJah-0016z2-4F; Fri, 14 Jul 2023 14:16:15 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id A3DB730057E; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 60EA0213728AD; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141218.746077262@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:01 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 02/10] futex: Extend the FUTEX2 flags References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C39771C0036 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: dex55nwhix8jhf78hyoypy9xr336scmk X-HE-Tag: 1689344186-198602 X-HE-Meta: U2FsdGVkX18q5eRMa+DH9RPTkRj05pneaZrve/R2mhyiokzVQVSsBliW0H+7PkcgXvkKVyyt6IYUUkXsLMytn4tAKKoN50vUV4G4tpiw4HPo2ZLinnXDN15phEmDCbO33O24rRe3wAFIatzLi7Am6MauZEeZE9oXyvDXTmHt4o6ANreBF0CqVxxQNErADWyCLjiBtjDWYOzLolDlUlDUGMuJIceInmr0WaHnIKywDMCjv/QGsmjFERpUSGqr0SWh5vqf1m4Szfz7W058p09Ek3oYazxk8EvcA+n4OTk+hioBWj8SeGCHgETf/Sd4I3yjXmJn3SeCMdlIFHkEaaPYv9dxZPzh1SZuCYnldaOs+Cvk/7dkR6Ka4K8b0Q1CpG6tp82o5lab4VF1GKvrmOhSNNd8cxK9kfJErSPxTpOIokchicJCz1No0l5tTnXl5V8IjQzhwaCbAQuPRqNgHJJ6iwsZbmzA2aodW2ob/2V5BO63SJo+onRAXYiHexra1xreZqja6GHRYx559hD2CZ9jQXC/AvA1j6GCXi+/L7JB0L3HLITCn9TAfLCAGeHApQXbv3wRPnEloKcv89UBFfyAJ841u8aC2duNuriksI9j7jIhTjhmnLIiNZiB5RviwoIjiFeiQcOTR5Re+VAKwTENHq3Sw+CrHX+NRbRCdTST0OAa0aQeAS8bAazx6e0LuMJbHu2Ym+VwzZZ7rBQw6qCn+oDOYFxGvHxmRfJhGLGJzDM7H5raBgaspbkAAunI6ZdCko1f9MYl2WwITydprF+vfIl9BlXIWci1z+KOeaaNFv6Pej23eeHtEhb8BldGeSNLdnxh4NgTg2FTMXaWouOtpDy43oYlB6JZx76VwMiZi5ay0WMQZXeu60ojzRpQU/H1/joqqOaSSuFQ+G8aLe1AW164fIjkWxAV7ktHxL0B+SYv0L+d2ZeIc6HUgnOFXXyT5r2VUNz3J11pd6u7c4Z s78//sR/ ot1Ub4VyGy9ez5Ailo5cazs8JKUZ3nYpH5nf2bxBAsgeGz5tD5WxzCylvyHD8zD4a/b9XdZ1oAIzgzqXyIIDhoFY4bu5tSBTc7gVKcwye0dAatufz4FTFbHJESCZgAMuhvPDkjglMPXfn8zAi5Ve1C5rESitPLK7Eiu4iJwfm6HaaPzfM3eAFNXepVe8TsYf1s7gc9iZdXdMcUWrxfAPe3w4DPeY91r/dMPCbTSwhJTtDoE6LV2OuA7qjywD/psXSHIfBINH0j3oukGz1gSzDrA/vJA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the definition for the missing but always intended extra sizes, and add a NUMA flag for the planned numa extention. Signed-off-by: Peter Zijlstra (Intel) --- include/uapi/linux/futex.h | 7 ++++--- kernel/futex/syscalls.c | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-) --- a/include/uapi/linux/futex.h +++ b/include/uapi/linux/futex.h @@ -46,10 +46,11 @@ /* * Flags for futex2 syscalls. */ - /* 0x00 */ - /* 0x01 */ +#define FUTEX2_8 0x00 +#define FUTEX2_16 0x01 #define FUTEX2_32 0x02 - /* 0x04 */ +#define FUTEX2_64 0x03 +#define FUTEX2_NUMA 0x04 /* 0x08 */ /* 0x10 */ /* 0x20 */ --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -183,7 +183,7 @@ SYSCALL_DEFINE6(futex, u32 __user *, uad return do_futex(uaddr, op, val, tp, uaddr2, (unsigned long)utime, val3); } -#define FUTEX2_MASK (FUTEX2_32 | FUTEX2_PRIVATE) +#define FUTEX2_MASK (FUTEX2_64 | FUTEX2_PRIVATE) /** * futex_parse_waitv - Parse a waitv array from userspace @@ -207,7 +207,7 @@ static int futex_parse_waitv(struct fute if ((aux.flags & ~FUTEX2_MASK) || aux.__reserved) return -EINVAL; - if (!(aux.flags & FUTEX2_32)) + if ((aux.flags & FUTEX2_64) != FUTEX2_32) return -EINVAL; futexv[i].w.flags = aux.flags; From patchwork Fri Jul 14 13:39:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB7C0EB64DC for ; Fri, 14 Jul 2023 14:16:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 692DC8D0006; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 409A88D0005; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14D578D0007; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AC2A78D0006 for ; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 706E414029F for ; Fri, 14 Jul 2023 14:16:33 +0000 (UTC) X-FDA: 81010417866.06.79A1B3F Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf25.hostedemail.com (Postfix) with ESMTP id 4F1F8A001D for ; Fri, 14 Jul 2023 14:16:30 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=k0AkkJTw; dmarc=none; spf=none (imf25.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=98fwNtpV0Bbvya3mBoauq7smkVWN16/6yjAoJPAxLCQ=; b=5rjpPHyeO1bKvMlsko1r4S1LYdaeBHRq+wvJipX8o8PdVvkpbO4W6JkF8sGN2mV3C2jWa0 +Ua7Y60TTPMtorxblphFjJUnztTzMhXnrQyPyaTs+MOVt7p8Wv4GViRoTBdOIGqZBeuayE 2UlWWbmNCOkcoiQmASdXLVmKszOC774= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=k0AkkJTw; dmarc=none; spf=none (imf25.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344191; a=rsa-sha256; cv=none; b=dpA7gFIsipiDDtQKsnYOkNz5jQai2POMkza6JTAWI+5v29RooPbtN9BJoCsV2j1+Kd+M1G FK4AW746muGjt9FmG8WSwqN2QqEUVco2Ja729sqIZGzcJTnfWAzjnRwVygPlbG9YhtMeN3 Bv84OxTtIL/l3fZYeJyiPNRgbwRZi7U= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=98fwNtpV0Bbvya3mBoauq7smkVWN16/6yjAoJPAxLCQ=; b=k0AkkJTwfkefz5URce2ntHKAIC 6CkN91KKm1sEDmU8seNl1uV4Im1FILXye+5BHlydSaZELYf9BNA3/k7+QifPdr1WwzxLzYcvHvf6T P1AJrThlTJ4nWZgI8kec0ij5yNMrSO/t5B/BJ5VDAX+AR7E+PAaN9ykyCq7vT6csjBMqdhA19nELp lEz1sIwnXytxoy8hFGNf0qkpI0+Ch+wz56SHgu5eT13eNOFZ9pHKXoHl5VNSgkeZfGyJrfRhvnzkm 2UMzg2C3u8KfvGSkocm2zB5m0xp474j8eyUMo0BZX3hXkqvYrbCtZ7F7zHwbFS4yTB9IgE5FV80uX dly3b80w==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qKJah-006Iiw-21; Fri, 14 Jul 2023 14:16:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id A0834300362; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 691B3213728B7; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141218.813185695@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:02 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 03/10] futex: Flag conversion References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4F1F8A001D X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 4n3tjpchcn1juk67q3g9ytqa1ryk5ex9 X-HE-Tag: 1689344190-815927 X-HE-Meta: U2FsdGVkX1/FUM+96CxK3MGJcrgG+sW+qGAcJeq6/UyRoXK5/00vSNk9aTQphm5AVIjAuZzmRrqwLCOIOR2F5B2+cW/OsSWNqLIbCWnurvJmydWIggXDAZBs6BLqUUQKIkI8XN10g4tXwnxwq3O7eBzqsKLfnXWwYkf05HKrzNxzlrPdwrDEgBv/L3YkB5klUO+I1TX9PhMigMPfWiZuFrK3qC0QIKfLicTuKwmWxm2yvMRtpeLlWlpNhvbapMc1R/LkFDvp5nQTfuWZ+yq5QAPgQwL+KrXasugrEXyw0gisvdGxtUFydUT3ZyLEsDQkxf1kID9rIU5JOT+9+aPC/5hUpgEZje4I7nwXJ861ZofU8Vksazqzxo4ajyEq39r+3fSOqw+pVgavjkY6lXwEfXiidqzSl49U2462RXOQBhGzUijNjBg2qjHMrBJNAd6DFJooUlUMMKR5sESnJIItfQu+K7F5sXIq/RtJ8AZwO9aZvQaJEzpYH3ey5Acn37CmTklqyoHehhb3lgei4YjOlV1r43W1ZADDdUyf2UbWvJo70Z2MAsguFzfhIrUptfOj0NglOFpDdwTeJ5+HlKYzILcN7fBNudbeKYiqs05zco+ChE+VoHou2CNniD6NxPNjxAwly0LnUpVR6U2TPpbbsA+E/X2MzJjxKZLJXv9UFMd4J06hYLiVhkdMwPH0cHoywai8kAkiA+lvGiIwTUajIQf/lv/NzrzSsoWV58xmm+n3RF/A75RpkokSPtfo7eXjX4DgjbtUvbgzkxOfChMSJfKL0jJFvtKe9HqQcmCahOrxieW+JNYFTydp7JV+idRR53IljWZBaWzIO48U0r5ksYVJgvtWCjN48xzfUu2L5V81EG4h5ehmcnk+FsTY4lDxuXeCgXa3pkbkZ3GDvQrPyKfeL8Hx7W8QchziIpE1tGzS+M7vFJTY/OSL/tSC43fFNHdTCTqc8rPbxo06O+I sYIl1/Sy HmWoR8ZV9csyoN8uSgYhlu3Bn60CcBvcinFcfZoljkwTJQAtU95Zj9nY5DMw+3hfSOXGTabwO0Mhtc0cZmJxXEUAqg+F4UMlXxYQCaxiemBMUbswA90VC1u4of7agW54pP05f5/nVdB7xVk+66XUS+tv947ac5iQ1L/p/w6ocIfYIjq0Mlgr+9HrLUMp7yIgQkaqi3zjTTuiAcFA61jJU0ruwh0i9DuFqN8OLmAP2U8jnTvwZYZxhDGFd5SYJ1vrz957PJhoOBw/eeKWA61dF5P+b4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Futex has 3 sets of flags: - legacy futex op bits - futex2 flags - internal flags Add a few helpers to convert from the API flags into the internal flags. Signed-off-by: Peter Zijlstra (Intel) --- kernel/futex/futex.h | 48 +++++++++++++++++++++++++++++++++++++++++++++--- kernel/futex/syscalls.c | 21 +++++++++++++-------- kernel/futex/waitwake.c | 4 ++-- 3 files changed, 60 insertions(+), 13 deletions(-) --- a/kernel/futex/futex.h +++ b/kernel/futex/futex.h @@ -16,8 +16,15 @@ * Futex flags used to encode options to functions and preserve them across * restarts. */ +#define FLAGS_SIZE_8 0x00 +#define FLAGS_SIZE_16 0x01 +#define FLAGS_SIZE_32 0x02 +#define FLAGS_SIZE_64 0x03 + +#define FLAGS_SIZE_MASK 0x03 + #ifdef CONFIG_MMU -# define FLAGS_SHARED 0x01 +# define FLAGS_SHARED 0x10 #else /* * NOMMU does not have per process address space. Let the compiler optimize @@ -25,8 +32,43 @@ */ # define FLAGS_SHARED 0x00 #endif -#define FLAGS_CLOCKRT 0x02 -#define FLAGS_HAS_TIMEOUT 0x04 +#define FLAGS_CLOCKRT 0x20 +#define FLAGS_HAS_TIMEOUT 0x40 +#define FLAGS_NUMA 0x80 + +/* FUTEX_ to FLAGS_ */ +static inline unsigned int futex_to_flags(unsigned int op) +{ + unsigned int flags = FLAGS_SIZE_32; + + if (!(op & FUTEX_PRIVATE_FLAG)) + flags |= FLAGS_SHARED; + + if (op & FUTEX_CLOCK_REALTIME) + flags |= FLAGS_CLOCKRT; + + return flags; +} + +/* FUTEX2_ to FLAGS_ */ +static inline unsigned int futex2_to_flags(unsigned int flags2) +{ + unsigned int flags = flags2 & FUTEX2_64; + + if (!(flags2 & FUTEX2_PRIVATE)) + flags |= FLAGS_SHARED; + + if (flags2 & FUTEX2_NUMA) + flags |= FLAGS_NUMA; + + return flags; +} + +static inline unsigned int futex_size(unsigned int flags) +{ + unsigned int size = flags & FLAGS_SIZE_MASK; + return 1 << size; /* {0,1,2,3} -> {1,2,4,8} */ +} #ifdef CONFIG_FAIL_FUTEX extern bool should_fail_futex(bool fshared); --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -85,15 +85,12 @@ SYSCALL_DEFINE3(get_robust_list, int, pi long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout, u32 __user *uaddr2, u32 val2, u32 val3) { + unsigned int flags = futex_to_flags(op); int cmd = op & FUTEX_CMD_MASK; - unsigned int flags = 0; - if (!(op & FUTEX_PRIVATE_FLAG)) - flags |= FLAGS_SHARED; - - if (op & FUTEX_CLOCK_REALTIME) { - flags |= FLAGS_CLOCKRT; - if (cmd != FUTEX_WAIT_BITSET && cmd != FUTEX_WAIT_REQUEUE_PI && + if (flags & FLAGS_CLOCKRT) { + if (cmd != FUTEX_WAIT_BITSET && + cmd != FUTEX_WAIT_REQUEUE_PI && cmd != FUTEX_LOCK_PI2) return -ENOSYS; } @@ -201,6 +198,8 @@ static int futex_parse_waitv(struct fute unsigned int i; for (i = 0; i < nr_futexes; i++) { + unsigned int bits, flags; + if (copy_from_user(&aux, &uwaitv[i], sizeof(aux))) return -EFAULT; @@ -210,7 +209,13 @@ static int futex_parse_waitv(struct fute if ((aux.flags & FUTEX2_64) != FUTEX2_32) return -EINVAL; - futexv[i].w.flags = aux.flags; + flags = futex2_to_flags(aux.flags); + bits = 8 * futex_size(flags); + + if (bits < 64 && aux.val >> bits) + return -EINVAL; + + futexv[i].w.flags = flags; futexv[i].w.val = aux.val; futexv[i].w.uaddr = aux.uaddr; futexv[i].q = futex_q_init; --- a/kernel/futex/waitwake.c +++ b/kernel/futex/waitwake.c @@ -419,11 +419,11 @@ static int futex_wait_multiple_setup(str */ retry: for (i = 0; i < count; i++) { - if ((vs[i].w.flags & FUTEX_PRIVATE_FLAG) && retry) + if (!(vs[i].w.flags & FLAGS_SHARED) && retry) continue; ret = get_futex_key(u64_to_user_ptr(vs[i].w.uaddr), - !(vs[i].w.flags & FUTEX_PRIVATE_FLAG), + vs[i].w.flags & FLAGS_SHARED, &vs[i].q.key, FUTEX_READ); if (unlikely(ret)) From patchwork Fri Jul 14 13:39:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92420EB64DC for ; Fri, 14 Jul 2023 14:16:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0621E8D0002; Fri, 14 Jul 2023 10:16:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE0338D0001; Fri, 14 Jul 2023 10:16:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5AA58D0002; Fri, 14 Jul 2023 10:16:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B78238D0001 for ; Fri, 14 Jul 2023 10:16:31 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B5ED7C02A5 for ; Fri, 14 Jul 2023 14:16:28 +0000 (UTC) X-FDA: 81010417656.21.F29AA8F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 170BF180020 for ; Fri, 14 Jul 2023 14:16:24 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Ex87tFfx; dmarc=none; spf=none (imf24.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344185; a=rsa-sha256; cv=none; b=cYPfOY0QGNXGBg4dNAaqato/qCcjk+Rexo+jhKzVEaFVkHhmua9TRP/RL92p1NR0qcwZQ2 TKzJnuNAW+bHBLFVps3RCh6es83j/2AHLFk+U5Vrv9+snxE260W5LvbazLFF/l/RMRoX1N hJ3AQ6RwJYmGO896XGhJYZtEV+sS9Hk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Ex87tFfx; dmarc=none; spf=none (imf24.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344185; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=f9cIjtSvczJQzaoy/33m3QuVdm29wDRZG/Co3tko+Xc=; b=wgcTzSdUKRexOqxKfNZHjVthsvkAK9p0b8bSOPPyx/BhWIvokJg+ORKYhQWojsGYzsr35p 54qShJuWBc2P4s6YH3JBJY+Yeu5cNhSOfxZXsV0imJUznLqixsKoOXngcm6jS8D9GpSI/y odhaFYF8fvzTkANAMr3CBfL0HnQrwnk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=f9cIjtSvczJQzaoy/33m3QuVdm29wDRZG/Co3tko+Xc=; b=Ex87tFfxAGzCWArXtjjyD9/4ya nGclUVR+Zd0W0YM1P1AJRTaPXoWuAwIlPYNAyOLp52E2w1jE7yxw+UPqs0Whz/O9xNFK7IdnY0TVn XDoZNGxQMaTdB3YsUFOmbmq8L6mxzIcN0A2AiGnZdPV06KYiEnSrR8K9VhFnMnkB7pBs2UxnETzxY Fr3THWigl3xbUumsm8PutaMhgykj9zjmp7oyBoVLBq3NKWRSAjQMgDL441ZEc5Qi+D8C+VAL5Rdbk f+7OxaLt8zKZtoX9fJIP01ADwQn/Ki/S3hPyfyCyCm3jKe1WhdOrcjSSILHp6hwRXv9dqwqrePlFK HYWPxzxQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qKJah-0016z0-49; Fri, 14 Jul 2023 14:16:15 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 9ED6F3002CE; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 6CBBC213728B0; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141218.879715585@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:03 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 04/10] futex: Add sys_futex_wake() References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 170BF180020 X-Stat-Signature: hfydu5q45xuwg7cqqhftfkxxptt61p6n X-HE-Tag: 1689344184-215142 X-HE-Meta: U2FsdGVkX1/gxtSwynO668j6oMqo+is4Dp5E8lwV1yz+Px92tbcmxOWrQ8Lp56G8uxxPewuRaov3KEB1n5OuQuszXEgQ1mypWZzskkmFlcdiLINSIcvDPJdyqCvzeb7HZ/s9eHdF0n9cwPNelnNSN008aQudlKgQFwzoY1sic7jjvJq4yfubDg+rC4Jh3t9Wu5dFGNtnP7RnEg77f7LquXmd8LrF3u8sdcefh1Tr5qfAcJwMEXR5xZz6m8ECjvqzkjMmEIeGdDaeYI0q21o1BeB3Tr8UPFEcBhWpJtaiN8FYPGahf1C2lMR/LOnqU9kUhxKjUMe/ROLOOctayc3xH7SVVfagJP0yGSlGTko1Jt24haiAj6PNiSGSIdmdlwnH+lOBi7iid4deQnnbHE6IaRt8kp28Ik+IJ9cViaDRXZj2Dk6rW2/razFsv2EdVsNAYgqHzLbXCdr0IiZaD4LtRk3o7oi4ECTYqnEe/ww2726RCrtyqnMyokl9ZCIui+oq/pSsQktOMJQn11rWIRYnWpzYNUFZLIVHOoWfHAY6RFd+VW6J3zyG954TM522lHGqcD0JJ9S6RtdHtI0Pc0GBkpYoMDxv7FAxgrfg/PflaSiM05kktmahDNabEwojoud3aL/vpUGQTFYsDRtRlILoXaUpllglDTnWaIx6qe5ePGdtY4yJHvBgkd7WkUE+UXHA9EAXLJyWhETVkxCYm5iugaMhL8OL5DsGiyW3KrFOML3qjO0vVUy6qJopYuVzTl0OWBz4LIqTyBMMkoQfimfT9TOp3hknx1n5OxM7ttaiqeoJDF6ieuTiWfjccc427rKglEyIH77h/ucSy9vJ8cgAcRd/OPmUAUu8XVvA7a/3IvWLWvHSyDZawYcaRRPTBLHw3Im51ymT0YDqiBw3SVWPMs017hqDHyCMIPTt7tBaWEntjDM04xGq7tBtr9iXlfMgvKX6Fzh7Hx64dnzMNax 2KTks9yM GWpKuofiXCniKdqve6Xkd1ni/SzUsv/eTgDziaaM9NCwr9E2By7dTLdUGrivem7PgeHxs7oRt0/3ueNZM9WGtegv9G0VTB+kOwTq+rZ8/ysOtLG7qmeoVLX4ILIp7rsLFmM9U/bIVVF8xV4lJ3rkMT+1HJ28AF24X7FHnUPKwM/4Gd0KKbtrh5VHMvfrmwSVH3ozdPh9scYmTYzatH/O1fkLr4VIePRwsv+wzLeEbTPuAttDqXZm7ZAtAKRtdTZ8ezYQdbFTEn+681mkKRodMfG+Rc4B7bZeW7pLx4JEsYc7U+Z8sEwdxf5jeoS9LAKwPu7DJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To complement sys_futex_waitv() add sys_futex_wake(). Together they provide the basic Futex2 WAIT/WAKE functionality. Signed-off-by: Peter Zijlstra (Intel) --- arch/alpha/kernel/syscalls/syscall.tbl | 1 arch/arm/tools/syscall.tbl | 1 arch/arm64/include/asm/unistd32.h | 2 + arch/ia64/kernel/syscalls/syscall.tbl | 1 arch/m68k/kernel/syscalls/syscall.tbl | 1 arch/microblaze/kernel/syscalls/syscall.tbl | 1 arch/mips/kernel/syscalls/syscall_n32.tbl | 1 arch/mips/kernel/syscalls/syscall_n64.tbl | 1 arch/mips/kernel/syscalls/syscall_o32.tbl | 1 arch/parisc/kernel/syscalls/syscall.tbl | 1 arch/powerpc/kernel/syscalls/syscall.tbl | 1 arch/s390/kernel/syscalls/syscall.tbl | 1 arch/sh/kernel/syscalls/syscall.tbl | 1 arch/sparc/kernel/syscalls/syscall.tbl | 1 arch/x86/entry/syscalls/syscall_32.tbl | 1 arch/x86/entry/syscalls/syscall_64.tbl | 1 arch/xtensa/kernel/syscalls/syscall.tbl | 1 include/linux/syscalls.h | 3 ++ include/uapi/asm-generic/unistd.h | 5 ++- kernel/futex/syscalls.c | 37 ++++++++++++++++++++++++++++ kernel/sys_ni.c | 1 21 files changed, 62 insertions(+), 2 deletions(-) --- a/arch/alpha/kernel/syscalls/syscall.tbl +++ b/arch/alpha/kernel/syscalls/syscall.tbl @@ -491,3 +491,4 @@ 559 common futex_waitv sys_futex_waitv 560 common set_mempolicy_home_node sys_ni_syscall 561 common cachestat sys_cachestat +562 common futex_wake sys_futex_wake --- a/arch/arm/tools/syscall.tbl +++ b/arch/arm/tools/syscall.tbl @@ -465,3 +465,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/arm64/include/asm/unistd32.h +++ b/arch/arm64/include/asm/unistd32.h @@ -909,6 +909,8 @@ __SYSCALL(__NR_futex_waitv, sys_futex_wa __SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node) #define __NR_cachestat 451 __SYSCALL(__NR_cachestat, sys_cachestat) +#define __NR_futex_wake 452 +__SYSCALL(__NR_futex_wake, sys_futex_wake) /* * Please add new compat syscalls above this comment and update --- a/arch/ia64/kernel/syscalls/syscall.tbl +++ b/arch/ia64/kernel/syscalls/syscall.tbl @@ -372,3 +372,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/m68k/kernel/syscalls/syscall.tbl +++ b/arch/m68k/kernel/syscalls/syscall.tbl @@ -451,3 +451,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/microblaze/kernel/syscalls/syscall.tbl +++ b/arch/microblaze/kernel/syscalls/syscall.tbl @@ -457,3 +457,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/mips/kernel/syscalls/syscall_n32.tbl +++ b/arch/mips/kernel/syscalls/syscall_n32.tbl @@ -390,3 +390,4 @@ 449 n32 futex_waitv sys_futex_waitv 450 n32 set_mempolicy_home_node sys_set_mempolicy_home_node 451 n32 cachestat sys_cachestat +452 n32 futex_wake sys_futex_wake --- a/arch/mips/kernel/syscalls/syscall_n64.tbl +++ b/arch/mips/kernel/syscalls/syscall_n64.tbl @@ -366,3 +366,4 @@ 449 n64 futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 n64 cachestat sys_cachestat +452 n64 futex_wake sys_futex_wake --- a/arch/mips/kernel/syscalls/syscall_o32.tbl +++ b/arch/mips/kernel/syscalls/syscall_o32.tbl @@ -439,3 +439,4 @@ 449 o32 futex_waitv sys_futex_waitv 450 o32 set_mempolicy_home_node sys_set_mempolicy_home_node 451 o32 cachestat sys_cachestat +452 o32 futex_wake sys_futex_wake --- a/arch/parisc/kernel/syscalls/syscall.tbl +++ b/arch/parisc/kernel/syscalls/syscall.tbl @@ -450,3 +450,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/powerpc/kernel/syscalls/syscall.tbl +++ b/arch/powerpc/kernel/syscalls/syscall.tbl @@ -538,3 +538,4 @@ 449 common futex_waitv sys_futex_waitv 450 nospu set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/s390/kernel/syscalls/syscall.tbl +++ b/arch/s390/kernel/syscalls/syscall.tbl @@ -454,3 +454,4 @@ 449 common futex_waitv sys_futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat sys_cachestat +452 common futex_wake sys_futex_wake sys_futex_wake --- a/arch/sh/kernel/syscalls/syscall.tbl +++ b/arch/sh/kernel/syscalls/syscall.tbl @@ -454,3 +454,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/sparc/kernel/syscalls/syscall.tbl +++ b/arch/sparc/kernel/syscalls/syscall.tbl @@ -497,3 +497,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/arch/x86/entry/syscalls/syscall_32.tbl +++ b/arch/x86/entry/syscalls/syscall_32.tbl @@ -456,3 +456,4 @@ 449 i386 futex_waitv sys_futex_waitv 450 i386 set_mempolicy_home_node sys_set_mempolicy_home_node 451 i386 cachestat sys_cachestat +452 i386 futex_wake sys_futex_wake --- a/arch/x86/entry/syscalls/syscall_64.tbl +++ b/arch/x86/entry/syscalls/syscall_64.tbl @@ -373,6 +373,7 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake # # Due to a historical design error, certain syscalls are numbered differently --- a/arch/xtensa/kernel/syscalls/syscall.tbl +++ b/arch/xtensa/kernel/syscalls/syscall.tbl @@ -422,3 +422,4 @@ 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node 451 common cachestat sys_cachestat +452 common futex_wake sys_futex_wake --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -563,6 +563,9 @@ asmlinkage long sys_set_robust_list(stru asmlinkage long sys_futex_waitv(struct futex_waitv *waiters, unsigned int nr_futexes, unsigned int flags, struct __kernel_timespec __user *timeout, clockid_t clockid); + +asmlinkage long sys_futex_wake(void __user *uaddr, int nr, unsigned int flags, u64 mask); + asmlinkage long sys_nanosleep(struct __kernel_timespec __user *rqtp, struct __kernel_timespec __user *rmtp); asmlinkage long sys_nanosleep_time32(struct old_timespec32 __user *rqtp, --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -816,12 +816,13 @@ __SYSCALL(__NR_process_mrelease, sys_pro __SYSCALL(__NR_futex_waitv, sys_futex_waitv) #define __NR_set_mempolicy_home_node 450 __SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node) - #define __NR_cachestat 451 __SYSCALL(__NR_cachestat, sys_cachestat) +#define __NR_futex_wake 452 +__SYSCALL(__NR_futex_wake, sys_futex_wake) #undef __NR_syscalls -#define __NR_syscalls 452 +#define __NR_syscalls 453 /* * 32 bit systems traditionally used different --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -309,6 +309,43 @@ SYSCALL_DEFINE5(futex_waitv, struct fute return ret; } +/* + * sys_futex_wake - Wake a number of futexes + * @uaddr: Address of the futex(es) to wake + * @nr: Number of the futexes to wake + * @flags: FUTEX2 flags + * @mask: bitmask + * + * Identical to the traditional FUTEX_WAKE_BITSET op, except it matches futex_waitv() above + * in that it enables u64 futex values and has a new flags set. + * + * NOTE: u64 futexes are not actually supported yet, but both these interfaces + * should allow for this to happen. + */ + +SYSCALL_DEFINE4(futex_wake, + void __user *, uaddr, + int, nr, + unsigned int, flags, + u64, mask) +{ + int bits; + + if (flags & ~FUTEX2_MASK) + return -EINVAL; + + if ((flags & FUTEX2_64) != FUTEX2_32) + return -EINVAL; + + flags = futex2_to_flags(flags); + bits = 8 * futex_size(flags); + + if (bits < 64 && mask >> bits) + return -EINVAL; + + return futex_wake(uaddr, flags, nr, mask); +} + #ifdef CONFIG_COMPAT COMPAT_SYSCALL_DEFINE2(set_robust_list, struct compat_robust_list_head __user *, head, --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -87,6 +87,7 @@ COND_SYSCALL_COMPAT(set_robust_list); COND_SYSCALL(get_robust_list); COND_SYSCALL_COMPAT(get_robust_list); COND_SYSCALL(futex_waitv); +COND_SYSCALL(futex_wake); COND_SYSCALL(kexec_load); COND_SYSCALL_COMPAT(kexec_load); COND_SYSCALL(init_module); From patchwork Fri Jul 14 13:39:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA23BEB64DC for ; Fri, 14 Jul 2023 14:16:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 156458D0008; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 086EF8D0001; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD5B18D0008; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A12908D0005 for ; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 70C01C02B3 for ; Fri, 14 Jul 2023 14:16:33 +0000 (UTC) X-FDA: 81010417866.05.FBEEF9D Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf15.hostedemail.com (Postfix) with ESMTP id 7E8D9A0019 for ; Fri, 14 Jul 2023 14:16:31 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=X9gijMCm; spf=none (imf15.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=vd7IZ5Q9sG2aEX5C+B5mpNsAgNCXrsR4bE5i4Du1qnc=; b=cDvjgF2mr/cCz6sQlp9/yLASjSpdEFGVvLcf+n1GkAIf+P0GSfgGb0r7Su47tZOv2EUOh2 ZEaNvwWmmp04m4gPHvRax/cJJZEelVmNJp0dqaHs9cdqTmaxxDdS0DrTTrWzmmkUmKzwYS 6Hdj/VbaQNotvB1IL8K/wB+l/pc8kpo= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=X9gijMCm; spf=none (imf15.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344191; a=rsa-sha256; cv=none; b=icKeZtcc3klNxFgV48aP4XRbuR1w5xYFWrhAX7IEGyWAKV/HglYbtejz5HypxBeGUZWifY a/V96nLLF/xUtTjaRnXgDyCaCL8WZmC/gkUY3aOqQsyog1I52pf4f1fBKqhKvvR5jGQysn wVyyv2C+lVf8VU+xoqGVLTII4OmUKyI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=vd7IZ5Q9sG2aEX5C+B5mpNsAgNCXrsR4bE5i4Du1qnc=; b=X9gijMCmfaPrSsayyWL0NuHiOe pGOhwRRTPBsZ40IXNFRORyzItcRWv4X+M3D/uQhswBiY0asbLlff6VJbJ0JHPEZfFMXjzmVs0I3h2 zIKqL8xHXO1+pjyWMBoxtpmq/8rxGe7cTHGeZ59jD2gdh/RhQ1gSNvuGbpNqbMkv4kEbOqjxxt/Bk BXDNnfyaPMAZRHuxBHH6Wfgfjrb5LqQ6zKpgZIsXsl9TYZBQrbC9LXYFJBv8YGZQfOAumvP+LAfQH DKKaLDkbVrzm/PSaPiAwV1U7mLa+G0455XJv/+IXRtYsqpaPxRb6KOlKfb+F71dSxkGZZpqg+7ydQ MafKW0bA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qKJah-006Iiz-2D; Fri, 14 Jul 2023 14:16:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B2786300C88; Fri, 14 Jul 2023 16:16:14 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 72FA0213728B8; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141218.947137012@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:04 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 05/10] mm: Add vmalloc_huge_node() References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7E8D9A0019 X-Rspam-User: X-Stat-Signature: beyzg51tt4mdqn5c8tgkw8x1s3gsjuqp X-Rspamd-Server: rspam01 X-HE-Tag: 1689344191-179349 X-HE-Meta: U2FsdGVkX1+99gTVBiTEnmaP52zj+jI+bDu3Mah7zenLXaCGIc8iSuXOLhF95BvMqSw3O1WFHq69nKVjMa42DyhYLmeF2YvqFTtBe5Os87VkiYBOGMNjeU3q/L4qDrfIiozhQgbgAt3RbU1IYN6RMo5qDYXhPwfFI89HmsVg5k0Dkd6SJU/7A9d4Pw7KEulxPWs1ON/9XChJDsjBemLA65mltVZMnWh9kqq4kM8c8LHqUWIteYZZ7gngXrtcHzxyrH2mAGrk4pbVjMUrZQc2BOVQiVvWb2uxWC7AqdGZxJA/4OaK5OCEq5kHkkGKHpVIV5+KGaZTNlxr9a5Esz7LfQD0+9wUHDYD8/SWvC9VJWDqf/wDChEz2ZSsvhtOdWpHL31qMN4brDCaLEvdB2dK2ScX1zO1JajLCft5JHluLImuL6Q2cnrBWS6ddJEMT14lP/uhnrKAmXKIgVQXjzns3uVraaNFfla8xuOQ2gq2AwWBi4/sDN+fO4w+/LRrI+57UcRtRDZiafNwIhm4fU1abjx5AX+kJXKLR38GrxkQjHPmVuSG7OmSF2sm4QRaLiiZMp2MA9MuBwf33mCwZSIzVuEuGsG/WBGLtnKkqBURGs3V0oAcGvuEWfZ165Yoo7ZQSaCs67oKbgmmU+WaVMowgH0P6nj0NAGIxt/ePZCy7bWSzjosJeJoosKzCUcBksa22WLxS4LhzFxDPglfdguCp9O7kjyRytDSiQFAhVDkdidxPeN1Qb3bCB/USBiXS1oEmwFCS5jDVKwBsnCSI3OSiUs8/11/IEOUtuGb3Gsadi9NvKGFOQaxLVbmT+7vRtUAoNyo53HARYns7QlJtJaBANdoHf2vRRVKBj6Qe4DQqAFoUK9o4Qt3E9XDE/InWYYgKvRm4In6P9dmfrWXybuG31nRLQkss0YWnjCwON8xHGXGBJcJ5DDX4PL3lxJUH+K2GfsQ5nn9KnmpPW4+VPZ DWcibznv beFWzA5A189ZPyrbzed288Kkeh2in5u0jL+KtITJ4XWP1JgrO7vQ6IHoojxFRIl1FqlIYF127ibITgl/KY/IVt7mBZHiUa1XDZa/ScTHoi6c6nrgZNr+vfEjMN1jfN+M8FWmZSVIGNV8BmU72s7UFap/NoBC0XN3m/fDiin6s326t56DuG1VOFnUr3k7TZS1jHduMMTeBSAu1UoWwVukjKwp9Mo6sEts64+2UUrwIMcchwyzxc1MDQTqmn8Fnzefac0pT5bxh+K0HmqRy7aJ+mncJi7PCjkD9tmK957IbJdfSy9VF0+F3VqWDc/A2lb35GeDq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To enable node specific hash-tables. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/vmalloc.h | 1 + mm/vmalloc.c | 11 ++++++++--- 2 files changed, 9 insertions(+), 3 deletions(-) --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -152,6 +152,7 @@ extern void *__vmalloc_node_range(unsign void *__vmalloc_node(unsigned long size, unsigned long align, gfp_t gfp_mask, int node, const void *caller) __alloc_size(1); void *vmalloc_huge(unsigned long size, gfp_t gfp_mask) __alloc_size(1); +void *vmalloc_huge_node(unsigned long size, gfp_t gfp_mask, int node) __alloc_size(1); extern void *__vmalloc_array(size_t n, size_t size, gfp_t flags) __alloc_size(1, 2); extern void *vmalloc_array(size_t n, size_t size) __alloc_size(1, 2); --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3416,6 +3416,13 @@ void *vmalloc(unsigned long size) } EXPORT_SYMBOL(vmalloc); +void *vmalloc_huge_node(unsigned long size, gfp_t gfp_mask, int node) +{ + return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, + gfp_mask, PAGE_KERNEL, VM_ALLOW_HUGE_VMAP, + node, __builtin_return_address(0)); +} + /** * vmalloc_huge - allocate virtually contiguous memory, allow huge pages * @size: allocation size @@ -3430,9 +3437,7 @@ EXPORT_SYMBOL(vmalloc); */ void *vmalloc_huge(unsigned long size, gfp_t gfp_mask) { - return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, - gfp_mask, PAGE_KERNEL, VM_ALLOW_HUGE_VMAP, - NUMA_NO_NODE, __builtin_return_address(0)); + return vmalloc_huge_node(size, gfp_mask, NUMA_NO_NODE); } EXPORT_SYMBOL_GPL(vmalloc_huge); From patchwork Fri Jul 14 13:39:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CF27C001DC for ; Fri, 14 Jul 2023 14:16:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 370A16B0074; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 282C26B0075; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AEAA6B0078; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EEA1F6B0074 for ; Fri, 14 Jul 2023 10:16:28 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 87D5A1202A1 for ; Fri, 14 Jul 2023 14:16:28 +0000 (UTC) X-FDA: 81010417656.26.A3359BC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP id 8A8DE4001A for ; Fri, 14 Jul 2023 14:16:26 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=nSMf8b1r; spf=none (imf17.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344186; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=gMJRupFzL+218xO81Os5o1A1M02NMRGWkhUhH51/Eio=; b=lqZI0veRcAIw1TthGUM9xzGxKP+FCXjqOCymMXnajSnnesko6zMqRA6rxNmrhAAhTzLQdx ja5EnLGGLVdqDrSS8/ivrJ+4vQPzW7/NBp8WRuzxv0AjUFxPkZKQOT6lFt7ivelIW2N7uL gs4lm9hpMtOXIE4iLHhqs/kUDnJjuHQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344186; a=rsa-sha256; cv=none; b=voQb/zNynPjvw20yguH9Rk26uVqKS3zOY82vpT2IySf82ETUhZX6ZdP56w/xrv73Tpm9mp Qe9fgShdByzOGV5unm/hhc6Rnd4BkSgInVyxi7p/VHV4Fn+uueqfWb+2ZvpGeBVUu+a40i hxNZdQ4r14BpyZ35oAhnIDTGOmdJfFQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=nSMf8b1r; spf=none (imf17.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=gMJRupFzL+218xO81Os5o1A1M02NMRGWkhUhH51/Eio=; b=nSMf8b1rDj20p7163tEl4RgaYw DSLe3ZSkxQr+7VY+r4TGNHy4IKVGgxGwlAnaBfZ1PYxgc+9BuGU4lppaUgLKt/OlT2VI9vmL+gtik RUUId1CkI+N7mJbbIBxkpyuDGdUa5aVpvS+Yc8iRIksH5FyqowxzwctWZ66ifZj4zpcRzxA1JwJLi xF0TcSWW2JGAR7aiFXIAyNyG7gAGrNLXJTPzDH9o8S5WGUPpfdHzDOaRFWdzQrzyqvi/yLQh8bPzw J3JoHmSbBfv75wEYSpd5GgPHirR718Rgz4S1L/+YVHpWN8Rfzc7tvdreKeoz9KiQE8w+Veqj80xY5 hVMxyKYA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qKJah-0016z7-Me; Fri, 14 Jul 2023 14:16:15 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B37713014BF; Fri, 14 Jul 2023 16:16:14 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 79A14213728BA; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141219.014386004@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:05 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 06/10] futex: Propagate flags into get_futex_key() References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8A8DE4001A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: axzgx6ea3i8hyjkh315jjfk8shokrriq X-HE-Tag: 1689344186-418799 X-HE-Meta: U2FsdGVkX1+DkUjhcnPfwhbiPI4EvwnZvgyHosU41W5znF+4XkL3vQhRzZYxz8SRnuLQcmv+BnNV3sFCI2QPMm8ihTOsPtVdKiWFGQCHIte8Xng3KxRE6nsM8yyra47meNBYN/A2krwkBOaN6cpzEnY9HH7erb3Id+BLEPymjlRvtNWEINVjvDkvqsiZzqKlJ6ueRnGAGwd0B8ozs6T16cGaPRs0Rrg8c1jgrpLHEUfHcvy0Kak20LtMDvwibLZu65F2dyVqlnXi0K9laYsvZ5vDKRz0w2Xfs9X7Eoqf16tvMKi6gw+BF+Mu+dD0KOZ5zbh0J/bCLhyEbubHgKewDgyphXhGWIYsu+rwyQIJpyRqQ5YzD6mm/DzQgtGyPslhVJY1+yVj5SreyyH2m06gMWRWAOcu1H33WtRDdNHeOfeqytpKPbvmw9gu5bHIKkiMtqs40iQkM3QYTGCcESF7RchUnmihQiR1D+Z9hrwHhRd+dyrdCTvv/fVcqXiLYEHeeAJT2LMis7us58swMvvx+RU1a9C+n6dOgdrmn+nLmmS84JnKuJdDcJPnvKxEhe6nmqIxaneO55LbdlbrZylpxtVqcVT9cuahtGIfdjVCjABGIAcLutMfCBFkhage7Kw0Fo9QgiJ2mLKKokLasxAW5E4JljGkPirzGX27DAdwCYIS1XJtZJ4pZ4gOUj2ijNcp/msTelpEATrbChEu0wZ+85ePE4FXlGZh24ShIq6m8wZbi75ZHeAqppl9S4jESl5TAurH4UIN1wLX6XrKBqoe2esEDKW7Wu0KEEyNJ+i+Y5zHIocVEQC7f9ih/R0JXnO1nZRyolOmXTjhcQdnyJvE814YmmBSbOK8voi1+65wFbBmfurjQf2G837zEQZdXRN6UuMrXMNgTXmHoXDaGVW/xXT22j23dFv5LB1Q/0T0+5U4pD/IgUglwcGYmELnkD2qRMzoD4Imm4MjcQa5XuJ Q53JQxhh ZiOM4tNGPiBd3JlbpSXS5x689CCNE3k+79gYr2p61dTDrT2TIpQ5f1TuV1kz08Av6glUNyitFqWXr8h3mCm0dV7RPXJR7x8botQe2G4u0cmcyO+L0QeHs2P6kob8I89cQs1tZk8pF4rQlYf0IB3nQc+EUMnjnikKVedcFnaPv9nRIBO/1BqNQ7Kjn9GWWZRO77jCMliMZgaQbiTpG2nstRIaWyfMv9OGoUxejY0zQ2z3EFDvABO2wwCu5EXpfSxXyP1LSvLEYIR/sdXEUPoEYtutkpwh56CjnB4gaNEbXYgo5krCozAmPLo2R04CqrgcggZRR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instead of only passing FLAGS_SHARED as a boolean, pass down flags as a whole. Signed-off-by: Peter Zijlstra (Intel) --- kernel/futex/core.c | 5 ++++- kernel/futex/futex.h | 2 +- kernel/futex/pi.c | 4 ++-- kernel/futex/requeue.c | 6 +++--- kernel/futex/waitwake.c | 15 ++++++++------- 5 files changed, 18 insertions(+), 14 deletions(-) --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -217,7 +217,7 @@ static u64 get_inode_sequence_number(str * * lock_page() might sleep, the caller should not hold a spinlock. */ -int get_futex_key(u32 __user *uaddr, bool fshared, union futex_key *key, +int get_futex_key(u32 __user *uaddr, unsigned int flags, union futex_key *key, enum futex_access rw) { unsigned long address = (unsigned long)uaddr; @@ -225,6 +225,9 @@ int get_futex_key(u32 __user *uaddr, boo struct page *page, *tail; struct address_space *mapping; int err, ro = 0; + bool fshared; + + fshared = flags & FLAGS_SHARED; /* * The futex address must be "naturally" aligned. --- a/kernel/futex/futex.h +++ b/kernel/futex/futex.h @@ -158,7 +158,7 @@ enum futex_access { FUTEX_WRITE }; -extern int get_futex_key(u32 __user *uaddr, bool fshared, union futex_key *key, +extern int get_futex_key(u32 __user *uaddr, unsigned int flags, union futex_key *key, enum futex_access rw); extern struct hrtimer_sleeper * --- a/kernel/futex/pi.c +++ b/kernel/futex/pi.c @@ -945,7 +945,7 @@ int futex_lock_pi(u32 __user *uaddr, uns to = futex_setup_timer(time, &timeout, flags, 0); retry: - ret = get_futex_key(uaddr, flags & FLAGS_SHARED, &q.key, FUTEX_WRITE); + ret = get_futex_key(uaddr, flags, &q.key, FUTEX_WRITE); if (unlikely(ret != 0)) goto out; @@ -1117,7 +1117,7 @@ int futex_unlock_pi(u32 __user *uaddr, u if ((uval & FUTEX_TID_MASK) != vpid) return -EPERM; - ret = get_futex_key(uaddr, flags & FLAGS_SHARED, &key, FUTEX_WRITE); + ret = get_futex_key(uaddr, flags, &key, FUTEX_WRITE); if (ret) return ret; --- a/kernel/futex/requeue.c +++ b/kernel/futex/requeue.c @@ -424,10 +424,10 @@ int futex_requeue(u32 __user *uaddr1, un } retry: - ret = get_futex_key(uaddr1, flags & FLAGS_SHARED, &key1, FUTEX_READ); + ret = get_futex_key(uaddr1, flags, &key1, FUTEX_READ); if (unlikely(ret != 0)) return ret; - ret = get_futex_key(uaddr2, flags & FLAGS_SHARED, &key2, + ret = get_futex_key(uaddr2, flags, &key2, requeue_pi ? FUTEX_WRITE : FUTEX_READ); if (unlikely(ret != 0)) return ret; @@ -789,7 +789,7 @@ int futex_wait_requeue_pi(u32 __user *ua */ rt_mutex_init_waiter(&rt_waiter); - ret = get_futex_key(uaddr2, flags & FLAGS_SHARED, &key2, FUTEX_WRITE); + ret = get_futex_key(uaddr2, flags, &key2, FUTEX_WRITE); if (unlikely(ret != 0)) goto out; --- a/kernel/futex/waitwake.c +++ b/kernel/futex/waitwake.c @@ -145,13 +145,13 @@ int futex_wake(u32 __user *uaddr, unsign struct futex_hash_bucket *hb; struct futex_q *this, *next; union futex_key key = FUTEX_KEY_INIT; - int ret; DEFINE_WAKE_Q(wake_q); + int ret; if (!bitset) return -EINVAL; - ret = get_futex_key(uaddr, flags & FLAGS_SHARED, &key, FUTEX_READ); + ret = get_futex_key(uaddr, flags, &key, FUTEX_READ); if (unlikely(ret != 0)) return ret; @@ -245,10 +245,10 @@ int futex_wake_op(u32 __user *uaddr1, un DEFINE_WAKE_Q(wake_q); retry: - ret = get_futex_key(uaddr1, flags & FLAGS_SHARED, &key1, FUTEX_READ); + ret = get_futex_key(uaddr1, flags, &key1, FUTEX_READ); if (unlikely(ret != 0)) return ret; - ret = get_futex_key(uaddr2, flags & FLAGS_SHARED, &key2, FUTEX_WRITE); + ret = get_futex_key(uaddr2, flags, &key2, FUTEX_WRITE); if (unlikely(ret != 0)) return ret; @@ -423,7 +423,7 @@ static int futex_wait_multiple_setup(str continue; ret = get_futex_key(u64_to_user_ptr(vs[i].w.uaddr), - vs[i].w.flags & FLAGS_SHARED, + vs[i].w.flags, &vs[i].q.key, FUTEX_READ); if (unlikely(ret)) @@ -435,7 +435,8 @@ static int futex_wait_multiple_setup(str for (i = 0; i < count; i++) { u32 __user *uaddr = (u32 __user *)(unsigned long)vs[i].w.uaddr; struct futex_q *q = &vs[i].q; - u32 val = (u32)vs[i].w.val; + unsigned int flags = vs[i].w.flags; + u32 val = vs[i].w.val; hb = futex_q_lock(q); ret = futex_get_value_locked(&uval, uaddr); @@ -599,7 +600,7 @@ int futex_wait_setup(u32 __user *uaddr, * while the syscall executes. */ retry: - ret = get_futex_key(uaddr, flags & FLAGS_SHARED, &q->key, FUTEX_READ); + ret = get_futex_key(uaddr, flags, &q->key, FUTEX_READ); if (unlikely(ret != 0)) return ret; From patchwork Fri Jul 14 13:39:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5A65C001DD for ; Fri, 14 Jul 2023 14:16:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 49ECA8D0001; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 212228D0006; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 012908D0005; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9E0FE8D0001 for ; Fri, 14 Jul 2023 10:16:33 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 63D3B14029B for ; Fri, 14 Jul 2023 14:16:33 +0000 (UTC) X-FDA: 81010417866.09.9AE453D Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf24.hostedemail.com (Postfix) with ESMTP id 73A28180020 for ; Fri, 14 Jul 2023 14:16:31 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=YACvjEyF; spf=none (imf24.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=LKkIc+bVFvx2M3BQiNOMB5u+Bv/zv5TptYfDZLCVMJs=; b=VhhD/G2T9T0Hcqzc82U1RsHZTjMruPJ/sh7C3osAgHvbEocE61Pc1VUMCkbbiEcH33OSHP B8B1C1xn6Po/FwordpXBfMgPdRLFX67UeSApQp7JaTjbQagJdJNNKq81vTLxhgtv4RlFK8 JHYvBI0ny6qyc2IHtWnUGJC9bCXEqYo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=YACvjEyF; spf=none (imf24.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344191; a=rsa-sha256; cv=none; b=NDAYDENs6ucOPkVH6xoCStzKhIodVtikTQX1yzV/GC9PYSx9FdTMS7q/ZmHcI7PRz0XBPD GkujLsALxoU5ZjAPAmS8RnG4A2ZNzyJ4bVswaDgwODiuCD/iHWxvkQSa6zrmkOUUb2sPiB EmjCWMeCK+QvdqNjviFNGwD+5VR9O3Q= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=LKkIc+bVFvx2M3BQiNOMB5u+Bv/zv5TptYfDZLCVMJs=; b=YACvjEyFpBHjZfWdUxJnifglSz ea+gxPw67sni1nYRSulmOd5Io1CTpN5nFemshxus7UujKs08hppNBahXzwhDQBRYKGXQLVfwzl+Ql xZsBFlng7IRFd1Vqa9MBl6kY8VJ+WIgAfMJ4dfRSUFbEZ8PnRpbkRlNyf92y5CEP8RGHyf9Z2cgEp BL1VegIdxBf93+2hNYgYqzhCf0ICdxo22/fYjcrn+pWw9zm2xQw0NSo6pPa19tMBuzgWnnAYCKcoP jcIeQ2RhDbyNm0k85zyrS8tIY7ZZAD4DttNbjAC+2ns4jOUY1zl13S2SQPfb06ejazyJtE1tnOemz gS1wCzbA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qKJah-006Iiy-2I; Fri, 14 Jul 2023 14:16:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B27623001E7; Fri, 14 Jul 2023 16:16:14 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 7E6BB213728BB; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141219.081639007@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:06 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 07/10] futex: Implement FUTEX2_NUMA References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 73A28180020 X-Rspam-User: X-Stat-Signature: 8d3rx83rw437mmgkk5zwoyixhnsw7zxq X-Rspamd-Server: rspam01 X-HE-Tag: 1689344191-785767 X-HE-Meta: U2FsdGVkX1/czlNCLscrIUuWjvzw0eQRanU3UE8wXIRrMx006WS1dJ0bX8jYN5r6/GKLBJhZxwehftJm1QV25dkq5hIMtZXZFl+6ua6lDsEoIdxHZEA1A3tP4x8aC3t+ZiF4B/YLSi05DXp1hzzw+Y21yNAZ1d3v4dLQiDt/24KbRuXs9/+LCyoygxHruCGKS4LTPzGyrwpcXwVGgfmCJu56xiJMFp89ZRYavF92KVP2bWlo0g8QJ7BWZlLR8+ohHFqND18xfQK6BzZkJSrr+5z4E1UZ9MJdtIknI8X7WpGgb7fHrfmYBpw0LCeL8TefKoA9bKcF7NV+PxxTCkaoaG9aQZUqnJjA/tjmN/X+X2q7RevbPAKbm1hwuB8ElTgjpWq3cjAQB4NzsLtP9BZoQcOTeFiGSOV7LOgl/FvS2PoD7cq4Srf/gqgW5zKM++nEMZbnlU2S0hY7CK2fu02h81ulmhMVjkqIcwkv+DwXsUghcjksA4tBXiSyw5DKo7HRXjOPvqWVawvfAVnH5Kp8Sprxgwy1ZnvBOmm2nAxXoAkbkhJ/wVIVeC+UCtE8hoJLU7dnclSc9EWfC5efutGzWSE3IuZ0JWOWlrYcQCYESU0iTSQhPEsg3ExwEiBmAPt37lVTHWzbiEhJz33MFW/RKt+b0p6VD9m7/1+VAonJ7cr0kOrunW/kHbJvUJeBgbKE0HN4pVmitl4Ei5Z7NX6Kkf5m+hRDKvHzPI+lOMqwYkCEkHrzQvkXaAKP3dhN6bsD7KhquBXLh4Hgre3yBWFOi6XXKkmBSgJEE6NAXSIKw8avQ7y9u/siWnbiwipEO+v181zdhjAlB3ELSqWsOAEUjcRsC/IwFJ1TT1JnMLTRDOhQqDp+biLMHxAtJoPRuZQJb/biC7Zc+CA8F8cuGU8ibX38Ag9vyZ2muBw5DGFvz7x5JbmUhfbxZZA+VEdXsMSY/+un79Nee8Dv59ClZ8n hYhsSJuo HWYtDC6UUszFRkhOhhCpbe8brhdKJeOC3Tnal6lrJiEhIKEAz3g05sja0M1aPgACrjyaDjSU8frF9vxSCnzts1+OzZqfJRTQA/w5C+Jak7fH1JfsJCAB/AkhktNoU4rGyCzwXj7L40FS54/IGsfd3nwneq14DY4uoBEhuGR1XTH8bzb6Sqxm4CHbbC1fEBD4YljNOXlq6BR3KVeZkajP75RrNYWxRW5vNAVRAgUU0Qn8+F5Hd8Vi412BwecsojhPbVXetarkTiR/4bm+e9tJQqbIxUFDhbwLOZk9WoFxQMIUv2B9ewwEijEJoUBSy3WMVewpz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Extend the futex2 interface to be numa aware. When FUTEX2_NUMA is specified for a futex, the user value is extended to two words (of the same size). The first is the user value we all know, the second one will be the node to place this futex on. struct futex_numa_32 { u32 val; u32 node; }; When node is set to ~0, WAIT will set it to the current node_id such that WAKE knows where to find it. If userspace corrupts the node value between WAIT and WAKE, the futex will not be found and no wakeup will happen. When FUTEX2_NUMA is not set, the node is simply an extention of the hash, such that traditional futexes are still interleaved over the nodes. This is done to avoid having to have a separate !numa hash-table. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/futex.h | 3 + kernel/futex/core.c | 125 +++++++++++++++++++++++++++++++++++++++--------- kernel/futex/futex.h | 2 kernel/futex/syscalls.c | 2 4 files changed, 107 insertions(+), 25 deletions(-) --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -34,6 +34,7 @@ union futex_key { u64 i_seq; unsigned long pgoff; unsigned int offset; + /* unsigned int node; */ } shared; struct { union { @@ -42,11 +43,13 @@ union futex_key { }; unsigned long address; unsigned int offset; + /* unsigned int node; */ } private; struct { u64 ptr; unsigned long word; unsigned int offset; + unsigned int node; /* NOT hashed! */ } both; }; --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -47,12 +47,14 @@ * reside in the same cacheline. */ static struct { - struct futex_hash_bucket *queues; unsigned long hashsize; + unsigned int hashshift; + struct futex_hash_bucket *queues[MAX_NUMNODES]; } __futex_data __read_mostly __aligned(2*sizeof(long)); -#define futex_queues (__futex_data.queues) -#define futex_hashsize (__futex_data.hashsize) +#define futex_hashsize (__futex_data.hashsize) +#define futex_hashshift (__futex_data.hashshift) +#define futex_queues (__futex_data.queues) /* * Fault injections for futexes. @@ -105,6 +107,26 @@ late_initcall(fail_futex_debugfs); #endif /* CONFIG_FAIL_FUTEX */ +static int futex_get_value(u32 *val, u32 __user *from, unsigned int flags) +{ + switch (futex_size(flags)) { + case 1: return __get_user(*val, (u8 __user *)from); + case 2: return __get_user(*val, (u16 __user *)from); + case 4: return __get_user(*val, (u32 __user *)from); + default: BUG(); + } +} + +static int futex_put_value(u32 val, u32 __user *to, unsigned int flags) +{ + switch (futex_size(flags)) { + case 1: return __put_user(val, (u8 __user *)to); + case 2: return __put_user(val, (u16 __user *)to); + case 4: return __put_user(val, (u32 __user *)to); + default: BUG(); + } +} + /** * futex_hash - Return the hash bucket in the global hash * @key: Pointer to the futex key for which the hash is calculated @@ -114,10 +136,20 @@ late_initcall(fail_futex_debugfs); */ struct futex_hash_bucket *futex_hash(union futex_key *key) { - u32 hash = jhash2((u32 *)key, offsetof(typeof(*key), both.offset) / 4, + u32 hash = jhash2((u32 *)key, + offsetof(typeof(*key), both.offset) / sizeof(u32), key->both.offset); + int node = key->both.node; + + if (node == -1) { + /* + * In case of !FLAGS_NUMA, use some unused hash bits to pick a + * node. + */ + node = (hash >> futex_hashshift) % num_possible_nodes(); + } - return &futex_queues[hash & (futex_hashsize - 1)]; + return &futex_queues[node][hash & (futex_hashsize - 1)]; } @@ -217,32 +249,64 @@ static u64 get_inode_sequence_number(str * * lock_page() might sleep, the caller should not hold a spinlock. */ -int get_futex_key(u32 __user *uaddr, unsigned int flags, union futex_key *key, +int get_futex_key(void __user *uaddr, unsigned int flags, union futex_key *key, enum futex_access rw) { unsigned long address = (unsigned long)uaddr; struct mm_struct *mm = current->mm; struct page *page, *tail; struct address_space *mapping; - int err, ro = 0; + int node, err, size, ro = 0; bool fshared; fshared = flags & FLAGS_SHARED; + size = futex_size(flags); /* * The futex address must be "naturally" aligned. */ key->both.offset = address % PAGE_SIZE; - if (unlikely((address % sizeof(u32)) != 0)) + if (unlikely((address % size) != 0)) return -EINVAL; address -= key->both.offset; - if (unlikely(!access_ok(uaddr, sizeof(u32)))) + if (flags & FLAGS_NUMA) + size *= 2; + + if (unlikely(!access_ok(uaddr, size))) return -EFAULT; if (unlikely(should_fail_futex(fshared))) return -EFAULT; + key->both.node = -1; + if (flags & FLAGS_NUMA) { + void __user *naddr = uaddr + size/2; + + if (futex_get_value(&node, naddr, flags)) + return -EFAULT; + + if (node == -1) { + node = numa_node_id(); + if (futex_put_value(node, naddr, flags)) + return -EFAULT; + } + + if (node >= num_possible_nodes()) + return -EINVAL; + + key->both.node = node; + } + + /* + * Encode the futex size in the offset. This makes cross-size + * wake-wait fail -- see futex_match(). + * + * NOTE that cross-size wake-wait is fundamentally broken wrt + * FLAGS_NUMA but could possibly work for !NUMA. + */ + key->both.offset |= FUT_OFF_SIZE * (flags & FLAGS_SIZE_MASK); + /* * PROCESS_PRIVATE futexes are fast. * As the mm cannot disappear under us and the 'key' only needs @@ -1125,27 +1189,42 @@ void futex_exit_release(struct task_stru static int __init futex_init(void) { - unsigned int futex_shift; - unsigned long i; + unsigned int order, n; + unsigned long size, i; #if CONFIG_BASE_SMALL futex_hashsize = 16; #else - futex_hashsize = roundup_pow_of_two(256 * num_possible_cpus()); + futex_hashsize = 256 * num_possible_cpus(); + futex_hashsize /= num_possible_nodes(); + futex_hashsize = roundup_pow_of_two(futex_hashsize); #endif + futex_hashshift = ilog2(futex_hashsize); + size = sizeof(struct futex_hash_bucket) * futex_hashsize; + order = get_order(size); + + for_each_node(n) { + struct futex_hash_bucket *table; + + if (order > MAX_ORDER) + table = vmalloc_huge_node(size, GFP_KERNEL, n); + else + table = alloc_pages_exact_nid(n, size, GFP_KERNEL); + + BUG_ON(!table); + + for (i = 0; i < futex_hashsize; i++) { + atomic_set(&table[i].waiters, 0); + spin_lock_init(&table[i].lock); + plist_head_init(&table[i].chain); + } - futex_queues = alloc_large_system_hash("futex", sizeof(*futex_queues), - futex_hashsize, 0, - futex_hashsize < 256 ? HASH_SMALL : 0, - &futex_shift, NULL, - futex_hashsize, futex_hashsize); - futex_hashsize = 1UL << futex_shift; - - for (i = 0; i < futex_hashsize; i++) { - atomic_set(&futex_queues[i].waiters, 0); - plist_head_init(&futex_queues[i].chain); - spin_lock_init(&futex_queues[i].lock); + futex_queues[n] = table; } + pr_info("futex hash table, %d nodes, %ld entries (order: %d, %lu bytes)\n", + num_possible_nodes(), + futex_hashsize, order, + sizeof(struct futex_hash_bucket) * futex_hashsize); return 0; } --- a/kernel/futex/futex.h +++ b/kernel/futex/futex.h @@ -158,7 +158,7 @@ enum futex_access { FUTEX_WRITE }; -extern int get_futex_key(u32 __user *uaddr, unsigned int flags, union futex_key *key, +extern int get_futex_key(void __user *uaddr, unsigned int flags, union futex_key *key, enum futex_access rw); extern struct hrtimer_sleeper * --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -180,7 +180,7 @@ SYSCALL_DEFINE6(futex, u32 __user *, uad return do_futex(uaddr, op, val, tp, uaddr2, (unsigned long)utime, val3); } -#define FUTEX2_MASK (FUTEX2_64 | FUTEX2_PRIVATE) +#define FUTEX2_MASK (FUTEX2_64 | FUTEX2_NUMA | FUTEX2_PRIVATE) /** * futex_parse_waitv - Parse a waitv array from userspace From patchwork Fri Jul 14 13:39:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B9B0C001DC for ; Fri, 14 Jul 2023 14:16:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D81456B0078; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D09DC8D0001; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAAFB6B007D; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A41926B0078 for ; Fri, 14 Jul 2023 10:16:29 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 34079B04B8 for ; Fri, 14 Jul 2023 14:16:29 +0000 (UTC) X-FDA: 81010417698.09.18489A3 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id D3BD540029 for ; Fri, 14 Jul 2023 14:16:26 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=cOl8XfYm; spf=none (imf27.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344187; a=rsa-sha256; cv=none; b=qGxt7ocHxk1GQRzyhY9cj52eoB2dPG1nR5/tvZBNBBADEOrjvjHAdU1khhBrv2e0c7l46C TxPtqpmdq+gm+RFBiZF7DDfMrMVNhyk52m1XrRT8GDZcvckou9w+UOjy7BTMyqrHTq0kMy 96wK/4iQJPgyfVPgFNLAmNGIZUQWBW0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=cOl8XfYm; spf=none (imf27.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344186; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=YZtbZT4V1GJ1CVh10G4kmmhbwertk2B4UiVE4SdkC2M=; b=1FV6zt9To+j4e2cnBtCxWY+xm6fWLu1qLeuyEcQlRdq+Ul6El6HqHH3PvUgqmUYdvoJ0LU FadyDZEg5DDilAmORvOkxuGQeyBrGI+l+iM9AarB3LQ9HR885T7TElXnJJUu+q+oCQ8n4z GqJ9U0ZZNUcRRHzXg2WUXCU6QcuqCJ0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=YZtbZT4V1GJ1CVh10G4kmmhbwertk2B4UiVE4SdkC2M=; b=cOl8XfYmCU/a0b/BBC7Nn/k0cT jBGUjXr8YbhcZHuC4EI7XNNB/CSbBBJrMRHM1uNNOcQI8p4hKMQuy/VnTzEcSLjV/fzlbhknFLYoC dyWPMzFBFHM0hllwEVXnC3zMrQpN8LeZu3QtawgherNAdSWLu/Us8LNZDlNosZz82Qv/54HfQqNRR IjvdglEd/PN8ktcC2J5/z4X0KzDvGesqRa/Bq9pZEWVw2PbspvLbdwCABlamLxGYvQxbmNJap4jHU Fkt/rlPiR+riomGl78Bo/4NsC2PJtU1xj+JL23YTLLHaFPgeNfUblbA1Oz7pZCRr3CKgStl1nHf3k LRhUC0Eg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qKJah-0016z6-M2; Fri, 14 Jul 2023 14:16:15 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B280030114F; Fri, 14 Jul 2023 16:16:14 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 82B8B244088B0; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141219.148373175@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:07 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 08/10] futex: Propagate flags into futex_get_value_locked() References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D3BD540029 X-Stat-Signature: 8mhjzfgzw3yjgo3ipterm7cw6qqtmx7s X-Rspam-User: X-HE-Tag: 1689344186-674648 X-HE-Meta: U2FsdGVkX1+FrUz7nQST8Nii0U2HgYjviAqzW6SwqFEtRSxj88ELZoEr6jbCfPr0z5gQ+k5po0y44lwQZ3H95B6LL1frugQ+MiaKarcXFSDJYD8GThYDZsunCp4bIfy2nW0zTnLswUMy6tZdbYmZgzzSv3BxuKpHOsWeKyyZv/jIDfVxapVf50ulkNBllM153jJbfUNeWukghSSc9Bn74QBeZbzZr7/U9HIeoGAlNd8EtqOj9tbynUi9wC1tffxOitUgJ/Tf2LeTyXkmSpGZlBWyR542tI3l6aYkli6HD9UPt+ca+hN5ZGFUzbGjFAFnIAZ8eddwSjG67XxXoTUfvq45A1mepA+1UXxPcwd6PbKrY8HPhZrtg/S49NoEwsRWZn0N6opXCJ3qo2P+aOCMo6xmi9bZp73hMrZL+SMtitcVDfSx90Og6YR2/1hKBZGQ6uVStLTKCEAz2iEtT1mooIg99TaQDpje2J76QdEgCvG3Gw88tch7+JTeXuPr+tfXQENRaHe5IoFJwGy51NZkT26MCsVzGfgB+R22KcFVhAXOZKDHVej1EVR556tBdcvC0sDCR3HNY59IzmrfsrbwkoVPrYL/ClDvl2EYb+HRTLAqPEECveN3OzzKiWB6x2/6fvfbxUNYvXa5Wqac3AEQOcNrynuZfx5X8ipbp2gCQV83EzqLY3brBHdeMUHdm+SXmPXm/Q6CJdQJtiwAeEEJ5BUkub9nb7C2GL9RSJC3t1kcncs7FYxnHw82WvALDUgRwfnKRAcZVpi1S/8qB2JYhP25Y4LHI/YMeA5xvmUbC7Ih8L8KIvPWodOsTguHeOBg0ycoKqf7bIFGrk723feZrUpDJG9jtU9aRWNH20ZM+sjjLLZw0hKigd+7jAZ5EvyDgHAt9p6U0Dnz1kp18/6VNBVyS95vZEZ5Sdq1A5s1THhLvmWSUsfmWBbMlK/B6QyGbpsEd+mbZpsZYhPLhEE 3uemIUIw kuIFqgk9YSsQDUGN4BCNoJEJKQzOHnw5cKvcJRE/Eh1QmusPkKiObCT1wN7XH7WZKf/tZP0DyZIFo+A/QXYpLijdK35jwixM7ALFwS+VuqqHYtSwbUG2Ie7PXFuilx7JKqoK2SYZ64z7D3hvoiZ2p8svhkpVEiyNmH4g6gDtP+EhxzlRcHTyf/iMJS5WVe9Dqq79VBnvx+QZvVeTb5/tvChb8+IQ4gVHIOBLUKCuP2oChq9qm6HuaH+1ZlknMXmn3ELd3EuwTk1fymh5OhSR4E/ixmvP/G9sZKU6oYOpTJQKr4XleT5i72O2I8CKHVsUMLwOD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In order to facilitate variable sized futexes propagate the flags into futex_get_value_locked(). Signed-off-by: Peter Zijlstra (Intel) --- kernel/futex/core.c | 4 ++-- kernel/futex/futex.h | 2 +- kernel/futex/pi.c | 8 ++++---- kernel/futex/requeue.c | 4 ++-- kernel/futex/waitwake.c | 4 ++-- 5 files changed, 11 insertions(+), 11 deletions(-) --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -506,12 +506,12 @@ int futex_cmpxchg_value_locked(u32 *curv return ret; } -int futex_get_value_locked(u32 *dest, u32 __user *from) +int futex_get_value_locked(u32 *dest, u32 __user *from, unsigned int flags) { int ret; pagefault_disable(); - ret = __get_user(*dest, from); + ret = futex_get_value(dest, from, flags); pagefault_enable(); return ret ? -EFAULT : 0; --- a/kernel/futex/futex.h +++ b/kernel/futex/futex.h @@ -190,7 +190,7 @@ extern void futex_wake_mark(struct wake_ extern int fault_in_user_writeable(u32 __user *uaddr); extern int futex_cmpxchg_value_locked(u32 *curval, u32 __user *uaddr, u32 uval, u32 newval); -extern int futex_get_value_locked(u32 *dest, u32 __user *from); +extern int futex_get_value_locked(u32 *dest, u32 __user *from, unsigned int flags); extern struct futex_q *futex_top_waiter(struct futex_hash_bucket *hb, union futex_key *key); extern void __futex_unqueue(struct futex_q *q); --- a/kernel/futex/pi.c +++ b/kernel/futex/pi.c @@ -239,7 +239,7 @@ static int attach_to_pi_state(u32 __user * still is what we expect it to be, otherwise retry the entire * operation. */ - if (futex_get_value_locked(&uval2, uaddr)) + if (futex_get_value_locked(&uval2, uaddr, FLAGS_SIZE_32)) goto out_efault; if (uval != uval2) @@ -358,7 +358,7 @@ static int handle_exit_race(u32 __user * * The same logic applies to the case where the exiting task is * already gone. */ - if (futex_get_value_locked(&uval2, uaddr)) + if (futex_get_value_locked(&uval2, uaddr, FLAGS_SIZE_32)) return -EFAULT; /* If the user space value has changed, try again. */ @@ -526,7 +526,7 @@ int futex_lock_pi_atomic(u32 __user *uad * Read the user space value first so we can validate a few * things before proceeding further. */ - if (futex_get_value_locked(&uval, uaddr)) + if (futex_get_value_locked(&uval, uaddr, FLAGS_SIZE_32)) return -EFAULT; if (unlikely(should_fail_futex(true))) @@ -762,7 +762,7 @@ static int __fixup_pi_state_owner(u32 __ if (!pi_state->owner) newtid |= FUTEX_OWNER_DIED; - err = futex_get_value_locked(&uval, uaddr); + err = futex_get_value_locked(&uval, uaddr, FLAGS_SIZE_32); if (err) goto handle_err; --- a/kernel/futex/requeue.c +++ b/kernel/futex/requeue.c @@ -273,7 +273,7 @@ futex_proxy_trylock_atomic(u32 __user *p u32 curval; int ret; - if (futex_get_value_locked(&curval, pifutex)) + if (futex_get_value_locked(&curval, pifutex, FLAGS_SIZE_32)) return -EFAULT; if (unlikely(should_fail_futex(true))) @@ -449,7 +449,7 @@ int futex_requeue(u32 __user *uaddr1, un if (likely(cmpval != NULL)) { u32 curval; - ret = futex_get_value_locked(&curval, uaddr1); + ret = futex_get_value_locked(&curval, uaddr1, FLAGS_SIZE_32); if (unlikely(ret)) { double_unlock_hb(hb1, hb2); --- a/kernel/futex/waitwake.c +++ b/kernel/futex/waitwake.c @@ -439,7 +439,7 @@ static int futex_wait_multiple_setup(str u32 val = vs[i].w.val; hb = futex_q_lock(q); - ret = futex_get_value_locked(&uval, uaddr); + ret = futex_get_value_locked(&uval, uaddr, FLAGS_SIZE_32); if (!ret && uval == val) { /* @@ -607,7 +607,7 @@ int futex_wait_setup(u32 __user *uaddr, retry_private: *hb = futex_q_lock(q); - ret = futex_get_value_locked(&uval, uaddr); + ret = futex_get_value_locked(&uval, uaddr, FLAGS_SIZE_32); if (ret) { futex_q_unlock(*hb); From patchwork Fri Jul 14 13:39:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72F91C001B0 for ; Fri, 14 Jul 2023 14:16:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 262628D0007; Fri, 14 Jul 2023 10:16:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E7368D0005; Fri, 14 Jul 2023 10:16:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B0328D0007; Fri, 14 Jul 2023 10:16:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F0CAC8D0005 for ; Fri, 14 Jul 2023 10:16:34 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B864016028F for ; Fri, 14 Jul 2023 14:16:34 +0000 (UTC) X-FDA: 81010417908.17.C51D014 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf06.hostedemail.com (Postfix) with ESMTP id 3112A180017 for ; Fri, 14 Jul 2023 14:16:30 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=RAyQOPIK; dmarc=none; spf=none (imf06.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=50j8lu0fq8DZ70IkYc8oVFT5B2TMdBpxV7U6GuBpl7Q=; b=SgYLbgBL+4Bn3uy1WIZgRyUjpQ4Kj/VKr25+wupojr/JabB2zZiZJ5bV0AOyzeAJ6pTwTY bnZ3Tw74gxBROAJLXSgHIVs752yTXD1DAlgyv+IJVBThsC9qyIoAz/L42zKYe6xJWmCyDT SH3P5of0dISxcPkYnZnSkNzgIMTHoq8= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=RAyQOPIK; dmarc=none; spf=none (imf06.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344191; a=rsa-sha256; cv=none; b=DkNVFz5XEQzRLmbEi92XaiuBb36w4i8akPqofal9OzRObI2w9YBk8i7SIPv4sWW5suDxbZ E9fnwhUK5FNl6SpRqYgx+zf0zxjxIOgB63L8NHMm2OTaZYFC6NLZEceBMI8eHaegiTQ5Go EX+4E4RBIgx9CMG3uuTjA7hNUDMPz6k= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=50j8lu0fq8DZ70IkYc8oVFT5B2TMdBpxV7U6GuBpl7Q=; b=RAyQOPIKt+lXLX555aDpPJzUC9 L7EOxKUfSVd10nX0K16zDGniwOxKc30Cd/+xG8LzFUBBkCLdSF5xP40Cevv5Y/l6dyWqJa+dVphas 6Gr3l8Rz6zoUJVtRlBiLy943qRAqpj/vIhSitOCh6gTyqaAbRgo3JiXP+Dc7K9WePvtmFiCyQfKj8 RCN4PcykMUeb1gdeFB8LRCRweSK248CKdk4wGCboxrybdoWYOf/v45Z/R/lBU7b2/xNeycJx/kjtz pIi/tFYgUgJFMSvi1VEusccqOJXl+n6NkSpusiNz4nBzjjmriQXxNwJcJZ5QOUmUYPapDvpPLr6Ya W0DmGT1Q==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qKJah-006Iix-2B; Fri, 14 Jul 2023 14:16:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B27AC300E86; Fri, 14 Jul 2023 16:16:14 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 8732D245EFFAA; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141219.215288670@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:08 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [RFC][PATCH 09/10] futex: Enable FUTEX2_{8,16} References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3112A180017 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: nnk1bb1zkit6a9f3m7fexu3ehq9ymbih X-HE-Tag: 1689344190-610280 X-HE-Meta: U2FsdGVkX19RwH8nmMJiVlOXQQk8logzP8Q5i6hg0g0hPZ1F7Uc/x+CHki+UXru7GFSVK+pgh9TyiIQdWsZvb6EO7gTJl5aF7Y2o/fmwZXnQmqNRfBF9NsGIMkXJbG/zWo0SQ5uuH3FB8Vc/pNEN2Gw/e4732XvhvTBf/k6UyuVohQhLbUSPb8n1IDvXbfVjK/NCHZvM7iksLtMgaXP0tNLu0q7Na0SCmo9bLeATSX/IQYIsNGrvHA7X6YlLbJGPQPN479EExbojyq+NbGFHQ/cKCtgLtucoTD60vzRl7hv23n510VhfvXLIkXgLGgJqLe+PYVMOnVOtvgFs+VTxnhzeiKN1k60F7h0B3tYUP4rSnPRClgVPxzvkx5ytNkvuOU3gBjZhenZSjhtwmKNRjXzNuVWQU8p8O0ts0gM8hNxu1d+4Mg+k2O2yLTXoZ6IyO4b18VkesR2feTGqbMiUZw0mAL3ErmLR2381THNO7YdAIJH/CU65eWn2tSMeJQ9h/QElxmIe4Rqhtgn14y29ZI5ftAQjQVuFJsRamr3HvsOihxTLEBpM0G6u36+82H7dGo1Ms0Zlw1kVazUdF6exo5D7Z9UrKkKuc0K9zujo9cKCkBIQgbOWx/FpE1M+hdYh+6H1dTU8X3seHbtq8X3teTmIzZ2y89yVbGCpOeFzfG5Ba6Ywqh++BGeQxXREAynshjHilDOP8V5/J186TZmaIJ2k8ST6rcZ1NaWKJgswxFxKP1mjU6tvCJx2T3INPFlt6GMns5jvZ2EOVkzoCgkHuOzSCbNW5KeYFzNC6sy4LZgjADcBL15iQt2XVCcy8PLj+pDtplY3RO3a6gX8SV/fH0y89rl1KOcMqyMWaUvZ8J/wGGofokd44a4nvJlDISMPJx9cf3hu2X/4cdu2/J/2jMMPRW8yDLA5H5Ffw6yIP8RUMpV0ezxBIF1QXy9j5Z4nGHr2kEvOiRvMW6B79so w0O4WEwK n6zm8IoPK1Z5DhGsqC0vLmasE/VbpgHjI6KHFShnBaLg4D8q+ul7bJ85cPDjANr9u2MfZQB2d/JwCFGqa6nj5K2R9e6sVa5Jc0Y/WA0+zccSLAU0fxWCQgGnSBMjbGrN0YRiffwoT0Vio8kxg6MKr0UZMifcdrxCprAGQCs1yOSU5AZwE+S7aomLVavj4eOKF9imgeFKNdaYX17JVkm677nSdYcMGdItbDWnpR36lDeX+o52T5R1YNZep+iMNN0USeE4V6uB+eNlP0/RKkHt/UlkoeXWkI3eZDcnBAMhR8Ba2fCcHFHU/UplfByKnOg0KzzDQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When futexes are no longer u32 aligned, the lower offset bits are no longer available to put type info in. However, since offset is the offset within a page, there are plenty bits available on the top end. After that, pass flags into futex_get_value_locked() for WAIT and disallow FUTEX2_64 instead of mandating FUTEX2_32. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/futex.h | 11 ++++++----- kernel/futex/syscalls.c | 4 ++-- kernel/futex/waitwake.c | 4 ++-- 3 files changed, 10 insertions(+), 9 deletions(-) --- a/include/linux/futex.h +++ b/include/linux/futex.h @@ -16,18 +16,19 @@ struct task_struct; * The key type depends on whether it's a shared or private mapping. * Don't rearrange members without looking at hash_futex(). * - * offset is aligned to a multiple of sizeof(u32) (== 4) by definition. - * We use the two low order bits of offset to tell what is the kind of key : + * offset is the position within a page and is in the range [0, PAGE_SIZE). + * The high bits of the offset indicate what kind of key this is: * 00 : Private process futex (PTHREAD_PROCESS_PRIVATE) * (no reference on an inode or mm) * 01 : Shared futex (PTHREAD_PROCESS_SHARED) * mapped on a file (reference on the underlying inode) * 10 : Shared futex (PTHREAD_PROCESS_SHARED) * (but private mapping on an mm, and reference taken on it) -*/ + */ -#define FUT_OFF_INODE 1 /* We set bit 0 if key has a reference on inode */ -#define FUT_OFF_MMSHARED 2 /* We set bit 1 if key has a reference on mm */ +#define FUT_OFF_INODE (PAGE_SIZE << 0) +#define FUT_OFF_MMSHARED (PAGE_SIZE << 1) +#define FUT_OFF_SIZE (PAGE_SIZE << 2) union futex_key { struct { --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -206,7 +206,7 @@ static int futex_parse_waitv(struct fute if ((aux.flags & ~FUTEX2_MASK) || aux.__reserved) return -EINVAL; - if ((aux.flags & FUTEX2_64) != FUTEX2_32) + if ((aux.flags & FUTEX2_64) == FUTEX2_64) return -EINVAL; flags = futex2_to_flags(aux.flags); @@ -334,7 +334,7 @@ SYSCALL_DEFINE4(futex_wake, if (flags & ~FUTEX2_MASK) return -EINVAL; - if ((flags & FUTEX2_64) != FUTEX2_32) + if ((flags & FUTEX2_64) == FUTEX2_64) return -EINVAL; flags = futex2_to_flags(flags); --- a/kernel/futex/waitwake.c +++ b/kernel/futex/waitwake.c @@ -439,7 +439,7 @@ static int futex_wait_multiple_setup(str u32 val = vs[i].w.val; hb = futex_q_lock(q); - ret = futex_get_value_locked(&uval, uaddr, FLAGS_SIZE_32); + ret = futex_get_value_locked(&uval, uaddr, flags); if (!ret && uval == val) { /* @@ -607,7 +607,7 @@ int futex_wait_setup(u32 __user *uaddr, retry_private: *hb = futex_q_lock(q); - ret = futex_get_value_locked(&uval, uaddr, FLAGS_SIZE_32); + ret = futex_get_value_locked(&uval, uaddr, flags); if (ret) { futex_q_unlock(*hb); From patchwork Fri Jul 14 13:39:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 13313717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C68B1EB64DC for ; Fri, 14 Jul 2023 14:16:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E3516B0072; Fri, 14 Jul 2023 10:16:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3776C6B0075; Fri, 14 Jul 2023 10:16:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 233AC6B0074; Fri, 14 Jul 2023 10:16:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 157206B0071 for ; Fri, 14 Jul 2023 10:16:28 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DDC62A02B6 for ; Fri, 14 Jul 2023 14:16:27 +0000 (UTC) X-FDA: 81010417614.01.59F0597 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP id 68F2C1C0034 for ; Fri, 14 Jul 2023 14:16:24 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vBKCfPzs; spf=none (imf20.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689344185; a=rsa-sha256; cv=none; b=n29io5+KDEsRvmLS0F/Wbo8kPZo8+Qq4prXKpHhfIL1Jrf/HP7teSq+IkGyhNbJjYqIzdB 9uyz83o7Es+r9MLUSRq0WpOpJbU9vA1ZJr9tYRrPWzBFDgarmjEUDF5dmOi+8uh+giFezE KjgYerAx6w86fy4MJNtwCVqlMKFQ7mE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vBKCfPzs; spf=none (imf20.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689344185; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=GuDJ92pMIScKKU1gBauDG1HK4VUsLXyIVGMSjzGcY4Y=; b=raqW/0TDHEj4R0dK3GP1Zu71oncIEh76SPokk1nkrNuUKvy/5dL7LNSpByge8S/r66p2AS zSgDyPkiLtAiOktZbhiutOILwlYmhlyLuTuCtYzxnV+PW2DF8hPS0UVU2Lto0XWNIENDYZ Rs02XP6juvukqRWAN51C05NGhaXe/y8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=GuDJ92pMIScKKU1gBauDG1HK4VUsLXyIVGMSjzGcY4Y=; b=vBKCfPzsXlmZQm0fbY9sm4XN73 kHbqhYPRg24KF+nwElURpqsTJDlryKmDUOG3hRkpWvd0PhoT+M+UUAAfZagygwwYN+EQF96rxRzwh y6Ksh55FaQO2qES2eZnYAQMF0W7I/N7hS39vamjL/ghCgcLWgEwlM2yccrC8KGWmH4aRt4oV3JFNS hotzaBdyxJlbgmQPxYdyPurXiCwYhcjscrxIqsdlOamEmJ65hOEFV33r2X0xNYQURheEouIYUgXH7 LZl2gTROHDF1JASKWhc+5uERTwqfjo5eGhlgZOjCV9CM0JpWWZtTMWyghP3XtRYjLU3AJZkGy2PqC A1aFb12A==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qKJah-0016z8-NN; Fri, 14 Jul 2023 14:16:15 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B952E301A2F; Fri, 14 Jul 2023 16:16:14 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 8BA61245EFFAB; Fri, 14 Jul 2023 16:16:13 +0200 (CEST) Message-ID: <20230714141219.282650897@infradead.org> User-Agent: quilt/0.66 Date: Fri, 14 Jul 2023 15:39:09 +0200 From: Peter Zijlstra To: tglx@linutronix.de, axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: [HACK][PATCH 10/10] futex: Munge size and numa into the legacy interface References: <20230714133859.305719029@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 68F2C1C0034 X-Stat-Signature: p85bbjtrhdrrsz8dyshn4y8a64ie7j3w X-Rspam-User: X-HE-Tag: 1689344184-467759 X-HE-Meta: U2FsdGVkX18vkIy/VfKZm0GCKCT7RHrMu4CQnbQC6z3lp38RFwFiSr0rV9Qagna+w78e4IlMNDZmBcM+0fczVz8294lbk6mkLPhEkky7BCIbplPcB4DBu8KUrS2x39fmoCin/PwOPZ41pHgblJdvDJyWe7E8mAOiiTXnpWGB0RmQnCKBMvhHON1VzBOewE2QSW/mSApktwV7KuvzB2dLPazcmyjQXQAuH8soHIlTt4xn5fbF9NxOZTypQ0EBF1ubGog6H8c0b/dDehk+ZBs5oWYZDynkVp1v0yLQ3+kn0d3YX4JangocyGdc6zdWKB9PR+cCTHljG67vzvTcWkNKQVP0kQbCPmiyAKL4FlRyIFRw59ItY91YvfgB3ChAEvuf9qIStXYsqqrLcK6SXU077ZPl5CzBdVzgv1AkR6WlX1OIiSR+CIfF9TRWvfc+dxp8ukcl6iernCYYFa9Hljd8sQchM4//xlYsh7SiTor8fsSMrCMshH2sTjXOGhd72ReZrg+HCpEUU69ao5mttfW+ZPyHORID5SzkMLn8mjmqJ0+nAZGP3zPlJqkl2jE/7QZZaoZAT5qfpSLLIq/44Ipo5fwAQxgXbX7bf5cnJptBga6XqzbC3AXI22IZABvXhNfnFLcgGmzScfxvLIS3vaDIDKwL20a0ugwQDoSlMLWjTD9VvgWCAdIHmgNpyPnAzw8sWvKv24O68DWZJ+9i8RWe1ToqrNjzBZQgQuri1/r/fRjY/7u5AybK2d6ATv1KXUT3u12877ffRhTIRLWofgg/vdlhfGJzYBe/Nl1lPsC6GEcLUoIGn0VRNgE7+IT77/3dpeUqL4StCj7SsZU62vM0Wb/1xoK1KZjLwyjYV5gJ+N0KKFLYHhVQe9BBPnLVmnwoU9FoND0pmjdGQrlUO7CfWutiPP+Y85ZbLDsWkAg9WelJWuWTlHKq4GDEBG3uCYcR3TYR0k/iX/WTAaj2xZL Tf2CdRgo tElu5gkeeR5LPnHzXbRAdVDl1gQ7flhnhprhaZZLja0DP/nWuNJx09mC14oTdRQd0WZSNo/JL5E896wK0B6q9pfbCaaGePlpRjFNGAYUhNrsv2xJphb9fFMjxozjplhcrB8h6iZ9KiK7UPBdCEJ2xkprAUMZoDsNswV+fpV6Zw3p2vWUyYDdiTwsEqEM3ZQpzqXd2qCzczApBnVnKFdhUJkKDl2u98QzQrBjFKB0nkenLQcOKFQP5qCIG0o8TxbR1yngD8yy/heNuRvZXvpycpnSD6g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Avert your eyes... Arguably just the NUMA thing wouldn't be too bad. Signed-off-by: Peter Zijlstra (Intel) --- include/uapi/linux/futex.h | 15 ++++++++++++--- kernel/futex/futex.h | 9 ++++++++- kernel/futex/syscalls.c | 18 ++++++++++++++++++ 3 files changed, 38 insertions(+), 4 deletions(-) --- a/include/uapi/linux/futex.h +++ b/include/uapi/linux/futex.h @@ -23,9 +23,18 @@ #define FUTEX_CMP_REQUEUE_PI 12 #define FUTEX_LOCK_PI2 13 -#define FUTEX_PRIVATE_FLAG 128 -#define FUTEX_CLOCK_REALTIME 256 -#define FUTEX_CMD_MASK ~(FUTEX_PRIVATE_FLAG | FUTEX_CLOCK_REALTIME) +#define FUTEX_PRIVATE_FLAG (1 << 7) +#define FUTEX_CLOCK_REALTIME (1 << 8) +#define FUTEX_NUMA (1 << 9) +#define FUTEX_SIZE_32 (0 << 10) /* backwards compat */ +#define FUTEX_SIZE_64 (1 << 10) +#define FUTEX_SIZE_8 (2 << 10) +#define FUTEX_SIZE_16 (3 << 10) + +#define FUTEX_CMD_MASK ~(FUTEX_PRIVATE_FLAG | \ + FUTEX_CLOCK_REALTIME | \ + FUTEX_NUMA | \ + FUTEX_SIZE_16) #define FUTEX_WAIT_PRIVATE (FUTEX_WAIT | FUTEX_PRIVATE_FLAG) #define FUTEX_WAKE_PRIVATE (FUTEX_WAKE | FUTEX_PRIVATE_FLAG) --- a/kernel/futex/futex.h +++ b/kernel/futex/futex.h @@ -39,7 +39,7 @@ /* FUTEX_ to FLAGS_ */ static inline unsigned int futex_to_flags(unsigned int op) { - unsigned int flags = FLAGS_SIZE_32; + unsigned int sz, flags = 0; if (!(op & FUTEX_PRIVATE_FLAG)) flags |= FLAGS_SHARED; @@ -47,6 +47,13 @@ static inline unsigned int futex_to_flag if (op & FUTEX_CLOCK_REALTIME) flags |= FLAGS_CLOCKRT; + if (op & FUTEX_NUMA) + flags |= FLAGS_NUMA; + + /* { 2,3,0,1 } -> { 0,1,2,3 } */ + sz = ((op + FUTEX_SIZE_8) & FUTEX_SIZE_16) >> 10; + flags |= sz; + return flags; } --- a/kernel/futex/syscalls.c +++ b/kernel/futex/syscalls.c @@ -95,6 +95,24 @@ long do_futex(u32 __user *uaddr, int op, return -ENOSYS; } + /* can't support u64 with a u32 based interface */ + if ((flags & FLAGS_SIZE_MASK) == FLAGS_SIZE_64) + return -ENOSYS; + + switch (cmd) { + case FUTEX_WAIT: + case FUTEX_WAIT_BITSET: + case FUTEX_WAKE: + case FUTEX_WAKE_BITSET: + /* u8, u16, u32 */ + break; + + default: + /* only u32 for now */ + if ((flags & FLAGS_SIZE_MASK) != FLAGS_SIZE_32) + return -ENOSYS; + } + switch (cmd) { case FUTEX_WAIT: val3 = FUTEX_BITSET_MATCH_ANY;