From patchwork Fri Oct 21 22:32:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13015558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9554EC3A59D for ; Fri, 21 Oct 2022 22:33:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35A718E0005; Fri, 21 Oct 2022 18:33:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E2718E0001; Fri, 21 Oct 2022 18:33:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15D6A8E0005; Fri, 21 Oct 2022 18:33:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 00F708E0001 for ; Fri, 21 Oct 2022 18:33:12 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C75661C6274 for ; Fri, 21 Oct 2022 22:33:12 +0000 (UTC) X-FDA: 80046408624.09.4934C0C Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf30.hostedemail.com (Postfix) with ESMTP id 6856280035 for ; Fri, 21 Oct 2022 22:33:12 +0000 (UTC) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-3584be56efbso41710747b3.8 for ; Fri, 21 Oct 2022 15:33:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UQIfOxRnlwRRwhWBO+p+5sf8htyV9CkZvkGFbADgqo4=; b=J8FQd3gsTR8s3Po46QnUbCcF1zAKwjsTF/q7AFSouetX0+G+1Rfi/UFIUWZtP/VMtN y3vQUPhUC20FPxpkNLVEO3vP7gT1h66wqxxc9sDsxETC8t43yuzQ4reM0O9cZmObc4o6 aCRFjf4SohxTnroY4G0CsJ2PNj1Oy3HkWR6BLfTijl8BT3+vdpul5wIZKx92L4VQploK X8Ufy8HlNlL3R9o1thXcAWA+jZc+Uxf6FqqRTTsnwiIC/r2BUuOkcOOy9vzzUeCE9c4D yiIabya8h9uONz0LytMhFHj2UGgqyWryZqcwCJfO3rWd9KJVsC9v9QwYeRvrgY/Px+On WHPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UQIfOxRnlwRRwhWBO+p+5sf8htyV9CkZvkGFbADgqo4=; b=UuvNfldoB9/F1fS78wRYhLDSGWkw9s/aMDSPcOwizEaW8OlBGtNJsYSzk1tAaLDtXS g5xpPhElkaNtTgNe5pbFocvcNdNyQov3tdB4iJkr1mwhoncXVoLtuX63nmBRkoXFVpzC P2wHU52/ZwPjqvjQJVxUldXUJpQ1rvjXfw15iii1NtQXEemux0LEmV7WGdmCXn1Wq0x7 dV79x10X/kZI8iLpIcEXiUcA1R3HJIGIlfuQL3jylDb1bBwN+wS7UmeiIiE5q5Dlwuiv oVo7VM9WRrACPO7bRPN5cN/H5YEDMFI6NESKvbkJcr8Pwliu/rKLdomQLjQUwOJThNK0 O47g== X-Gm-Message-State: ACrzQf3hbgU8NwudeHz5U6HTTUfrLNktVk8JZI39IhR481KNODtD5/z+ 9ZG7Ng70ULn80B/wIA7e+h9FyMQwcGVv X-Google-Smtp-Source: AMsMyM48OQoNtRhgvDbQpBn956ebPCd9YP3ElWOp9cq6UrF7FeGGnbi2A4XkVoszNMT0+oU5sO9aXZ0xI340 X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a05:6902:1209:b0:6bc:7da1:cd9b with SMTP id s9-20020a056902120900b006bc7da1cd9bmr18950874ybu.0.1666391591620; Fri, 21 Oct 2022 15:33:11 -0700 (PDT) Date: Fri, 21 Oct 2022 15:32:57 -0700 In-Reply-To: <20221021223300.3675201-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221021223300.3675201-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog Message-ID: <20221021223300.3675201-2-zokeefe@google.com> Subject: [PATCH man-pages v3 1/4] madvise.2: update THP file/shmem documentation for +5.4 From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666391592; a=rsa-sha256; cv=none; b=WR/enqGI7dow8WUz5ZnwlK9yhCNiwZh+qToodK2LHKiDvX9+apygsQ41ds6il61dgSPzKN IRz+HhtHQdle8j4ATdPxkbbNToSDidsSyxMfQ4ArJaurmUhLsitPei8upFVQ59udwzdWQq tMQDizLZLDlfPcoYBqBlmrHg45PfJXg= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=J8FQd3gs; spf=pass (imf30.hostedemail.com: domain of 3Jx5TYwcKCNALA600102AA270.yA8749GJ-886Hwy6.AD2@flex--zokeefe.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Jx5TYwcKCNALA600102AA270.yA8749GJ-886Hwy6.AD2@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666391592; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UQIfOxRnlwRRwhWBO+p+5sf8htyV9CkZvkGFbADgqo4=; b=iuZOEgOn7KemqBHf8AftyPxs5Jc+87frnoetyQNWBj34CJlswhox6fiZhFn0CP1mJ8qM9+ 3eqsbEEcljEWw1YbE0mZYk5CMVdUZLhS4a3LGhQ56fxgzfY0jHahR+br8TGIAowbkYqxu1 M0vLQQeQaL75BkcOxls1IH/tv3BmfE0= X-Stat-Signature: m4kicmi4pcn3io5sr9oeiyxwpjr5mfas X-Rspamd-Queue-Id: 6856280035 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=J8FQd3gs; spf=pass (imf30.hostedemail.com: domain of 3Jx5TYwcKCNALA600102AA270.yA8749GJ-886Hwy6.AD2@flex--zokeefe.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Jx5TYwcKCNALA600102AA270.yA8749GJ-886Hwy6.AD2@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1666391592-732639 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe Since Linux 5.4, Transparent Huge Pages now support both file-backed memory and shmem memory. Update MADV_HUGEPAGE advice description to reflect this. Additionally, expand the description of requirements for memory to be considered eligible for THP: alignment / mapping requirements, VMA flags, prctl(2) settings, inode status, etc. Signed-off-by: Zach O'Keefe --- man2/madvise.2 | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/man2/madvise.2 b/man2/madvise.2 index 81cce56af..64f788ace 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -320,8 +320,6 @@ Enable Transparent Huge Pages (THP) for pages in the range specified by .I addr and .IR length . -Currently, Transparent Huge Pages work only with private anonymous pages (see -.BR mmap (2)). The kernel will regularly scan the areas marked as huge page candidates to replace them with huge pages. The kernel will also allocate huge pages directly when the region is @@ -354,12 +352,46 @@ an access pattern that the developer knows in advance won't risk to increase the memory footprint of the application when transparent hugepages are enabled. .IP +.\" commit 99cb0dbd47a15d395bf3faa78dc122bc5efe3fc0 +Since Linux 5.4, +automatic scan of eligible areas and replacement by huge pages works with +private anonymous pages (see +.BR mmap (2)), +shmem pages, +and file-backed pages. +For all memory types, +memory may only be replaced by huge pages on hugepage-aligned boundaries. +For file-mapped memory \(em including tmpfs (see +.BR tmpfs (2)) +\(em the mapping must also be naturally hugepage-aligned within the file. +Additionally, +for file-backed, +non-tmpfs memory, +the file must not be open for write and the mapping must be executable. +.IP +The VMA must not be marked +.BR VM_NOHUGEPAGE , +.BR VM_HUGETLB , +.BR VM_IO , +.BR VM_DONTEXPAND , +.BR VM_MIXEDMAP , +or +.BR VM_PFNMAP , +nor can it be stack memory or backed by a DAX-enabled device +(unless the DAX device is hot-plugged as System RAM). +The process must also not have +.B PR_SET_THP_DISABLE +set (see +.BR prctl (2) ). +.IP The .B MADV_HUGEPAGE and .B MADV_NOHUGEPAGE operations are available only if the kernel was configured with -.BR CONFIG_TRANSPARENT_HUGEPAGE . +.B CONFIG_TRANSPARENT_HUGEPAGE +and file/shmem memory is only supported if the kernel was configured with +.BR CONFIG_READ_ONLY_THP_FOR_FS . .TP .BR MADV_NOHUGEPAGE " (since Linux 2.6.38)" Ensures that memory in the address range specified by From patchwork Fri Oct 21 22:32:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13015559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61C6FECDFA1 for ; Fri, 21 Oct 2022 22:33:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE0DB8E0006; Fri, 21 Oct 2022 18:33:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D69BF8E0001; Fri, 21 Oct 2022 18:33:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE4658E0006; Fri, 21 Oct 2022 18:33:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AA3658E0001 for ; Fri, 21 Oct 2022 18:33:14 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7EF7A80439 for ; Fri, 21 Oct 2022 22:33:14 +0000 (UTC) X-FDA: 80046408708.16.52D20CB Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf07.hostedemail.com (Postfix) with ESMTP id 2A11240034 for ; Fri, 21 Oct 2022 22:33:13 +0000 (UTC) Received: by mail-pf1-f201.google.com with SMTP id cu10-20020a056a00448a00b00562f2ff1058so1972291pfb.23 for ; Fri, 21 Oct 2022 15:33:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=j2YCFcqV/9HrzAdGPsM8brD/lu5Yer6yqOKsV1muLNk=; b=HHfec0IrAIaL4AtU3Wpm7aVpC4i94PMX7nrLVtU6PvrMsavLXNOlDhshgg27hWZdIt idPOGOjnsq77chS50l9alJOA+dp4fbdKYtel6mYHNQ0akrF1p8ChpF5/BYN52LnexEBP 4ve9YZatUDehddh8CDFokDoIQ9E4TWQaMCb7hlSA0ZSD3X5/3xZiZwsnuj4TULdfP21s rJxUGOJwsrRBDKXUMAi2FYITq1O0NUNm0y+ZwLxQ+lhk/d7Hmj7xI0EzASF75pcDym+d vZDT9jSnS45yobGQuRhMMbj2hdTyKiuHpvOcVNhdpt+koA5Fu0eDrCUVicuoZ3OvKAgl kgTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=j2YCFcqV/9HrzAdGPsM8brD/lu5Yer6yqOKsV1muLNk=; b=UILP8I3f4KF+R9Fgxfv7CReNBI2Gwg+fkmWa9QafNIetFY8aQ/8rdp65+81/lV1oY4 W1fJ4eKuqW93zl8ivKqx2DWJymfNL2sN4ej0usfjq+9nvQi87KFsZiYfX4tzNZgh8Axz lW9bY+46VrEdFG/b8z4UR3qrm36OyQu2N7w+lznsGc0HdIAjBDPp7Y6p97Us43eO4iMp K37uq1MvWO+dp3pqfVa9VFtEdk7+Qdk4pC/SnapHVt4HhR4vhrG8vNJn42r6UzFq+mYH MnWghXBgMQlYqMxHpmzjMIja0x9G/RfA1Yu5+eRDINrcKzNbvYV4M8FQdwQxZf2eSq6f bSjQ== X-Gm-Message-State: ACrzQf2Rfr+nUkyLmi4bDMWieOoAd6FjfbbHRkjrh9xEVFBazWxGxiHb 6bDB2ZlnX2Qe/emhhVUnx6r7BKJ+6gx4 X-Google-Smtp-Source: AMsMyM5jbebFfHwkbcLcmu7Jew0Ty0YvJLAwsxWJiSk2lsX8c+ywMk9uxffW2GmlBDmylLmhg9xrUyNlvbUA X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a62:144b:0:b0:562:38de:9a0e with SMTP id 72-20020a62144b000000b0056238de9a0emr21455891pfu.78.1666391593062; Fri, 21 Oct 2022 15:33:13 -0700 (PDT) Date: Fri, 21 Oct 2022 15:32:58 -0700 In-Reply-To: <20221021223300.3675201-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221021223300.3675201-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog Message-ID: <20221021223300.3675201-3-zokeefe@google.com> Subject: [PATCH man-pages v3 2/4] madvise.2: document reliable probe for advice support From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666391594; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j2YCFcqV/9HrzAdGPsM8brD/lu5Yer6yqOKsV1muLNk=; b=3YnP+HlyDiUhKfinyvYMzpgs7NPklK2rvOJyaVYLBB2lcoBOXf9hF+6O9yl4q3Ib41hfW1 sfstUWXzBywX5GP/SEKfYJ7thBhv9MkpjdhrTMYsOJc8O7jMLSQqYKPB/0P0g0pjNpPzj1 lzuOP6qYV/ZNKmi01sc0X8xR76Oq1VQ= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HHfec0Ir; spf=pass (imf07.hostedemail.com: domain of 3KR5TYwcKCNINC822324CC492.0CA96BIL-AA8Jy08.CF4@flex--zokeefe.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3KR5TYwcKCNINC822324CC492.0CA96BIL-AA8Jy08.CF4@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666391594; a=rsa-sha256; cv=none; b=eVB0oqcV/XsUmg/wHmGSeYNjpx/Ns3hPyUu87UcT7PsD9XkKmxTe4ZF+8JV6tctxzGrzxj mGFiaxdvkLk8+9yuBOXeoYqbVnsh0p4eRQhNGw7H8WNhBWqNZe1oof6n6ArtKK+982Rw+U dvGjWqldESoY6TDuu/oFQaFHW3htPX0= X-Stat-Signature: yngray8ujqn5gicgabzunkmsdx7qojxa X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HHfec0Ir; spf=pass (imf07.hostedemail.com: domain of 3KR5TYwcKCNINC822324CC492.0CA96BIL-AA8Jy08.CF4@flex--zokeefe.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3KR5TYwcKCNINC822324CC492.0CA96BIL-AA8Jy08.CF4@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2A11240034 X-HE-Tag: 1666391593-13779 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe EINVAL is an overloaded error code for madvise(2) and it's not clear under what context it means "advice is not valid" vs another error. Explicitly document that madvise(0, 0, advice) can reliably be used to probe for kernel support for "advice", returning zero iff "advice" is supported by the kernel. Signed-off-by: Zach O'Keefe --- man2/madvise.2 | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/man2/madvise.2 b/man2/madvise.2 index 64f788ace..df3413cc8 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -790,6 +790,11 @@ that are not mapped, the Linux version of ignores them and applies the call to the rest (but returns .B ENOMEM from the system call, as it should). +.PP +.IR madvise(0,\ 0,\ advice) +will return zero iff +.I advice +is supported by the kernel and can be relied on to probe for support. .\" .SH HISTORY .\" The .\" .BR madvise () From patchwork Fri Oct 21 22:32:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13015560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF924C433FE for ; Fri, 21 Oct 2022 22:33:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C6618E0007; Fri, 21 Oct 2022 18:33:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 728C18E0001; Fri, 21 Oct 2022 18:33:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52E468E0007; Fri, 21 Oct 2022 18:33:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3F8168E0001 for ; Fri, 21 Oct 2022 18:33:16 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 11F5740A11 for ; Fri, 21 Oct 2022 22:33:16 +0000 (UTC) X-FDA: 80046408792.30.514379B Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf13.hostedemail.com (Postfix) with ESMTP id A59392001B for ; Fri, 21 Oct 2022 22:33:15 +0000 (UTC) Received: by mail-pl1-f202.google.com with SMTP id m3-20020a170902bb8300b0017f7e7e4385so2398080pls.20 for ; Fri, 21 Oct 2022 15:33:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2mfLSi9N+JZYnQf9EtD/iIl2pLa/hOhtVxEVkuJcdM8=; b=lle+N5TrpCI1LNnjia1zJiidMB+e23mByhA6yA8Vit77UUdoQYM+pwQoiVsTaYTKyT N4oaZiegEhXqpWIdopwuN4BNJfgSeQ759NCA6ON9MwhxIxGVTg/jQ2646la3DV6DFDKa 6HvoFVNZiRwKbAqBW74PRLN4JlbW4hhTOmfdpsyiJ0OKudLFy++cBVyGOEj5WNJlljXd wABZ0t+Q6fb7c9S8a8xYGx1boByY+WGIiPGBL6lSnPDdf9LnoRtZTtk2mHw9Aqk4dgLQ DQBlnegZ+rTSY0xoPJU35qcOt+nFV8OWm6dr3zrMQ0o6STzWhYG6uWDRb40eMAnsWgbs gAgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2mfLSi9N+JZYnQf9EtD/iIl2pLa/hOhtVxEVkuJcdM8=; b=brVCAI9sf1SXy1Ym/PEl1pNCi201EupWtic1G1bAx/3ysVbaxCzpD8H/apnlpmAPPc Di9qXBRxmVJYlZfUK81Kg/dPs+Q/wU20+9ZoiJ0I4mSyfkiM0XgMQ4sUIA5k2qJzRW91 9KxBTR9bPO7Ud+rMSoQykqw+eafY2o39d6h6Iht3qW5RqkOW9MQuEWkohpnuvp7TdTWX JsEl4OzWvGra96yasWMYQ4aOI11ZsPbepKnW8dMag9mWxUp7BdjpoDrEGeVxnpZ8i/Uh mUZjW4A1AB3kFnTjNf0IMO8suwKZPbPft2qE36jdvNr3440wWTgNA8l3WafruzCxlEis JUmg== X-Gm-Message-State: ACrzQf3h7KbYxltd7D5nyASzg5Re/eznuNbTNB1Eq05Jv7V8i1qYzQlR 5MONoSyfsfb2yA9bV/cGQOIcoY9n9glT X-Google-Smtp-Source: AMsMyM7ykWOvJQhnYq5jJgK0UIS3OByEZBiFkO+t8lJMkUXraxyk1KUhcnhVDNpBp0OeeK7nYEVmNP4GhzH8 X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:903:24e:b0:179:b755:b82f with SMTP id j14-20020a170903024e00b00179b755b82fmr20595013plh.34.1666391594831; Fri, 21 Oct 2022 15:33:14 -0700 (PDT) Date: Fri, 21 Oct 2022 15:32:59 -0700 In-Reply-To: <20221021223300.3675201-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221021223300.3675201-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog Message-ID: <20221021223300.3675201-4-zokeefe@google.com> Subject: [PATCH man-pages v3 3/4] process_madvise.2: fix capability and ptrace requirements From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" , Suren Baghdasaryan , Minchan Kim ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666391595; a=rsa-sha256; cv=none; b=C6mXGvtdK+MTyrRmC8J4G4glqZsCeQA6Y8PjE/yfp+ozVST9j7xUgIgphhn8uPpbWbtpFK 2yvBgMFaOQ3ib3lIhux8oydh6v26vK8HZrRwMZ5QEVVhuiZkl1briFGcAUgZgNsNnsaBmu 5qMGt4lMAHVO/FZ24kkaG2PqP96Zus8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lle+N5Tr; spf=pass (imf13.hostedemail.com: domain of 3Kh5TYwcKCNMOD933435DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--zokeefe.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3Kh5TYwcKCNMOD933435DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666391595; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2mfLSi9N+JZYnQf9EtD/iIl2pLa/hOhtVxEVkuJcdM8=; b=AbRlS2mQyVXMIQ9653wE3I5z22Bri7OvDtODt+kQhUOA7y/WzewiXLQb2DkpnZK9bpxaEc mLbaqHnlzpdRiQ1C5qhwkATa+4H1M0TD4iyZh+qIQ5l2YGY9nNHQ0ysV/MqJNLcPrbDaxe 8A7o3Zw6RxhBM+XJipWIopH4ZL8QxD4= Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lle+N5Tr; spf=pass (imf13.hostedemail.com: domain of 3Kh5TYwcKCNMOD933435DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--zokeefe.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3Kh5TYwcKCNMOD933435DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: f33s4dm6uz3huy176f5dq5brymtgman9 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: A59392001B X-Rspam-User: X-HE-Tag: 1666391595-125846 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe The initial commit of process_madvise(2) to Linux, commit ecb8ac8b1f14 ("mm/madvise: introduce process_madvise() syscall: an external memory hinting API"), relied on PTRACE_MODE_ATTACH_FSCREDS (see ptrace(2)), but was amended by commit 96cfe2c0fd23 ("mm/madvise: replace ptrace attach requirement for process_madvise") which replaced this with a combination of PTRACE_MODE_READ and CAP_SYS_NICE (PTRACE_MODE_READ to prevent leaking ASLR metadata and CAP_SYS_NICE for influencing process performance). The initial commit of process_madvise(2) to man-pages project, made after the second patch, included two errors: 1) CAP_SYS_ADMIN instead of CAP_SYS_NICE 2) PTRACE_MODE_READ_REALCREDS instead of PTRACE_MODE_READ_FSCREDS Correct this in the man-page for process_madvise(2). Fixes: a144f458b ("process_madvise.2: Document process_madvise(2)") Cc: Suren Baghdasaryan Cc: Minchan Kim Signed-off-by: Zach O'Keefe Reviewed-by: Suren Baghdasaryan --- man2/process_madvise.2 | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/man2/process_madvise.2 b/man2/process_madvise.2 index 6208206e4..44d3b94e8 100644 --- a/man2/process_madvise.2 +++ b/man2/process_madvise.2 @@ -105,16 +105,20 @@ remote process. No further elements will be processed beyond that point. (See the discussion regarding partial advice in RETURN VALUE.) .PP -Permission to apply advice to another process is governed by a +.\" commit 96cfe2c0fd23ea7c2368d14f769d287e7ae1082e +Starting in Linux 5.12, +permission to apply advice to another process is governed by ptrace access mode -.B PTRACE_MODE_READ_REALCREDS +.B PTRACE_MODE_READ_FSCREDS check (see .BR ptrace (2)); in addition, because of the performance implications of applying the advice, the caller must have the -.B CAP_SYS_ADMIN -capability. +.B CAP_SYS_NICE +capability +(see +.BR capabilities (7)). .SH RETURN VALUE On success, .BR process_madvise () @@ -180,6 +184,15 @@ configuration option. The .BR process_madvise () system call is Linux-specific. +.SH NOTES +When this system call first appeared in Linux 5.10, +permission to apply advice to another process was entirely governed by +ptrace access mode +.B PTRACE_MODE_ATTACH_FSCREDS +check (see +.BR ptrace (2)). +This requirement was relaxed in Linux 5.12 so that the caller didn't require +full control over the target process. .SH SEE ALSO .BR madvise (2), .BR pidfd_open (2), From patchwork Fri Oct 21 22:33:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13015561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C040BC38A2D for ; Fri, 21 Oct 2022 22:33:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 619E88E0008; Fri, 21 Oct 2022 18:33:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 52BEE8E0001; Fri, 21 Oct 2022 18:33:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 332428E0008; Fri, 21 Oct 2022 18:33:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 200F88E0001 for ; Fri, 21 Oct 2022 18:33:18 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E42AAC0A91 for ; Fri, 21 Oct 2022 22:33:17 +0000 (UTC) X-FDA: 80046408834.12.DFD035C Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf24.hostedemail.com (Postfix) with ESMTP id 7F4DE180019 for ; Fri, 21 Oct 2022 22:33:17 +0000 (UTC) Received: by mail-pl1-f202.google.com with SMTP id l16-20020a170902f69000b001865f863784so2381163plg.2 for ; Fri, 21 Oct 2022 15:33:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=Fhq9+ubrffUDOgK8gNyGgNPDHRU1+55DMZbYHuijl2E=; b=Y177hO7earZyQ0NgvX45x7cQ+QQcbkSCyGj10syjwEY+O+MT0crQAgP7c53piVwCRR E8jqObD43Mw/CAVnzfeH/GKucD9VnR+8zjWBqsJQy++OPzQeluqj7uJzJW2wIgXRZuZW B90bxgjZzBp0Ejc6Dk1FwOwuZ791xd8fBCSz5ZW5mYRDkzPjmV8WYLNDpyBN4dzzHtdR WdcpYlF2n4uGjVgp4DC75k42aqHMZ8gJE3Mvl9TC5mlAJCnVDYy9uWQ+FPL3ezl6gR1U 6euz62GLoGPc3CVoJ4AR6MOWamjFuCeIQsVifzdQkdls34lXwUSWmvtAaxcmCT3cNWnN KtFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=Fhq9+ubrffUDOgK8gNyGgNPDHRU1+55DMZbYHuijl2E=; b=JKOI2enSApCY7mDzabFzX1D3MU3xjRtGnPQPsXdIWozQQGFWpf9dR8NtJFlMJpoxJU bM84/l2HRQ4v+PlBnxJISVvGKc1dceGKT78BZ0x1YIJ4xkVE92sD9yLPKAmXiAK9urES vAIn+x2mPn1hqUF1tdRh6Aa9winXf4qdeZKTAJTWj1KSw1z0HsKV2irqf6NrEWIZT2Uc /hvwrZrNZS4P8XnUJJBa57X2RUnv4owQMKXWrKYIG9nJoD0YCh2oSLxy5+z7B9VOZDWm /ApUy/slQFmb/d5CfeBM0P+OrT4nrrcUv6F16WSJtmpk360Xw0kvruZFGLnqfCx2EUjR yVZg== X-Gm-Message-State: ACrzQf2zujuK34sQdve1RhmRbsyvdMtU/WLBs1h/jKVCD6eKf//F9o4V 3aOaKOtiCTArwNTJWZQdTSafgy5UnySh X-Google-Smtp-Source: AMsMyM5tDeCjjt4bJTgjV2UIJ2FL/zqgZQGgoBq56Gw08dtgmaViPv5SnsyL/cVsZ8cXHGFFFaWGnL3Im69q X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:902:e742:b0:17f:6a44:ee4a with SMTP id p2-20020a170902e74200b0017f6a44ee4amr21719351plf.103.1666391596561; Fri, 21 Oct 2022 15:33:16 -0700 (PDT) Date: Fri, 21 Oct 2022 15:33:00 -0700 In-Reply-To: <20221021223300.3675201-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221021223300.3675201-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog Message-ID: <20221021223300.3675201-5-zokeefe@google.com> Subject: [PATCH man-pages v3 4/4] madvise.2: add documentation for MADV_COLLAPSE From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666391597; a=rsa-sha256; cv=none; b=ZK9VM1AsgzZoBynb0oFj+yHbegz2a8/srS0h8vH94sVNzklanTBOyh54YrKfXrOVO1i0De rcaG2QrVhRiV7sIjTsgYEpztF/TeSk7VYgALajYjWIMvhIBfpHuVeaf4GazL+W4sU9qbMW xx3I1Dje2p3UPzX+uWLDffPEv/3MisE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Y177hO7e; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3LB5TYwcKCNUQFB55657FF7C5.3FDC9ELO-DDBM13B.FI7@flex--zokeefe.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3LB5TYwcKCNUQFB55657FF7C5.3FDC9ELO-DDBM13B.FI7@flex--zokeefe.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666391597; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fhq9+ubrffUDOgK8gNyGgNPDHRU1+55DMZbYHuijl2E=; b=d3Ay7DvbizfI1PjRWBLPgzpnBrQfPLUl7zcjHRFiNT/AQYXgrFTCOHZiooxbUjUSuAG6bp UuN/3v5PB8w2BO4DWOOSmnVUlGaBWghH/57EiTKmd2Yu6k6xQvGYyXuE7LSEwycNRzd1jN TSPOnZyMPoGlc8I3iF/Ns/2Ar1tDb9o= Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Y177hO7e; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3LB5TYwcKCNUQFB55657FF7C5.3FDC9ELO-DDBM13B.FI7@flex--zokeefe.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3LB5TYwcKCNUQFB55657FF7C5.3FDC9ELO-DDBM13B.FI7@flex--zokeefe.bounces.google.com X-Rspamd-Server: rspam04 X-Rspam-User: X-Stat-Signature: gux475ifc9zp7dut37d83rizxxqj6j4t X-Rspamd-Queue-Id: 7F4DE180019 X-HE-Tag: 1666391597-394568 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe Linux 6.1 introduced MADV_COLLAPSE in upstream commit 7d8faaf15545 ("mm/madvise: introduce MADV_COLLAPSE sync hugepage collapse") and upstream commit 34488399fa08 ("mm/madvise: add file and shmem support to MADV_COLLAPSE"). Update the man-pages for madvise(2) and process_madvise(2). Link: https://lore.kernel.org/linux-mm/20220922224046.1143204-1-zokeefe@google.com/ Link: https://lore.kernel.org/linux-mm/20220706235936.2197195-1-zokeefe@google.com/ Signed-off-by: Zach O'Keefe --- man2/madvise.2 | 90 +++++++++++++++++++++++++++++++++++++++++- man2/process_madvise.2 | 10 +++++ 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/man2/madvise.2 b/man2/madvise.2 index df3413cc8..b03fc731d 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -385,9 +385,10 @@ set (see .BR prctl (2) ). .IP The -.B MADV_HUGEPAGE +.BR MADV_HUGEPAGE , +.BR MADV_NOHUGEPAGE , and -.B MADV_NOHUGEPAGE +.B MADV_COLLAPSE operations are available only if the kernel was configured with .B CONFIG_TRANSPARENT_HUGEPAGE and file/shmem memory is only supported if the kernel was configured with @@ -400,6 +401,81 @@ and .I length will not be backed by transparent hugepages. .TP +.BR MADV_COLLAPSE " (since Linux 6.1)" +.\" commit 7d8faaf155454f8798ec56404faca29a82689c77 +.\" commit 34488399fa08faaf664743fa54b271eb6f9e1321 +Perform a best-effort synchronous collapse of the native pages mapped by the +memory range into Transparent Huge Pages (THPs). +.B MADV_COLLAPSE +operates on the current state of memory of the calling process and makes no +persistent changes or guarantees on how pages will be mapped, +constructed, +or faulted in the future. +.IP +.B MADV_COLLAPSE +supports private anonymous pages (see +.BR mmap (2)), +shmem pages, +and file-backed pages. +See +.B MADV_HUGEPAGE +for general information on memory requirements for THP. +If the range provided spans multiple VMAs, +the semantics of the collapse over each VMA is independent from the others. +If collapse of a given huge page-aligned/sized region fails, +the operation may continue to attempt collapsing the remainder of the +specified memory. +.B MADV_COLLAPSE +will automatically clamp the provided range to be hugepage-aligned. +.IP +All non-resident pages covered by the range will first be +swapped/faulted-in, +before being copied onto a freshly allocated hugepage. +If the native pages compose the same PTE-mapped hugepage, +and are suitably aligned, +allocation of a new hugepage may be elided and collapse may happen +in-place. +Unmapped pages will have their data directly initialized to 0 in the new +hugepage. +However, +for every eligible hugepage-aligned/sized region to be collapsed, +at least one page must currently be backed by physical memory. +.IP +.BR MADV_COLLAPSE +is independent of any sysfs +(see +.BR sysfs (5)) +setting under +.IR /sys/kernel/mm/transparent_hugepage , +both in terms of determining THP eligibility, +and allocation semantics. +See Linux kernel source file +.I Documentation/admin\-guide/mm/transhuge.rst +for more information. +.BR MADV_COLLAPSE +also ignores +.B huge= +tmpfs mount when operating on tmpfs files. +Allocation for the new hugepage may enter direct reclaim and/or compaction, +regardless of VMA flags +(though +.BR VM_NOHUGEPAGE +is still respected). +.IP +When the system has multiple NUMA nodes, +the hugepage will be allocated from the node providing the most native +pages. +.IP +If all hugepage-sized/aligned regions covered by the provided range were +either successfully collapsed, +or were already PMD-mapped THPs, +this operation will be deemed successful. +Note that this doesn’t guarantee anything about other possible mappings of +the memory. +Also note that many failures might have occurred since the operation may +continue to collapse in the event collapse of a single hugepage-sized/aligned +region fails. +.TP .BR MADV_DONTDUMP " (since Linux 3.4)" .\" commit 909af768e88867016f427264ae39d27a57b6a8ed .\" commit accb61fe7bb0f5c2a4102239e4981650f9048519 @@ -619,6 +695,11 @@ A kernel resource was temporarily unavailable. .B EBADF The map exists, but the area maps something that isn't a file. .TP +.B EBUSY +(for +.BR MADV_COLLAPSE ) +Could not charge hugepage to cgroup: cgroup limit exceeded. +.TP .B EFAULT .I advice is @@ -716,6 +797,11 @@ maximum resident set size. Not enough memory: paging in failed. .TP .B ENOMEM +(for +.BR MADV_COLLAPSE ) +Not enough memory: could not allocate hugepage. +.TP +.B ENOMEM Addresses in the specified range are not currently mapped, or are outside the address space of the process. .TP diff --git a/man2/process_madvise.2 b/man2/process_madvise.2 index 44d3b94e8..8b0ddccdd 100644 --- a/man2/process_madvise.2 +++ b/man2/process_madvise.2 @@ -73,6 +73,10 @@ argument is one of the following values: See .BR madvise (2). .TP +.B MADV_COLLAPSE +See +.BR madvise (2). +.TP .B MADV_PAGEOUT See .BR madvise (2). @@ -173,6 +177,12 @@ The caller does not have permission to access the address space of the process .TP .B ESRCH The target process does not exist (i.e., it has terminated and been waited on). +.PP +See +.BR madvise (2) +for +.IR advice -specific +errors. .SH VERSIONS This system call first appeared in Linux 5.10. .\" commit ecb8ac8b1f146915aa6b96449b66dd48984caacc