[v4,0/5] THP aware uprobe

Message

Song Liu June 13, 2019, 5:57 p.m. UTC

This set makes uprobe aware of THPs.

Currently, when uprobe is attached to text on THP, the page is split by
FOLL_SPLIT. As a result, uprobe eliminates the performance benefit of THP.

This set makes uprobe THP-aware. Instead of FOLL_SPLIT, we introduces
FOLL_SPLIT_PMD, which only split PMD for uprobe. After all uprobes within
the THP are removed, the PTEs are regrouped into huge PMD.

Note that, with uprobes attached, the process runs with PTEs for the huge
page. The performance benefit of THP is recovered _after_ all uprobes on
the huge page are detached.

This set (plus a few THP patches) is also available at

   https://github.com/liu-song-6/linux/tree/uprobe-thp

Changes since v3:
1. Simplify FOLL_SPLIT_PMD case in follow_pmd_mask(), (Kirill A. Shutemov)
2. Fix try_collapse_huge_pmd() to match change in follow_pmd_mask().

Changes since v2:
1. For FOLL_SPLIT_PMD, populated the page table in follow_pmd_mask().
2. Simplify logic in uprobe_write_opcode. (Oleg Nesterov)
3. Fix page refcount handling with FOLL_SPLIT_PMD.
4. Much more testing, together with THP on ext4 and btrfs (sending in
   separate set).
5. Rebased up on Linus's tree:
   commit 35110e38e6c5 ("Merge tag 'media/v5.2-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media")

Changes since v1:
1. introduces FOLL_SPLIT_PMD, instead of modifying split_huge_pmd*();
2. reuse pages_identical() from ksm.c;
3. rewrite most of try_collapse_huge_pmd().

Song Liu (5):
  mm: move memcmp_pages() and pages_identical()
  uprobe: use original page when all uprobes are removed
  mm, thp: introduce FOLL_SPLIT_PMD
  uprobe: use FOLL_SPLIT_PMD instead of FOLL_SPLIT
  uprobe: collapse THP pmd after removing all uprobes

 include/linux/huge_mm.h |  7 +++++
 include/linux/mm.h      |  8 +++++
 kernel/events/uprobes.c | 54 +++++++++++++++++++++++++-------
 mm/gup.c                |  9 ++++--
 mm/huge_memory.c        | 69 +++++++++++++++++++++++++++++++++++++++++
 mm/ksm.c                | 18 -----------
 mm/util.c               | 13 ++++++++
 7 files changed, 146 insertions(+), 32 deletions(-)

--
2.17.1

Comments

Song Liu June 17, 2019, 4:27 a.m. UTC | #1

On Thu, Jun 13, 2019 at 10:58 AM Song Liu <[e-mail]> wrote:
>
> This set makes uprobe aware of THPs.
>
> Currently, when uprobe is attached to text on THP, the page is split by
> FOLL_SPLIT. As a result, uprobe eliminates the performance benefit of THP.
>
> This set makes uprobe THP-aware. Instead of FOLL_SPLIT, we introduces
> FOLL_SPLIT_PMD, which only split PMD for uprobe. After all uprobes within
> the THP are removed, the PTEs are regrouped into huge PMD.
>
> Note that, with uprobes attached, the process runs with PTEs for the huge
> page. The performance benefit of THP is recovered _after_ all uprobes on
> the huge page are detached.
>
> This set (plus a few THP patches) is also available at
>
>    https://github.com/liu-song-6/linux/tree/uprobe-thp
>
> Changes since v3:
> 1. Simplify FOLL_SPLIT_PMD case in follow_pmd_mask(), (Kirill A. Shutemov)
> 2. Fix try_collapse_huge_pmd() to match change in follow_pmd_mask().
>
> Changes since v2:
> 1. For FOLL_SPLIT_PMD, populated the page table in follow_pmd_mask().
> 2. Simplify logic in uprobe_write_opcode. (Oleg Nesterov)
> 3. Fix page refcount handling with FOLL_SPLIT_PMD.
> 4. Much more testing, together with THP on ext4 and btrfs (sending in
>    separate set).
> 5. Rebased up on Linus's tree:
>    commit 35110e38e6c5 ("Merge tag 'media/v5.2-2' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media")
>
> Changes since v1:
> 1. introduces FOLL_SPLIT_PMD, instead of modifying split_huge_pmd*();
> 2. reuse pages_identical() from ksm.c;
> 3. rewrite most of try_collapse_huge_pmd().
>

Hi Kirill and Oleg,

Does this version look good to you? If so, could you please reply with
your Acked-by and/or Reviewed-by?

Thanks,
Song