From patchwork Thu Sep 16 13:47:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12499123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2DC4C433EF for ; Thu, 16 Sep 2021 13:52:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 74DF861056 for ; Thu, 16 Sep 2021 13:52:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 74DF861056 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 17D966B0071; Thu, 16 Sep 2021 09:52:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12E4C6B0075; Thu, 16 Sep 2021 09:52:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1067900002; Thu, 16 Sep 2021 09:52:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0080.hostedemail.com [216.40.44.80]) by kanga.kvack.org (Postfix) with ESMTP id E45566B0071 for ; Thu, 16 Sep 2021 09:52:27 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id AE6AD8248D7C for ; Thu, 16 Sep 2021 13:52:27 +0000 (UTC) X-FDA: 78593576334.32.0F0EB7F Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf18.hostedemail.com (Postfix) with ESMTP id 75AF84002098 for ; Thu, 16 Sep 2021 13:52:27 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id mv7-20020a17090b198700b0019c843e7233so1072412pjb.4 for ; Thu, 16 Sep 2021 06:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=os6W81L8Jp53y3z498y4ntoeCrrr5kt+lNChMkb1a3E=; b=cFoQbgMUuO71GUUl31WSwW9Dbd6Ng9hWx2q8bhXjtr2f6iVWCmSB4vMoHCnSRtKcgL w0T97hle6ch5Nk6/EEndFfacy14u+8F/bJCRXxT3WNWbZn4LjKOhnUBqV6YFkwwpYppD BwltsgwY0L6sqKNzKVT1mwVq7Ft69f1x1Pw5HxaopGobPq+AVEF4WzSOEF8cA9bYtD34 KOmMg+k9cvV1uDnKDWLdvHTqVliBAOjPC5XeiNFTYO/tB8SZIsfwQldVc7iX5e1F1RuH LxJot3R06na+lEc/QyRd75fmonwEGERx+Jj1AIpHsbWDBDR126XpyfTziZSuAQwSxnXF 3zvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=os6W81L8Jp53y3z498y4ntoeCrrr5kt+lNChMkb1a3E=; b=ZL8DELdsAR/bDiJ1yAOY3SH0qlcTUV1LbVUllpVoJwhp38TNXSIc2AVimLAnW37IDm cC1t6OU1jCLtLvF4EajVFZxQcRfJ3xdxgItU5nTB1Oeq8zMcV/36dCfqp55uZU27Hvy3 1SJuAfLrDhyUN4iMHujVVfyKowL80svQr4hV1nzD9hW9lpUnGxvPQDapWi4fOpxBVX1m we97Y51uZSEpnm3cSwR29zE8h8I/on1x4VMh6p/0im7+KgUp1GGjr0rBhofC5PJmX798 1hJzwkyR61/Zd/jf1d+eAvMtO6m+nbvI/d39Y3rEWW2dPsC/pJTSYG736c6TGQHlaz/G UaFw== X-Gm-Message-State: AOAM530UZPYQQEP44dboZswmyoq9lrBV52I32gd4dy/mECCFRugsmt7f bMUweIueScM2+fnj1dg4h5Sa5w== X-Google-Smtp-Source: ABdhPJy9fa7IUXtp12uZvQnAxns54IHKulitYG16Aq9stdgfxKixbhCQfcEEbJ0+OtWXqL8ifok1yQ== X-Received: by 2002:a17:90a:5513:: with SMTP id b19mr15279213pji.16.1631800346633; Thu, 16 Sep 2021 06:52:26 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.226]) by smtp.gmail.com with ESMTPSA id o9sm3617443pfh.217.2021.09.16.06.52.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Sep 2021 06:52:26 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, bsingharora@gmail.com, shy828301@gmail.com, alexs@kernel.org, smuchun@gmail.com, zhengqi.arch@bytedance.com, Muchun Song Subject: [PATCH v2 03/13] mm: memcontrol: introduce compact_lock_page_irqsave Date: Thu, 16 Sep 2021 21:47:38 +0800 Message-Id: <20210916134748.67712-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210916134748.67712-1-songmuchun@bytedance.com> References: <20210916134748.67712-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 75AF84002098 X-Stat-Signature: oc9aohuc1b5cxshapc8q36ue4pfhwaia Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=cFoQbgMU; spf=pass (imf18.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.41 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-HE-Tag: 1631800347-812740 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we reuse the objcg APIs to charge LRU pages, the page_memcg() can be changed when the LRU pages reparented. In this case, we need to acquire the new lruvec lock. lruvec = mem_cgroup_page_lruvec(page); // The page is reparented. compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); // Acquired the wrong lruvec lock and need to retry. But compact_lock_irqsave() only take lruvec lock as the parameter, we cannot aware this change. If it can take the page as parameter to acquire the lruvec lock. When the page memcg is changed, we can use the page_memcg() detect whether we need to reacquire the new lruvec lock. So compact_lock_irqsave() is not suitable for us. Similar to lock_page_lruvec_irqsave(), introduce compact_lock_page_irqsave() to acquire the lruvec lock in the compaction routine. Signed-off-by: Muchun Song Acked-by: Roman Gushchin --- mm/compaction.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index bfc93da1c2c7..bf1a6048b5a3 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -509,6 +509,29 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags, return true; } +static struct lruvec *compact_lock_page_irqsave(struct page *page, + unsigned long *flags, + struct compact_control *cc) +{ + struct lruvec *lruvec; + + lruvec = mem_cgroup_page_lruvec(page); + + /* Track if the lock is contended in async mode */ + if (cc->mode == MIGRATE_ASYNC && !cc->contended) { + if (spin_trylock_irqsave(&lruvec->lru_lock, *flags)) + goto out; + + cc->contended = true; + } + + spin_lock_irqsave(&lruvec->lru_lock, *flags); +out: + lruvec_memcg_debug(lruvec, page); + + return lruvec; +} + /* * Compaction requires the taking of some coarse locks that are potentially * very heavily contended. The lock should be periodically unlocked to avoid @@ -1029,11 +1052,9 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (locked) unlock_page_lruvec_irqrestore(locked, flags); - compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); + lruvec = compact_lock_page_irqsave(page, &flags, cc); locked = lruvec; - lruvec_memcg_debug(lruvec, page); - /* Try get exclusive access under lock */ if (!skip_updated) { skip_updated = true;