From patchwork Fri Apr 21 22:12:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13220737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A428C7618E for ; Fri, 21 Apr 2023 22:13:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233760AbjDUWN1 (ORCPT ); Fri, 21 Apr 2023 18:13:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233734AbjDUWNY (ORCPT ); Fri, 21 Apr 2023 18:13:24 -0400 Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D533319A1 for ; Fri, 21 Apr 2023 15:13:22 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-63b73203e0aso17102407b3a.1 for ; Fri, 21 Apr 2023 15:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115202; x=1684707202; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u/AkHMm2p+EP+a6U0DbEFYOuvSo1i83VDzuubdv1yU4=; b=gWPYUh/qJS3hKUZOK+9h4MWoL8QjpqBENZF1/R77mtm+LeEDasMRdSowMwEpPxxS78 EP2q9aDYBJHna1dXO/vRjzhgMH3xXjajRzj+Gem3tqcctYYdBwGqaQ29Kog5RyRfy0yp UjVmuXQ923zKnlvwr5/ZYrwLOML+t858ymiAc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115202; x=1684707202; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u/AkHMm2p+EP+a6U0DbEFYOuvSo1i83VDzuubdv1yU4=; b=dQlWoNbJuOOi3AzJsvfz8rwgrcrbbodRu7ne2a2dmIIJOJ9GCzAV6ubU2cQCkuYJXr 0lIU0wUtxwq0S6BxV2qvp7Wd0bDG/hz+I/2hkbRFXR/efvqqaOjajs7DKHp8kMxGmnIs 1L579wSQcNSN+wYehzh6ygjCIJnJVxe7B6svJRANuerFQrdGn8+XQM5vHNdyLWpYMlbP jqXWAU0SIlRINb0k+MHRsfim4HZ9XcvOJE5nZIzOwtHXXkeR89NsxydNINxnWSOEnB9b wyjvcfVuqDmtOdxYlVLpfYVDE8KOpvE+ZjTKoS5A8sKXhfS4uLreOhFaB2nGYpTUPkSr ktWg== X-Gm-Message-State: AAQBX9cyUdfMjpA7Ih6bxZXYc8qBDH1MY9M1ePZ7NrRA35WIrNhaOeHL bc9VKjgvgIxRmrj0DopJ9Kjesw== X-Google-Smtp-Source: AKy350bcHsIIY4k+lLMfZH1OxrXb0oxc9pTN/Ivx+wmbrGQA//e/eiBuiaIqWiD0BPfrEISD1XsY2A== X-Received: by 2002:a05:6a20:8426:b0:f0:8708:2341 with SMTP id c38-20020a056a20842600b000f087082341mr7837599pzd.26.1682115202268; Fri, 21 Apr 2023 15:13:22 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:21 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson Subject: [PATCH v2 1/4] mm/filemap: Add folio_lock_timeout() Date: Fri, 21 Apr 2023 15:12:45 -0700 Message-ID: <20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a variant of folio_lock() that can timeout. This is useful to avoid unbounded waits for the page lock in kcompactd. Signed-off-by: Douglas Anderson --- Changes in v2: - "Add folio_lock_timeout()" new for v2. include/linux/pagemap.h | 16 ++++++++++++++ mm/filemap.c | 47 +++++++++++++++++++++++++++++------------ 2 files changed, 50 insertions(+), 13 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 0acb8e1fb7af..0f3ef9f79300 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -892,6 +892,7 @@ static inline bool wake_page_match(struct wait_page_queue *wait_page, } void __folio_lock(struct folio *folio); +int __folio_lock_timeout(struct folio *folio, long timeout); int __folio_lock_killable(struct folio *folio); bool __folio_lock_or_retry(struct folio *folio, struct mm_struct *mm, unsigned int flags); @@ -952,6 +953,21 @@ static inline void folio_lock(struct folio *folio) __folio_lock(folio); } +/** + * folio_lock_timeout() - Lock this folio, with a timeout. + * @folio: The folio to lock. + * @timeout: The timeout in jiffies; %MAX_SCHEDULE_TIMEOUT means wait forever. + * + * Return: 0 upon success; -ETIMEDOUT upon failure. + */ +static inline int folio_lock_timeout(struct folio *folio, long timeout) +{ + might_sleep(); + if (!folio_trylock(folio)) + return __folio_lock_timeout(folio, timeout); + return 0; +} + /** * lock_page() - Lock the folio containing this page. * @page: The page to lock. diff --git a/mm/filemap.c b/mm/filemap.c index 2723104cc06a..c6056ec41284 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1220,7 +1220,7 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, int sysctl_page_lock_unfairness = 5; static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, - int state, enum behavior behavior) + int state, enum behavior behavior, long timeout) { wait_queue_head_t *q = folio_waitqueue(folio); int unfairness = sysctl_page_lock_unfairness; @@ -1229,6 +1229,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, bool thrashing = false; unsigned long pflags; bool in_thrashing; + int err; if (bit_nr == PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { @@ -1295,10 +1296,13 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, /* Loop until we've been woken or interrupted */ flags = smp_load_acquire(&wait->flags); if (!(flags & WQ_FLAG_WOKEN)) { + if (!timeout) + break; + if (signal_pending_state(state, current)) break; - io_schedule(); + timeout = io_schedule_timeout(timeout); continue; } @@ -1324,10 +1328,10 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, } /* - * If a signal happened, this 'finish_wait()' may remove the last - * waiter from the wait-queues, but the folio waiters bit will remain - * set. That's ok. The next wakeup will take care of it, and trying - * to do it here would be difficult and prone to races. + * If a signal/timeout happened, this 'finish_wait()' may remove the + * last waiter from the wait-queues, but the folio waiters bit will + * remain set. That's ok. The next wakeup will take care of it, and + * trying to do it here would be difficult and prone to races. */ finish_wait(q, wait); @@ -1336,6 +1340,13 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, psi_memstall_leave(&pflags); } + /* + * If we don't meet the success criteria below then we've got an error + * of some sort. Differentiate between the two error cases. If there's + * no time left it must have been a timeout. + */ + err = !timeout ? -ETIMEDOUT : -EINTR; + /* * NOTE! The wait->flags weren't stable until we've done the * 'finish_wait()', and we could have exited the loop above due @@ -1350,9 +1361,9 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, * waiter, but an exclusive one requires WQ_FLAG_DONE. */ if (behavior == EXCLUSIVE) - return wait->flags & WQ_FLAG_DONE ? 0 : -EINTR; + return wait->flags & WQ_FLAG_DONE ? 0 : err; - return wait->flags & WQ_FLAG_WOKEN ? 0 : -EINTR; + return wait->flags & WQ_FLAG_WOKEN ? 0 : err; } #ifdef CONFIG_MIGRATION @@ -1442,13 +1453,15 @@ void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, void folio_wait_bit(struct folio *folio, int bit_nr) { - folio_wait_bit_common(folio, bit_nr, TASK_UNINTERRUPTIBLE, SHARED); + folio_wait_bit_common(folio, bit_nr, TASK_UNINTERRUPTIBLE, SHARED, + MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL(folio_wait_bit); int folio_wait_bit_killable(struct folio *folio, int bit_nr) { - return folio_wait_bit_common(folio, bit_nr, TASK_KILLABLE, SHARED); + return folio_wait_bit_common(folio, bit_nr, TASK_KILLABLE, SHARED, + MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL(folio_wait_bit_killable); @@ -1467,7 +1480,8 @@ EXPORT_SYMBOL(folio_wait_bit_killable); */ static int folio_put_wait_locked(struct folio *folio, int state) { - return folio_wait_bit_common(folio, PG_locked, state, DROP); + return folio_wait_bit_common(folio, PG_locked, state, DROP, + MAX_SCHEDULE_TIMEOUT); } /** @@ -1662,17 +1676,24 @@ EXPORT_SYMBOL_GPL(page_endio); void __folio_lock(struct folio *folio) { folio_wait_bit_common(folio, PG_locked, TASK_UNINTERRUPTIBLE, - EXCLUSIVE); + EXCLUSIVE, MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL(__folio_lock); int __folio_lock_killable(struct folio *folio) { return folio_wait_bit_common(folio, PG_locked, TASK_KILLABLE, - EXCLUSIVE); + EXCLUSIVE, MAX_SCHEDULE_TIMEOUT); } EXPORT_SYMBOL_GPL(__folio_lock_killable); +int __folio_lock_timeout(struct folio *folio, long timeout) +{ + return folio_wait_bit_common(folio, PG_locked, TASK_KILLABLE, + EXCLUSIVE, timeout); +} +EXPORT_SYMBOL_GPL(__folio_lock_timeout); + static int __folio_lock_async(struct folio *folio, struct wait_page_queue *wait) { struct wait_queue_head *q = folio_waitqueue(folio); From patchwork Fri Apr 21 22:12:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13220738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C2FAC7618E for ; Fri, 21 Apr 2023 22:13:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233802AbjDUWNm (ORCPT ); Fri, 21 Apr 2023 18:13:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233775AbjDUWN3 (ORCPT ); Fri, 21 Apr 2023 18:13:29 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8AE61BE7 for ; Fri, 21 Apr 2023 15:13:26 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-63b64a32fd2so3665877b3a.2 for ; Fri, 21 Apr 2023 15:13:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115205; x=1684707205; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q5AkNpnnC51XtdBu29L1YvEWhYLiet10/hNPEtnlk4k=; b=G4PoB7ojaq05ZoUvu0bGhZVqKkk7zuFsPAQngKsh3YkvxtolTqu1sE5QemG7mwI9bC +lav55U9fdGkURLbrLnJpkwmFYqt+QPV1H3FtmqsyKCWezQ4MHw4sDrMDuq7BBrcauHj vF1OEuZHKgdoBCROSFAfcnRZzV8U5N65M2Rkk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115205; x=1684707205; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q5AkNpnnC51XtdBu29L1YvEWhYLiet10/hNPEtnlk4k=; b=XH359akoa229YGwYu1VkW+fKwGTU8PKAr8Mz2gwMmQ53nORVQlL+3EkkqNRBn/OL7g YamusukWXlbVm/NTCQe6638Dsc0Ghxs0SaWFxu6BUbRvPlJ0WVF1RqrOBhYXfCd9+TZ9 RWSXI/9z9pvvh1sWUI8f6MgEvHWLU0o++sKRRuQnD8mfd4K7zlAcVtmaF6KVRQeVDdck dWCfve5/e32HjN5FtAfU8ty/ZknCkllQsleBgTRxTaitqkAAM0M0n7pU6eBzbT5iIRVp gwWnRedTZ1lXiJ4clIJyq9OMIs004BjMfRpEReROztj04YDiiHCwyiH3962ueEbUWA2g AclA== X-Gm-Message-State: AAQBX9cvfXbv4a+qlNKb2F2CUWDB1XiWsKRvCWl8OId13e0yjC4y2/L8 M79czi4N/9m4RfF6xWn+mQmA6Q== X-Google-Smtp-Source: AKy350Zf5b8NMr8PvbXnnwZSXrydNzrK4lJ4UsMs/fDtsJnXPPkwv+LxUNSlTubSDT1jS6eyThlL9A== X-Received: by 2002:a05:6a00:1393:b0:63f:15cc:9c1a with SMTP id t19-20020a056a00139300b0063f15cc9c1amr6134194pfg.1.1682115205182; Fri, 21 Apr 2023 15:13:25 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:24 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson , Bart Van Assche , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Ingo Molnar , Jan Kara , Juri Lelli , Mel Gorman , Mikulas Patocka , Peter Zijlstra , "Ritesh Harjani (IBM)" , Steven Rostedt , Valentin Schneider , Vincent Guittot , Will Deacon , Zhang Yi Subject: [PATCH v2 2/4] buffer: Add lock_buffer_timeout() Date: Fri, 21 Apr 2023 15:12:46 -0700 Message-ID: <20230421151135.v2.2.Ie146eec4d41480ebeb15f0cfdfb3bc9095e4ebd9@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a variant of lock_buffer() that can timeout. This is useful to avoid unbounded waits for the page lock in kcompactd. Signed-off-by: Douglas Anderson --- Changes in v2: - "Add lock_buffer_timeout()" new for v2. fs/buffer.c | 7 +++++++ include/linux/buffer_head.h | 10 ++++++++++ include/linux/wait_bit.h | 24 ++++++++++++++++++++++++ kernel/sched/wait_bit.c | 14 ++++++++++++++ 4 files changed, 55 insertions(+) diff --git a/fs/buffer.c b/fs/buffer.c index 9e1e2add541e..fcd19c270024 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -71,6 +71,13 @@ void __lock_buffer(struct buffer_head *bh) } EXPORT_SYMBOL(__lock_buffer); +int __lock_buffer_timeout(struct buffer_head *bh, unsigned long timeout) +{ + return wait_on_bit_lock_io_timeout(&bh->b_state, BH_Lock, + TASK_UNINTERRUPTIBLE, timeout); +} +EXPORT_SYMBOL(__lock_buffer_timeout); + void unlock_buffer(struct buffer_head *bh) { clear_bit_unlock(BH_Lock, &bh->b_state); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 8f14dca5fed7..2bae464f89d5 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -237,6 +237,7 @@ struct buffer_head *alloc_buffer_head(gfp_t gfp_flags); void free_buffer_head(struct buffer_head * bh); void unlock_buffer(struct buffer_head *bh); void __lock_buffer(struct buffer_head *bh); +int __lock_buffer_timeout(struct buffer_head *bh, unsigned long timeout); int sync_dirty_buffer(struct buffer_head *bh); int __sync_dirty_buffer(struct buffer_head *bh, blk_opf_t op_flags); void write_dirty_buffer(struct buffer_head *bh, blk_opf_t op_flags); @@ -400,6 +401,15 @@ static inline void lock_buffer(struct buffer_head *bh) __lock_buffer(bh); } +static inline int lock_buffer_timeout(struct buffer_head *bh, + unsigned long timeout) +{ + might_sleep(); + if (!trylock_buffer(bh)) + return __lock_buffer_timeout(bh, timeout); + return 0; +} + static inline struct buffer_head *getblk_unmovable(struct block_device *bdev, sector_t block, unsigned size) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 7725b7579b78..33f0f60b1c8c 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -30,6 +30,7 @@ void wake_up_bit(void *word, int bit); int out_of_line_wait_on_bit(void *word, int, wait_bit_action_f *action, unsigned int mode); int out_of_line_wait_on_bit_timeout(void *word, int, wait_bit_action_f *action, unsigned int mode, unsigned long timeout); int out_of_line_wait_on_bit_lock(void *word, int, wait_bit_action_f *action, unsigned int mode); +int out_of_line_wait_on_bit_lock_timeout(void *word, int, wait_bit_action_f *action, unsigned int mode, unsigned long timeout); struct wait_queue_head *bit_waitqueue(void *word, int bit); extern void __init wait_bit_init(void); @@ -208,6 +209,29 @@ wait_on_bit_lock_io(unsigned long *word, int bit, unsigned mode) return out_of_line_wait_on_bit_lock(word, bit, bit_wait_io, mode); } +/** + * wait_on_bit_lock_io_timeout - wait_on_bit_lock_io() with a timeout + * @word: the word being waited on, a kernel virtual address + * @bit: the bit of the word being waited on + * @mode: the task state to sleep in + * @timeout: the timeout in jiffies; %MAX_SCHEDULE_TIMEOUT means wait forever + * + * Returns zero if the bit was (eventually) found to be clear and was + * set. Returns non-zero if a timeout happened or a signal was delivered to + * the process and the @mode allows that signal to wake the process. + */ +static inline int +wait_on_bit_lock_io_timeout(unsigned long *word, int bit, unsigned mode, + unsigned long timeout) +{ + might_sleep(); + if (!test_and_set_bit(bit, word)) + return 0; + return out_of_line_wait_on_bit_lock_timeout(word, bit, + bit_wait_io_timeout, + mode, timeout); +} + /** * wait_on_bit_lock_action - wait for a bit to be cleared, when wanting to set it * @word: the word being waited on, a kernel virtual address diff --git a/kernel/sched/wait_bit.c b/kernel/sched/wait_bit.c index 0b1cd985dc27..629acd1c6c79 100644 --- a/kernel/sched/wait_bit.c +++ b/kernel/sched/wait_bit.c @@ -118,6 +118,20 @@ int __sched out_of_line_wait_on_bit_lock(void *word, int bit, } EXPORT_SYMBOL(out_of_line_wait_on_bit_lock); +int __sched out_of_line_wait_on_bit_lock_timeout(void *word, int bit, + wait_bit_action_f *action, + unsigned mode, + unsigned long timeout) +{ + struct wait_queue_head *wq_head = bit_waitqueue(word, bit); + DEFINE_WAIT_BIT(wq_entry, word, bit); + + wq_entry.key.timeout = jiffies + timeout; + + return __wait_on_bit_lock(wq_head, &wq_entry, action, mode); +} +EXPORT_SYMBOL(out_of_line_wait_on_bit_lock_timeout); + void __wake_up_bit(struct wait_queue_head *wq_head, void *word, int bit) { struct wait_bit_key key = __WAIT_BIT_KEY_INITIALIZER(word, bit); From patchwork Fri Apr 21 22:12:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13220739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A418C77B78 for ; Fri, 21 Apr 2023 22:13:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233831AbjDUWNo (ORCPT ); Fri, 21 Apr 2023 18:13:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233661AbjDUWN3 (ORCPT ); Fri, 21 Apr 2023 18:13:29 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FC622100 for ; Fri, 21 Apr 2023 15:13:28 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-63b60366047so2266269b3a.1 for ; Fri, 21 Apr 2023 15:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115208; x=1684707208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u93zuMvcW9crJ3xx/ibodoPbhN86+1UwtbTmDpeZKPY=; b=AkSpP2B6MEWZUl0frTvsfsvLWFK8V1GAoZ0xgM/K0816fyASmzFYyukMou4sJkPvvQ x8m+0LZ0Kley490iLqLstsoKAwAWY4snWPkkl+IHwzXeFi5Wamq/VJvtD6G0iFq0oHV0 OAD4MLYCsMD/2Dj11rG/SZNFI7U3EbP8CQc1w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115208; x=1684707208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u93zuMvcW9crJ3xx/ibodoPbhN86+1UwtbTmDpeZKPY=; b=ROH4SUN7mrssvagytf9J53W08+DN0elsUUOnBqrpZs912WRdJOZk1euIgSkac15UJ0 GxIYfNT7+e7HalobP36ZvImbB+09rHzVE2VzSGb0Sj5i5EFCFpOX3nhNskcn0GMSKY3b zEWQSF0jVyL+PEJnjvIqSt9hJ8NGKNnptcnJ0SO9KvQuEjkDkXbbutfs5pwzp2KRRXjO LSGSVualQRfLwBqQKHPrGMyHwLuOc3d3EQRGSO8Wd4sl/9xoZTri0Wt+gxQybrFmhrIv VKziYIyYXMD2QRfDSXvkCwxDU8Qd7icQLbNda1RYxOw7pYdjRFJyWuXVzIhlBKIeLIv1 wdDQ== X-Gm-Message-State: AAQBX9e2eTkq3QlSlrN42CkpBpjpDSjQGjSMapUq+ex8wPIM1M5y2A6Y TIoD/u2LgkTHxmvwogPQl9veLA== X-Google-Smtp-Source: AKy350YerkDyIlCdlYz+2bbDYDMPvivuf/02m3z67Um6oKR/8X1Rqic07nKJ4+CvFtUj6ZUAdN+jUw== X-Received: by 2002:a05:6a00:2196:b0:63a:5bcd:e580 with SMTP id h22-20020a056a00219600b0063a5bcde580mr8793160pfi.9.1682115207703; Fri, 21 Apr 2023 15:13:27 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:26 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson Subject: [PATCH v2 3/4] migrate_pages: Don't wait forever locking pages in MIGRATE_SYNC_LIGHT Date: Fri, 21 Apr 2023 15:12:47 -0700 Message-ID: <20230421151135.v2.3.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The MIGRATE_SYNC_LIGHT mode is intended to block for things that will finish quickly but not for things that will take a long time. Exactly how long is too long is not well defined, but waits of tens of milliseconds is likely non-ideal. Waiting on the folio lock in isolate_movable_page() is something that usually is pretty quick, but is not officially bounded. Nothing stops another process from holding a folio lock while doing an expensive operation. Having an unbounded wait like this is not within the design goals of MIGRATE_SYNC_LIGHT. When putting a Chromebook under memory pressure (opening over 90 tabs on a 4GB machine) it was fairly easy to see delays waiting for the lock of > 100 ms. While the laptop wasn't amazingly usable in this state, it was still limping along and this state isn't something artificial. Sometimes we simply end up with a lot of memory pressure. Putting the same Chromebook under memory pressure while it was running Android apps (though not stressing them) showed a much worse result (NOTE: this was on a older kernel but the codepaths here are similar). Android apps on ChromeOS currently run from a 128K-block, zlib-compressed, loopback-mounted squashfs disk. If we get a page fault from something backed by the squashfs filesystem we could end up holding a folio lock while reading enough from disk to decompress 128K (and then decompressing it using the somewhat slow zlib algorithms). That reading goes through the ext4 subsystem (because it's a loopback mount) before eventually ending up in the block subsystem. This extra jaunt adds extra overhead. Without much work I could see cases where we ended up blocked on a folio lock for over a second. With more more extreme memory pressure I could see up to 25 seconds. Let's bound the amount of time we can wait for the folio lock. The SYNC_LIGHT migration mode can already handle failure for things that are slow, so adding this timeout in is fairly straightforward. With this timeout, it can be seen that kcompactd can move on to more productive tasks if it's taking a long time to acquire a lock. NOTE: The reason I stated digging into this isn't because some benchmark had gone awry, but because we've received in-the-field crash reports where we have a hung task waiting on the page lock (which is the equivalent code path on old kernels). While the root cause of those crashes is likely unrelated and won't be fixed by this patch, analyzing those crash reports did point out this unbounded wait and it seemed like something good to fix. ALSO NOTE: the timeout mechanism used here uses "jiffies" and we also will retry up to 7 times. That doesn't give us much accuracy in specifying the timeout. On 1000 Hz machines we'll end up timing out in 7-14 ms. On 100 Hz machines we'll end up in 70-140 ms. Given that we don't have a strong definition of how long "too long" is, this is probably OK. Suggested-by: Mel Gorman Signed-off-by: Douglas Anderson --- Changes in v2: - Keep unbounded delay in "SYNC", delay with a timeout in "SYNC_LIGHT" mm/migrate.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index db3f154446af..60982df71a93 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -58,6 +58,23 @@ #include "internal.h" +/* Returns the schedule timeout for a non-async mode */ +static long timeout_for_mode(enum migrate_mode mode) +{ + /* + * We'll always return 1 jiffy as the timeout. Since all places using + * this timeout are in a retry loop this means that the maximum time + * we might block is actually NR_MAX_MIGRATE_SYNC_RETRY jiffies. + * If a jiffy is 1 ms that's 7 ms, though with the accuracy of the + * timeouts it often ends up more like 14 ms; if a jiffy is 10 ms + * that's 70-140 ms. + */ + if (mode == MIGRATE_SYNC_LIGHT) + return 1; + + return MAX_SCHEDULE_TIMEOUT; +} + bool isolate_movable_page(struct page *page, isolate_mode_t mode) { struct folio *folio = folio_get_nontail_page(page); @@ -1162,7 +1179,8 @@ static int migrate_folio_unmap(new_page_t get_new_page, free_page_t put_new_page if (current->flags & PF_MEMALLOC) goto out; - folio_lock(src); + if (folio_lock_timeout(src, timeout_for_mode(mode))) + goto out; } locked = true; From patchwork Fri Apr 21 22:12:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13220740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53EA5C7618E for ; Fri, 21 Apr 2023 22:13:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233787AbjDUWNq (ORCPT ); Fri, 21 Apr 2023 18:13:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233778AbjDUWNb (ORCPT ); Fri, 21 Apr 2023 18:13:31 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F8602D5F for ; Fri, 21 Apr 2023 15:13:30 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-63d2ba63dddso2285426b3a.2 for ; Fri, 21 Apr 2023 15:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1682115210; x=1684707210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dc5jbER8wEc7OW3/IotC+kA2A3o7OZfeQaN33ZlBlP4=; b=Bj7hb2g1Ygk+uI1Xxfx+Lf8COV76humjsOT5sJKw8qFdCUzNXODMPPSgSmVEXfumLP gUzdVnED6ugNaAcG2SKbDhlEqm5/AA1x3lyiaYtvzKe1z+7yY9FeeWZjZ4j8ZGNYMvpT nSEalWU0KsSv7vdtph2qjlryw/U/r62x/nUxA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682115210; x=1684707210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dc5jbER8wEc7OW3/IotC+kA2A3o7OZfeQaN33ZlBlP4=; b=ZU8ByIIIQajQt2X6q74RFlkCmizK8ITnStpf/u4Pa8GTylBz1lqvRNsf5/NL5Q0FLS MJ1QjCiN3zbK3bGYhjEtJbFStcm1ahnfmnHy3++p/m+MypEjYxGkiuP+TMXjoM7zyK38 bntaW8oDJEC2gwyQoQ8s1a/Pp/0oAXt3SO/lVcFysr0ufEaevnyJdtfEEZ9ZL/CrdQJQ g7QlgDXqBWb5u6UodRUz3sMdQxxZo0+6xQ+jeJEHKWi8JE6Q1w4s/SIj/TgRN4Gsy/YC VzR/ZS6BLdnFBBe6StcJu9Ip6HEMtnI7GcHXhyp9KMUpqJ8oAR26lK2MiwGhRE1fXqdQ f7xQ== X-Gm-Message-State: AAQBX9dkO3ZRXfsgV6aPPK++mLZdHcdHS1ciWijKHWiyVskkwgiTWfx1 4RfKfOdNS1Rposn1Y6eBC6Gz1g== X-Google-Smtp-Source: AKy350aBr9TvmZeKHCBJSqIpG2B/nWcBgQOHXkcvQWceE4r02qrPTXFu3nXc3bs+0O5+8kUMfAUyog== X-Received: by 2002:a05:6a00:1a56:b0:63d:3789:733f with SMTP id h22-20020a056a001a5600b0063d3789733fmr8752575pfv.15.1682115209980; Fri, 21 Apr 2023 15:13:29 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:87cc:9018:e569:4a27]) by smtp.gmail.com with ESMTPSA id y72-20020a62644b000000b006372791d708sm3424715pfb.104.2023.04.21.15.13.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Apr 2023 15:13:29 -0700 (PDT) From: Douglas Anderson To: Andrew Morton , Mel Gorman , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Douglas Anderson Subject: [PATCH v2 4/4] migrate_pages: Don't wait forever locking buffers in MIGRATE_SYNC_LIGHT Date: Fri, 21 Apr 2023 15:12:48 -0700 Message-ID: <20230421151135.v2.4.Ic39f0b16516acf4af1ce5d923150f93ee85a9398@changeid> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230421221249.1616168-1-dianders@chromium.org> References: <20230421221249.1616168-1-dianders@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Just as talked about in the patch ("migrate_pages: Don't wait forever locking pages in MIGRATE_SYNC_LIGHT"), we don't really want unbounded waits when we're running in MIGRATE_SYNC_LIGHT mode. Waiting on the buffer lock is a second such unbounded wait. Let's put a timeout on it. While measurement didn't show this wait to be quite as bad as the one waiting for the folio lock, it could still be measured to be over a second in some cases. Signed-off-by: Douglas Anderson --- Changes in v2: - "Don't wait forever locking buffers in MIGRATE_SYNC_LIGHT" new for v2. mm/migrate.c | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 60982df71a93..97c93604eb4c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -715,25 +715,16 @@ static bool buffer_migrate_lock_buffers(struct buffer_head *head, enum migrate_mode mode) { struct buffer_head *bh = head; + bool locked; - /* Simple case, sync compaction */ - if (mode != MIGRATE_ASYNC) { - do { - lock_buffer(bh); - bh = bh->b_this_page; - - } while (bh != head); - - return true; - } - - /* async case, we cannot block on lock_buffer so use trylock_buffer */ do { - if (!trylock_buffer(bh)) { - /* - * We failed to lock the buffer and cannot stall in - * async migration. Release the taken locks - */ + if (mode == MIGRATE_ASYNC) + locked = trylock_buffer(bh); + else + locked = !lock_buffer_timeout(bh, timeout_for_mode(mode)); + + if (!locked) { + /* We failed to lock the buffer. Release the taken locks. */ struct buffer_head *failed_bh = bh; bh = head; while (bh != failed_bh) {