From patchwork Tue May 29 21:17:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 10437085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 50CC7604D4 for ; Tue, 29 May 2018 21:17:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42C6528991 for ; Tue, 29 May 2018 21:17:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 36AE52899C; Tue, 29 May 2018 21:17:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF9A128991 for ; Tue, 29 May 2018 21:17:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 419E66B0266; Tue, 29 May 2018 17:17:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3A5426B0269; Tue, 29 May 2018 17:17:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2466C6B026A; Tue, 29 May 2018 17:17:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f197.google.com (mail-qk0-f197.google.com [209.85.220.197]) by kanga.kvack.org (Postfix) with ESMTP id EB04F6B0266 for ; Tue, 29 May 2018 17:17:38 -0400 (EDT) Received: by mail-qk0-f197.google.com with SMTP id s133-v6so6955173qke.21 for ; Tue, 29 May 2018 14:17:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references; bh=OnLfniKgYvskwWLXSUKc15Ogio1wVy7+KmfqXSJANSw=; b=dIgHG4S/0v3VQXVJ668DUf6uT1IzZgIxIZ227667vnlm1d4mf7qIchprw7hLIFdJxe yqSBUQJDrSUKaGba2dPmprVTedKtslP/tl1ZX1u/FZHiNkENnUG+dd8359EZM4ivtZcY vm55US44GMauM8nZzLY0SnDTxLidUwDJUZgHwKovRjO3+o2ABnZHXwH2/gO1i0Z0j63H b3Pn1Jjcd2bQ0c2e0/46TB6PxDmugTsb11Ay67kr3f3RU2VxRm/teqnbQ08loS3BCp33 TKRWvcjbuDL77o8kp4PIWBJ4vRSCrliIr4gnlP1BTDK9JEq3PTdKXdenhCSpiUnE5U3p kWHQ== X-Gm-Message-State: APt69E2KRAzSv8gbgD072zyvukDqV+1KdvH+0lKAGVvn/OnXvB0tqke0 GOCsg6WyUyLSV1/beauWz4gioJh2nwG/TVkL3yLZUcklHJCmHAPvQakz51aXNVmC8iv6CiRkhPK w6N2k1jfOnSy8oNjzRHzInvZIn9OgFf/IQR6GDdr86HTfvkX6Jse1EbzPHr9NVI3no3EEhg0hb8 WCMelYBLpM1AQx8AlZ5pSPY3jkWPB4hH1b9B4OgVHwAQQyptGST6gPNi4UN19BICuQL6TrRRMWp d71l1fzPwIA1X3rNnyUVTdJZqkBCG5eBuCuY9UMMUDGP93IcznIhHKDENt71vqRa2nTHFCOCm5P BxwrStzwbGHBFo4ZwovtKsnSWVvlAylSGrSsHbRhMqYT/XfP0PZLBvj4VpNlo5k1yYgLl6gIx0+ y X-Received: by 2002:a0c:9308:: with SMTP id d8-v6mr49915qvd.191.1527628658728; Tue, 29 May 2018 14:17:38 -0700 (PDT) X-Received: by 2002:a0c:9308:: with SMTP id d8-v6mr49878qvd.191.1527628657989; Tue, 29 May 2018 14:17:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527628657; cv=none; d=google.com; s=arc-20160816; b=y3qzapYx090yQZbWE/Tcg2xUF9mkv+PxXOt3Zq0UEZYW+OzyQqG1QbbdZP9T8wEC6E LAvpDpPomBFQ76ph9JrpJaXBcfHiIJmYeWzkMqrt5aAoKgvgG9wToIl8Gua+warRoDCH /a6UzfVIMJh10VfBsqmOTnhSYHCYyMgMUWlDXF3hdL9L496jNsozESbonybGmi0jrTsr OQ6UXdZitZzpFGQ2voTWO0vtVPhilu+UlijdLehxEb5+tsYNBHHVpULLffpfk6J1xuOF VV48lij1ksUOI23me3+F1pFByjH0xiIfHcHt5Pb5tbY5n69DXu74eODWPTHOmLg1k5EC v+jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:to:from :dkim-signature:arc-authentication-results; bh=OnLfniKgYvskwWLXSUKc15Ogio1wVy7+KmfqXSJANSw=; b=NGO2WN6S5JvOjxqZ/tYakgc7nONVw4WnUb6U7n7QbaaeEZ/Rvnqk5Rf+iJFGh+Vvbj tugGBNLnXoCEzwVb4nkq6hoe5FOLbYFFBm9OILHMVgC/R4/rbXxXh6rtXLTds2tcBYqd Yi5iVLZoVd0n1mhU0emmtBi+a0EPOV5k3W7wpJzc6IhEP1oNx7CPTRVOL+RqcmgMtBzT GBSl5FvTOHJXvBP1O3u5CrEpyLmwxlqTSX4pP90LavUCbDSTnH+iUJ+n7miMctJq+d1V jn6uqhGq/uNEE6EN9WL8oEatklf7E+GFcX2AeqB04kxjdXO+oC+P4v+x96kU2jRRxrZ4 4Ntg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@toxicpanda-com.20150623.gappssmtp.com header.s=20150623 header.b=ba/vFuCJ; spf=neutral (google.com: 209.85.220.65 is neither permitted nor denied by best guess record for domain of josef@toxicpanda.com) smtp.mailfrom=josef@toxicpanda.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id f31-v6sor21317475qta.8.2018.05.29.14.17.37 for (Google Transport Security); Tue, 29 May 2018 14:17:37 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.65 is neither permitted nor denied by best guess record for domain of josef@toxicpanda.com) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@toxicpanda-com.20150623.gappssmtp.com header.s=20150623 header.b=ba/vFuCJ; spf=neutral (google.com: 209.85.220.65 is neither permitted nor denied by best guess record for domain of josef@toxicpanda.com) smtp.mailfrom=josef@toxicpanda.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=OnLfniKgYvskwWLXSUKc15Ogio1wVy7+KmfqXSJANSw=; b=ba/vFuCJsuymw9g2OXls8eUduizbkpWsZE1tBt53Bxls7YwdBimttLnBz2ClxClVoL zEU905JfzhUpXdtUxbZHMqFjQWpJkteixUGMD37n14N0qn7vvq1NSTZHc4lRFFTGvbrI iJN/ShF/RV6rTNBlp5uBQiYn61p5HgVgqz/L/UAj9LWWTjn52HycRr8Hr3jhP80OZhNs N4sO1sovnk60hs2oBqqO5R9B+CIx/hSVMB/J4B3R+oSda9jh6CxdtORnYd3+Mgt6Uqns JiaLmZtFfW5bkn82efUTT17w2Pt6YeFuDE7CL7zdKIVGFwnEy8QN+1C+VDvuO3RylgEF uUnA== X-Google-Smtp-Source: ADUXVKKIvwUqU7vC0+vLDAA2I7uxeDDG1pW5+MNyzUxb6vg+Xyw9a5GM95vjIYEHesdo2tuTAhi7zQ== X-Received: by 2002:ac8:2e1c:: with SMTP id r28-v6mr89382qta.156.1527628657716; Tue, 29 May 2018 14:17:37 -0700 (PDT) Received: from localhost (cpe-2606-A000-4381-1201-225-22FF-FEB3-E51A.dyn6.twc.com. [2606:a000:4381:1201:225:22ff:feb3:e51a]) by smtp.gmail.com with ESMTPSA id r25-v6sm5236761qtc.39.2018.05.29.14.17.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 May 2018 14:17:37 -0700 (PDT) From: Josef Bacik To: axboe@kernel.dk, kernel-team@fb.com, linux-block@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, hannes@cmpxchg.org, linux-kernel@vger.kernel.org, tj@kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 07/13] memcontrol: schedule throttling if we are congested Date: Tue, 29 May 2018 17:17:18 -0400 Message-Id: <20180529211724.4531-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180529211724.4531-1-josef@toxicpanda.com> References: <20180529211724.4531-1-josef@toxicpanda.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Tejun Heo Memory allocations can induce swapping via kswapd or direct reclaim. If we are having IO done for us by kswapd and don't actually go into direct reclaim we may never get scheduled for throttling. So instead check to see if our cgroup is congested, and if so schedule the throttling. Before we return to user space the throttling stuff will only throttle if we actually required it. Signed-off-by: Tejun Heo --- include/linux/memcontrol.h | 13 +++++++++++++ mm/huge_memory.c | 6 +++--- mm/memcontrol.c | 24 ++++++++++++++++++++++++ mm/memory.c | 11 ++++++----- mm/shmem.c | 10 +++++----- 5 files changed, 51 insertions(+), 13 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index d99b71bc2c66..4d2e7f35f2dc 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -290,6 +290,9 @@ bool mem_cgroup_low(struct mem_cgroup *root, struct mem_cgroup *memcg); int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask, struct mem_cgroup **memcgp, bool compound); +int mem_cgroup_try_charge_delay(struct page *page, struct mm_struct *mm, + gfp_t gfp_mask, struct mem_cgroup **memcgp, + bool compound); void mem_cgroup_commit_charge(struct page *page, struct mem_cgroup *memcg, bool lrucare, bool compound); void mem_cgroup_cancel_charge(struct page *page, struct mem_cgroup *memcg, @@ -745,6 +748,16 @@ static inline int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm, return 0; } +static inline int mem_cgroup_try_charge_delay(struct page *page, + struct mm_struct *mm, + gfp_t gfp_mask, + struct mem_cgroup **memcgp, + bool compound) +{ + *memcgp = NULL; + return 0; +} + static inline void mem_cgroup_commit_charge(struct page *page, struct mem_cgroup *memcg, bool lrucare, bool compound) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a3a1815f8e11..9812ddad9961 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -555,7 +555,7 @@ static int __do_huge_pmd_anonymous_page(struct vm_fault *vmf, struct page *page, VM_BUG_ON_PAGE(!PageCompound(page), page); - if (mem_cgroup_try_charge(page, vma->vm_mm, gfp, &memcg, true)) { + if (mem_cgroup_try_charge_delay(page, vma->vm_mm, gfp, &memcg, true)) { put_page(page); count_vm_event(THP_FAULT_FALLBACK); return VM_FAULT_FALLBACK; @@ -1145,7 +1145,7 @@ static int do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, pmd_t orig_pmd, pages[i] = alloc_page_vma_node(GFP_HIGHUSER_MOVABLE, vma, vmf->address, page_to_nid(page)); if (unlikely(!pages[i] || - mem_cgroup_try_charge(pages[i], vma->vm_mm, + mem_cgroup_try_charge_delay(pages[i], vma->vm_mm, GFP_KERNEL, &memcg, false))) { if (pages[i]) put_page(pages[i]); @@ -1315,7 +1315,7 @@ int do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) goto out; } - if (unlikely(mem_cgroup_try_charge(new_page, vma->vm_mm, + if (unlikely(mem_cgroup_try_charge_delay(new_page, vma->vm_mm, huge_gfp, &memcg, true))) { put_page(new_page); split_huge_pmd(vma, vmf->pmd, vmf->address); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 2bd3df3d101a..81f61b66d27a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5458,6 +5458,30 @@ int mem_cgroup_try_charge(struct page *page, struct mm_struct *mm, return ret; } +int mem_cgroup_try_charge_delay(struct page *page, struct mm_struct *mm, + gfp_t gfp_mask, struct mem_cgroup **memcgp, + bool compound) +{ + struct mem_cgroup *memcg; + struct block_device *bdev; + int ret; + + ret = mem_cgroup_try_charge(page, mm, gfp_mask, memcgp, compound); + memcg = *memcgp; + + if (!(gfp_mask & __GFP_IO) || !memcg) + return ret; +#if defined(CONFIG_BLOCK) && defined(CONFIG_SWAP) + if (atomic_read(&memcg->css.cgroup->congestion_count) && + has_usable_swap()) { + map_swap_page(page, &bdev); + + blkcg_schedule_throttle(bdev_get_queue(bdev), true); + } +#endif + return ret; +} + /** * mem_cgroup_commit_charge - commit a page charge * @page: page to charge diff --git a/mm/memory.c b/mm/memory.c index 01f5464e0fd2..d0eea6d33b18 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2494,7 +2494,7 @@ static int wp_page_copy(struct vm_fault *vmf) cow_user_page(new_page, old_page, vmf->address, vma); } - if (mem_cgroup_try_charge(new_page, mm, GFP_KERNEL, &memcg, false)) + if (mem_cgroup_try_charge_delay(new_page, mm, GFP_KERNEL, &memcg, false)) goto oom_free_new; __SetPageUptodate(new_page); @@ -2994,8 +2994,8 @@ int do_swap_page(struct vm_fault *vmf) goto out_page; } - if (mem_cgroup_try_charge(page, vma->vm_mm, GFP_KERNEL, - &memcg, false)) { + if (mem_cgroup_try_charge_delay(page, vma->vm_mm, GFP_KERNEL, + &memcg, false)) { ret = VM_FAULT_OOM; goto out_page; } @@ -3156,7 +3156,8 @@ static int do_anonymous_page(struct vm_fault *vmf) if (!page) goto oom; - if (mem_cgroup_try_charge(page, vma->vm_mm, GFP_KERNEL, &memcg, false)) + if (mem_cgroup_try_charge_delay(page, vma->vm_mm, GFP_KERNEL, &memcg, + false)) goto oom_free_page; /* @@ -3652,7 +3653,7 @@ static int do_cow_fault(struct vm_fault *vmf) if (!vmf->cow_page) return VM_FAULT_OOM; - if (mem_cgroup_try_charge(vmf->cow_page, vma->vm_mm, GFP_KERNEL, + if (mem_cgroup_try_charge_delay(vmf->cow_page, vma->vm_mm, GFP_KERNEL, &vmf->memcg, false)) { put_page(vmf->cow_page); return VM_FAULT_OOM; diff --git a/mm/shmem.c b/mm/shmem.c index 9d6c7e595415..a96af5690864 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1219,8 +1219,8 @@ int shmem_unuse(swp_entry_t swap, struct page *page) * the shmem_swaplist_mutex which might hold up shmem_writepage(). * Charged back to the user (not to caller) when swap account is used. */ - error = mem_cgroup_try_charge(page, current->mm, GFP_KERNEL, &memcg, - false); + error = mem_cgroup_try_charge_delay(page, current->mm, GFP_KERNEL, + &memcg, false); if (error) goto out; /* No radix_tree_preload: swap entry keeps a place for page in tree */ @@ -1697,7 +1697,7 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, goto failed; } - error = mem_cgroup_try_charge(page, charge_mm, gfp, &memcg, + error = mem_cgroup_try_charge_delay(page, charge_mm, gfp, &memcg, false); if (!error) { error = shmem_add_to_page_cache(page, mapping, index, @@ -1803,7 +1803,7 @@ alloc_nohuge: page = shmem_alloc_and_acct_page(gfp, inode, if (sgp == SGP_WRITE) __SetPageReferenced(page); - error = mem_cgroup_try_charge(page, charge_mm, gfp, &memcg, + error = mem_cgroup_try_charge_delay(page, charge_mm, gfp, &memcg, PageTransHuge(page)); if (error) goto unacct; @@ -2276,7 +2276,7 @@ static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, __SetPageSwapBacked(page); __SetPageUptodate(page); - ret = mem_cgroup_try_charge(page, dst_mm, gfp, &memcg, false); + ret = mem_cgroup_try_charge_delay(page, dst_mm, gfp, &memcg, false); if (ret) goto out_release;