From patchwork Thu Mar 12 17:32:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435129 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB35214E5 for ; Thu, 12 Mar 2020 17:32:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E586206B1 for ; Thu, 12 Mar 2020 17:32:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="LB2Rpnhy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E586206B1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CA9CE6B006E; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5A2F6B0070; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B703C6B0071; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0108.hostedemail.com [216.40.44.108]) by kanga.kvack.org (Postfix) with ESMTP id 9F2FA6B006E for ; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 63DD2499606 for ; Thu, 12 Mar 2020 17:32:54 +0000 (UTC) X-FDA: 76587405468.22.dog63_366bc9d6cfa04 X-Spam-Summary: 2,0,0,69d90c6b393dadc6,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:966:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1535:1542:1593:1594:1595:1596:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2740:2898:2904:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3872:4321:4385:5007:6261:6653:8531:10004:10400:10450:10455:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13439:13846:13895:14096:14097:14181:14394:14721:19904:19999:21080:21444:21451:21627:21740:21990:30054:30064:30070,0,RBL:209.85.221.65:@chrisdown.name:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: dog63_366bc9d6cfa04 X-Filterd-Recvd-Size: 5797 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:32:53 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id n15so8534086wrw.13 for ; Thu, 12 Mar 2020 10:32:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=CcUuUlC43ZZSpfh7MoEkxpHRz/YQaDQ0sa4yKp/08VA=; b=LB2RpnhyFSTH3hVgjWSNFLifh6wVwkygWTfWLNDnxQPIIBKr4BpapuRzF1lJYa0M15 UbSB1s+QSe7P8Bx+GLZMQ+cV7gO4cfbRWGjeLOLvLCaijAjS6Lpr8ZOYSHWG8HOqN/hu kT1oLw9Sjvc/qjL4Kh04/aFvpCsvKUXlmqg/Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=CcUuUlC43ZZSpfh7MoEkxpHRz/YQaDQ0sa4yKp/08VA=; b=pD2RFz/84x+8xNyTqOO+ntoGjloTLmyi55G4iZQ7mvnEDjMpWWauHu2BSCuZeEXBsy 0aLSI4b4FA39jG6ogHAPHxLgTnGjIHZImaihlHLpsflUfBDSreccAgt7vIoe3gctANCD RX7icDws3D553PA4SWaPjvYglPHTFVJXtbcf+G0vYqCujk6fP59cy2YwXZ9mIKJ6f2PA HGS7pEKRCt8ZqeeLkSS5nyPIafbAk+b3OZfyeSpQ40n8/iKKcW+j2k6heEKlT6g1dTwQ 4wgO87AuQr7vR0VkKzLiosHom8J2jhf03ADNuuF0V8RbYIG5Oby3i2Q/MfS9c56QeWtj nVFw== X-Gm-Message-State: ANhLgQ3fBMSIi+l3+B/wWH2IoN1BqLKdVS6P/z7fgg5IWM8GR1Gx3VY3 74Wp3F5muJ7Tn+tObIABU2cQCg== X-Google-Smtp-Source: ADFU+vtgf9rQPYcJ7fNM8yiJe0orpohUFakZwlPq0AegakMNuqTr8+u5nhAj4dJIgtmv3wka95/7/g== X-Received: by 2002:a05:6000:1008:: with SMTP id a8mr6809577wrx.8.1584034372767; Thu, 12 Mar 2020 10:32:52 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id z6sm20259371wru.15.2020.03.12.10.32.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:32:52 -0700 (PDT) Date: Thu, 12 Mar 2020 17:32:51 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/6] mm, memcg: Prevent memory.high load/store tearing Message-ID: <2f66f7038ed1d4688e59de72b627ae0ea52efa83.1584034301.git.chris@chrisdown.name> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A mem_cgroup's high attribute can be concurrently set at the same time as we are trying to read it -- for example, if we are in memory_high_write at the same time as we are trying to do high reclaim. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 63bb6a2aab81..d32d3c0a16d4 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2228,7 +2228,7 @@ static void reclaim_high(struct mem_cgroup *memcg, gfp_t gfp_mask) { do { - if (page_counter_read(&memcg->memory) <= memcg->high) + if (page_counter_read(&memcg->memory) <= READ_ONCE(memcg->high)) continue; memcg_memory_event(memcg, MEMCG_HIGH); try_to_free_mem_cgroup_pages(memcg, nr_pages, gfp_mask, true); @@ -2545,7 +2545,7 @@ static int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, * reclaim, the cost of mismatch is negligible. */ do { - if (page_counter_read(&memcg->memory) > memcg->high) { + if (page_counter_read(&memcg->memory) > READ_ONCE(memcg->high)) { /* Don't bother a random interrupted task */ if (in_interrupt()) { schedule_work(&memcg->high_work); @@ -4257,7 +4257,8 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages, *pheadroom = PAGE_COUNTER_MAX; while ((parent = parent_mem_cgroup(memcg))) { - unsigned long ceiling = min(memcg->memory.max, memcg->high); + unsigned long ceiling = min(memcg->memory.max, + READ_ONCE(memcg->high)); unsigned long used = page_counter_read(&memcg->memory); *pheadroom = min(*pheadroom, ceiling - min(ceiling, used)); @@ -4978,7 +4979,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) if (!memcg) return ERR_PTR(error); - memcg->high = PAGE_COUNTER_MAX; + WRITE_ONCE(memcg->high, PAGE_COUNTER_MAX); memcg->soft_limit = PAGE_COUNTER_MAX; if (parent) { memcg->swappiness = mem_cgroup_swappiness(parent); @@ -5131,7 +5132,7 @@ static void mem_cgroup_css_reset(struct cgroup_subsys_state *css) page_counter_set_max(&memcg->tcpmem, PAGE_COUNTER_MAX); page_counter_set_min(&memcg->memory, 0); page_counter_set_low(&memcg->memory, 0); - memcg->high = PAGE_COUNTER_MAX; + WRITE_ONCE(memcg->high, PAGE_COUNTER_MAX); memcg->soft_limit = PAGE_COUNTER_MAX; memcg_wb_domain_size_changed(memcg); } @@ -5947,7 +5948,7 @@ static ssize_t memory_high_write(struct kernfs_open_file *of, if (err) return err; - memcg->high = high; + WRITE_ONCE(memcg->high, high); for (;;) { unsigned long nr_pages = page_counter_read(&memcg->memory); From patchwork Thu Mar 12 17:32:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435131 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E125714E5 for ; Thu, 12 Mar 2020 17:33:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A39152071B for ; Thu, 12 Mar 2020 17:33:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="RsW/vK4Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A39152071B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CA9A66B0071; Thu, 12 Mar 2020 13:32:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C59CF6B0072; Thu, 12 Mar 2020 13:32:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B70C56B0073; Thu, 12 Mar 2020 13:32:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0213.hostedemail.com [216.40.44.213]) by kanga.kvack.org (Postfix) with ESMTP id 99B9A6B0071 for ; Thu, 12 Mar 2020 13:32:59 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 532F08248076 for ; Thu, 12 Mar 2020 17:32:59 +0000 (UTC) X-FDA: 76587405678.18.glue95_372b89d4a6419 X-Spam-Summary: 2,0,0,690d4f1eb0ce41d8,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1535:1542:1593:1594:1595:1596:1711:1730:1747:1777:1792:2393:2559:2562:2892:2897:2901:3138:3139:3140:3141:3142:3353:3865:3867:3868:3871:3872:3874:4321:5007:6261:6653:10004:10400:11026:11232:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13439:13846:13895:14181:14394:14721:21080:21444:21451:21627:30029:30054:30056:30064:30075,0,RBL:209.85.128.68:@chrisdown.name:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: glue95_372b89d4a6419 X-Filterd-Recvd-Size: 5797 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:32:58 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id 6so7007310wmi.5 for ; Thu, 12 Mar 2020 10:32:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=rC6DPhMt4Bc6PZBfsWnH9m+te2MxYFVp1w0OPHy/TMg=; b=RsW/vK4ZQdfMF082vtv3xGDq0tnOXulVLSIzxMkgnp71K/k5wE9FsaYAqfwBT7Mi8T 1JGkpiXJS/cZde2IHQWUKO2lYZzaiwID5eItcGzhgvtHijHI/MSk4nmoIOz9KNm30+Yg FUB3Lg42z8vn9pQn08JwbZJE+I0nAu8whv3GM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=rC6DPhMt4Bc6PZBfsWnH9m+te2MxYFVp1w0OPHy/TMg=; b=KTKTxMkCEvo4q7eczA4K4hEU2SFeeIaqKQCFxUoi8UQdgrcPua7f4t0V3RAKyFJzlV AhlgTV+HI0meGVSxmBfu0tSi4uLhmVbsfxYcOYnHFXq2tXUkGmChH0YpuizER9Sj1ZFe mMGtj7Ilr7u4caY36nnlEilNnkxKBcC8KIP/BBtGJkNbQx+DljQN4mlY8LUGeWJkVUP2 BYdqZ3USUkFodPigRQFOlkxOBhqLDGL+4Y4SxgekKhi9LuKQ5BGWRoJhtjvqJ0SsBXYX G+AGejDRTsSdUyCIRmxv7fvn0Ispq6uMz4jvXx61zFsdS3OLRI6DSMWU3z7+/0IQLJ5X 4GXA== X-Gm-Message-State: ANhLgQ19lYfIZ062qFs4ZqgsZ7rn5NTYlcD86eyH6Qj+922zLu1rlmEZ AzBHQ/2IOcjI9o7E2+VeTp8U5Q== X-Google-Smtp-Source: ADFU+vuUWIJm8HtSoJx05RRAuGsJ1TYnW585UUfshNPcO1zgDpwRCCY5Y8dEGgjUnxBmIS6TJK1gTw== X-Received: by 2002:a7b:cfc9:: with SMTP id f9mr5541876wmm.127.1584034377704; Thu, 12 Mar 2020 10:32:57 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id y16sm19541988wrn.63.2020.03.12.10.32.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:32:57 -0700 (PDT) Date: Thu, 12 Mar 2020 17:32:56 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 2/6] mm, memcg: Prevent memory.max load tearing Message-ID: <50a31e5f39f8ae6c8fb73966ba1455f0924e8f44.1584034301.git.chris@chrisdown.name> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This one is a bit more nuanced because we have memcg_max_mutex, which is mostly just used for enforcing invariants, but we still need to READ_ONCE since (despite its name) it doesn't really protect memory.max access. On write (page_counter_set_max() and memory_max_write()) we use xchg(), which uses smp_mb(), so that's already fine. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d32d3c0a16d4..aca2964ea494 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1507,7 +1507,7 @@ void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) pr_info("memory: usage %llukB, limit %llukB, failcnt %lu\n", K((u64)page_counter_read(&memcg->memory)), - K((u64)memcg->memory.max), memcg->memory.failcnt); + K((u64)READ_ONCE(memcg->memory.max)), memcg->memory.failcnt); if (cgroup_subsys_on_dfl(memory_cgrp_subsys)) pr_info("swap: usage %llukB, limit %llukB, failcnt %lu\n", K((u64)page_counter_read(&memcg->swap)), @@ -1538,7 +1538,7 @@ unsigned long mem_cgroup_get_max(struct mem_cgroup *memcg) { unsigned long max; - max = memcg->memory.max; + max = READ_ONCE(memcg->memory.max); if (mem_cgroup_swappiness(memcg)) { unsigned long memsw_max; unsigned long swap_max; @@ -3006,7 +3006,7 @@ static int mem_cgroup_resize_max(struct mem_cgroup *memcg, * Make sure that the new limit (memsw or memory limit) doesn't * break our basic invariant rule memory.max <= memsw.max. */ - limits_invariant = memsw ? max >= memcg->memory.max : + limits_invariant = memsw ? max >= READ_ONCE(memcg->memory.max) : max <= memcg->memsw.max; if (!limits_invariant) { mutex_unlock(&memcg_max_mutex); @@ -3753,8 +3753,8 @@ static int memcg_stat_show(struct seq_file *m, void *v) /* Hierarchical information */ memory = memsw = PAGE_COUNTER_MAX; for (mi = memcg; mi; mi = parent_mem_cgroup(mi)) { - memory = min(memory, mi->memory.max); - memsw = min(memsw, mi->memsw.max); + memory = min(memory, READ_ONCE(mi->memory.max)); + memsw = min(memsw, READ_ONCE(mi->memsw.max)); } seq_printf(m, "hierarchical_memory_limit %llu\n", (u64)memory * PAGE_SIZE); @@ -4257,7 +4257,7 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages, *pheadroom = PAGE_COUNTER_MAX; while ((parent = parent_mem_cgroup(memcg))) { - unsigned long ceiling = min(memcg->memory.max, + unsigned long ceiling = min(READ_ONCE(memcg->memory.max), READ_ONCE(memcg->high)); unsigned long used = page_counter_read(&memcg->memory); From patchwork Thu Mar 12 17:33:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435133 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 943686CA for ; Thu, 12 Mar 2020 17:33:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 605A6206B1 for ; Thu, 12 Mar 2020 17:33:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="qdHoaPhx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 605A6206B1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 556A16B0072; Thu, 12 Mar 2020 13:33:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 506C16B0073; Thu, 12 Mar 2020 13:33:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41D306B0074; Thu, 12 Mar 2020 13:33:04 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0107.hostedemail.com [216.40.44.107]) by kanga.kvack.org (Postfix) with ESMTP id 28CA56B0072 for ; Thu, 12 Mar 2020 13:33:04 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DAAD98E7F for ; Thu, 12 Mar 2020 17:33:03 +0000 (UTC) X-FDA: 76587405846.16.root74_37d08df31a544 X-Spam-Summary: 2,0,0,9e24ad115754caf5,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1535:1542:1593:1594:1595:1596:1711:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3353:3865:3867:4321:4605:5007:6261:6653:7904:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13439:13846:13895:14096:14097:14181:14394:14721:21080:21444:21451:21627:21990:30054:30064:30090,0,RBL:209.85.221.68:@chrisdown.name:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: root74_37d08df31a544 X-Filterd-Recvd-Size: 5255 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:33:03 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id 6so8568929wre.4 for ; Thu, 12 Mar 2020 10:33:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Wi04zoB+1lwUS/JkrIWiJtz6qrlQUGWxV2d3AofBbt4=; b=qdHoaPhxQicwt6H3R0TCGLXYf5k3L9ooFg374fONxjv8C3rvNxgZwp+11xl3vl5riH 0+LwXAumxpp4RyUzxzdWqtEjG5BG9CDRPl4N6+xAtMsl5UUdZ8hz2enRswBOwYzfdka9 XGBadlITdSJeRvdGcod95gw7PRehSEFfoQdJU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Wi04zoB+1lwUS/JkrIWiJtz6qrlQUGWxV2d3AofBbt4=; b=ucltX9TPHMCXJOZWmtXrtY+LUxo6A1YWvtEnxYxUe9JhGj6w5VlduABdgWevTUqIjj f6HJJU1XIyHjUkpKpkxaIlAXW7TuxoIX3dXjTs6Ae6QoDfldHw+PajcaSaVXXrGK1zjf z+dWROL8KHDWPMqRupG9LQPO2Uv2GDsaBdNipm9q3i7D88R7up5EJ4DK3Ubx/RTwNLZw AU6KTvF/FsvUo/+jqnllmZJgk+BXr5N+kI2YhoqR10a4TozYGNxjf/KhClHuu8NWPFwZ 7IuAqagT1nHyf+LxVHTRMfDN8SUJzfKS6Ivye4rg5UpoPu7yiMI/dvqQei36Po6Y9iCK ZEnQ== X-Gm-Message-State: ANhLgQ2X+XL/LE7d5JXZhXBVjNNbVJ4d1B+ecy5hv+0TWcU3Qt+K0XWu yCMJgqfXzTnokIqMakn8/sfy3w== X-Google-Smtp-Source: ADFU+vs6lh4JtrMC8cO3JiGvHgD7AQUqQmn80t3o/bMOWdGzTkGWTWbLaTayMV0C4e7AmUIfCKkQxw== X-Received: by 2002:adf:e4ca:: with SMTP id v10mr11831736wrm.305.1584034382407; Thu, 12 Mar 2020 10:33:02 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id s7sm9500905wro.10.2020.03.12.10.33.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:33:01 -0700 (PDT) Date: Thu, 12 Mar 2020 17:33:01 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 3/6] mm, memcg: Prevent memory.low load/store tearing Message-ID: <448206f44b0fa7be9dad2ca2601d2bcb2c0b7844.1584034301.git.chris@chrisdown.name> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This can be set concurrently with reads, which may cause the wrong value to be propagated. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 4 ++-- mm/page_counter.c | 9 ++++++--- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index aca2964ea494..c85a304fa4a1 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6262,7 +6262,7 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root, return MEMCG_PROT_NONE; emin = memcg->memory.min; - elow = memcg->memory.low; + elow = READ_ONCE(memcg->memory.low); parent = parent_mem_cgroup(memcg); /* No parent means a non-hierarchical mode on v1 memcg */ @@ -6291,7 +6291,7 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root, if (elow && parent_elow) { unsigned long low_usage, siblings_low_usage; - low_usage = min(usage, memcg->memory.low); + low_usage = min(usage, READ_ONCE(memcg->memory.low)); siblings_low_usage = atomic_long_read( &parent->memory.children_low_usage); diff --git a/mm/page_counter.c b/mm/page_counter.c index 50184929b61f..18b7f779f2e2 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -17,6 +17,7 @@ static void propagate_protected_usage(struct page_counter *c, unsigned long usage) { unsigned long protected, old_protected; + unsigned long low; long delta; if (!c->parent) @@ -34,8 +35,10 @@ static void propagate_protected_usage(struct page_counter *c, atomic_long_add(delta, &c->parent->children_min_usage); } - if (c->low || atomic_long_read(&c->low_usage)) { - if (usage <= c->low) + low = READ_ONCE(c->low); + + if (low || atomic_long_read(&c->low_usage)) { + if (usage <= low) protected = usage; else protected = 0; @@ -231,7 +234,7 @@ void page_counter_set_low(struct page_counter *counter, unsigned long nr_pages) { struct page_counter *c; - counter->low = nr_pages; + WRITE_ONCE(counter->low, nr_pages); for (c = counter; c; c = c->parent) propagate_protected_usage(c, atomic_long_read(&c->usage)); From patchwork Thu Mar 12 17:33:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2F8614E5 for ; Thu, 12 Mar 2020 17:33:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7F55520753 for ; Thu, 12 Mar 2020 17:33:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="up0/TKVQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F55520753 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 863E26B0073; Thu, 12 Mar 2020 13:33:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 813E56B0074; Thu, 12 Mar 2020 13:33:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 750E66B0075; Thu, 12 Mar 2020 13:33:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 5AD9E6B0073 for ; Thu, 12 Mar 2020 13:33:10 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F416282499A8 for ; Thu, 12 Mar 2020 17:33:09 +0000 (UTC) X-FDA: 76587406140.14.brass43_38b26213fef2f X-Spam-Summary: 2,0,0,f5c5f56139783b99,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1535:1542:1593:1594:1595:1596:1711:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3353:3865:3867:4605:5007:6261:6653:7904:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13439:13846:13895:14096:14097:14181:14394:14721:21080:21444:21451:21627:21990:30054:30064:30090,0,RBL:209.85.128.65:@chrisdown.name:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: brass43_38b26213fef2f X-Filterd-Recvd-Size: 5119 Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:33:09 +0000 (UTC) Received: by mail-wm1-f65.google.com with SMTP id a132so7026334wme.1 for ; Thu, 12 Mar 2020 10:33:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=XlHPYdHYVGL+ZH9Fy2ExhyAo23ZIkMLKS1ZUcqxxQY0=; b=up0/TKVQ7COgyqEd0H7isdZkegQuEbx6al4+9E95wP3QiMQsdxvH9836eQ+LluSIC5 RrbHhXsGGbJAFQXjNgehCBmeLiF93Sg8ahlpsQiK3ct/C8obWCIZR0CUU9z9GnAc5QUt /KMm82i4Ln7WBp4XE3Bm+5wpBL7eIdIAKbhSk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=XlHPYdHYVGL+ZH9Fy2ExhyAo23ZIkMLKS1ZUcqxxQY0=; b=cw+LgVsTA3i0EFsyCr/o3Pep8a5YvGWSgSjYwrpv3oEQQhiepw/zWJONUN3qmE+hX1 K6Fd1f+QWr9056IJ+nCDiz53jXnFGuka8+2S4xwdWHWFUv0vBg0GuSB28ytLjPumBeap nkqMxREesXaIU8mh7dT/+Yfy0zSG7uuJmQIVVMIfgm5sxp1zavU9IAh0ibGWk7kHDcAk LoKdAdDUdr5ST0/1lxwoE13biHSbO9C1jeuMZZNGARgFjI8JeVJvZ+Q3KI+vzYTju1R8 4bCeieQL04DIfIh1UwCL3ox2/mf/S/ki0PiS6Fbgs+K7azb2bhjHNmiheeyLpHFtrHhv 8nSA== X-Gm-Message-State: ANhLgQ1Bl1tuAHYOzggga503fatF30nvPV4OhOQl/LAmu+IPbX4nDTBv qT7OMtBObltCAyHoyjwezBd/mg== X-Google-Smtp-Source: ADFU+vvMEQBOPd6Zfe5WzhxMWesz4iewNlgOu1pm6OgWwYpcAaCXRS0o8aM/RRSitF4y+VwXVIB6pw== X-Received: by 2002:a1c:7d88:: with SMTP id y130mr6104452wmc.5.1584034388539; Thu, 12 Mar 2020 10:33:08 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id l17sm14887319wmg.23.2020.03.12.10.33.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:33:08 -0700 (PDT) Date: Thu, 12 Mar 2020 17:33:07 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 4/6] mm, memcg: Prevent memory.min load/store tearing Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This can be set concurrently with reads, which may cause the wrong value to be propagated. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 4 ++-- mm/page_counter.c | 10 ++++++---- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c85a304fa4a1..e0ed790a2a8c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6261,7 +6261,7 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root, if (!usage) return MEMCG_PROT_NONE; - emin = memcg->memory.min; + emin = READ_ONCE(memcg->memory.min); elow = READ_ONCE(memcg->memory.low); parent = parent_mem_cgroup(memcg); @@ -6277,7 +6277,7 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root, if (emin && parent_emin) { unsigned long min_usage, siblings_min_usage; - min_usage = min(usage, memcg->memory.min); + min_usage = min(usage, READ_ONCE(memcg->memory.min)); siblings_min_usage = atomic_long_read( &parent->memory.children_min_usage); diff --git a/mm/page_counter.c b/mm/page_counter.c index 18b7f779f2e2..ae471c7d255f 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -17,14 +17,16 @@ static void propagate_protected_usage(struct page_counter *c, unsigned long usage) { unsigned long protected, old_protected; - unsigned long low; + unsigned long low, min; long delta; if (!c->parent) return; - if (c->min || atomic_long_read(&c->min_usage)) { - if (usage <= c->min) + min = READ_ONCE(c->min); + + if (min || atomic_long_read(&c->min_usage)) { + if (usage <= min) protected = usage; else protected = 0; @@ -217,7 +219,7 @@ void page_counter_set_min(struct page_counter *counter, unsigned long nr_pages) { struct page_counter *c; - counter->min = nr_pages; + WRITE_ONCE(counter->min, nr_pages); for (c = counter; c; c = c->parent) propagate_protected_usage(c, atomic_long_read(&c->usage)); From patchwork Thu Mar 12 17:33:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFFB01668 for ; Thu, 12 Mar 2020 17:33:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9D6D320716 for ; Thu, 12 Mar 2020 17:33:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="L2f+CaL9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9D6D320716 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C7326B0074; Thu, 12 Mar 2020 13:33:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 777E16B0075; Thu, 12 Mar 2020 13:33:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68E6C6B0078; Thu, 12 Mar 2020 13:33:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 51AF46B0074 for ; Thu, 12 Mar 2020 13:33:14 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id F3991181AEF2A for ; Thu, 12 Mar 2020 17:33:13 +0000 (UTC) X-FDA: 76587406266.09.teeth09_394b6485f9d55 X-Spam-Summary: 2,0,0,3d3a493f4d3f8f46,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1534:1541:1593:1594:1595:1596:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3871:3874:5007:6261:6653:10004:10400:11026:11232:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13069:13161:13229:13311:13357:13439:13846:13895:14181:14394:14721:21080:21444:21451:21627:21990:30054:30056:30064,0,RBL:209.85.221.67:@chrisdown.name:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:31,LUA_SUMMARY:none X-HE-Tag: teeth09_394b6485f9d55 X-Filterd-Recvd-Size: 4641 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:33:13 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id l18so8528724wru.11 for ; Thu, 12 Mar 2020 10:33:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=urzcg1/zbcSllHCQ82ap2K/ib0OV1jVM8zTOxSBSSFk=; b=L2f+CaL9yyAPG75AYGECf5KTcw7S8M4rdsMmRBSZMeGR8jRzPXKEG698E3haidM1SB EphvHD/U1lU3Tsfx19T+saKeeS1FVjHvlGAnaixK8Xiuvpf85YAi1FH8cmHHyStdVQ6x s3ChNGG7YtV2ejMML1Tln752ExwS8WXNN1Xzw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=urzcg1/zbcSllHCQ82ap2K/ib0OV1jVM8zTOxSBSSFk=; b=n7SRViz5EMxVJf5bl7XsUcec9HGKtf/XPKQue0pjXSlAOGGLcp8jWBE0Xbv/1JBmJ+ GPFinFLbtd0VQN1dvlaCcH/EIHOECKT2NsiQgiCSC2nakBcLtzBuWUKVEwA5++TChGiK kmL1bXi2qAr0yM7B9XglCcAVmsK0W8riLFvJypySOjDlCCV44M4YajDvakAi8qsI5i23 7L52Z/yLz5iulNkv2cSdUnXFAVpjMJM3RA2TAMCIT6hB8Iz2myxe4pyarsQooJ1qHnEB wr2W8qeCguVH0t3pZ3k0XCMtbJDpy9MUREGsbx2a9aLvE6O7DP1HTrtVtBbFZUVb3+vW Btag== X-Gm-Message-State: ANhLgQ0XsRleslJBhQYHkMzLgUZd7fh0BtHdYNe38wmMcBtoky288nYE jZ5oMyrECMiCoTXPnnFdS1QwIQ== X-Google-Smtp-Source: ADFU+vsFeyzbj5Xi2Wyqn57AJA4UE5JVSsw2uGMltfcBbAOKBHZZOyTjRytBGKPSYXhiCWlTxa3BBA== X-Received: by 2002:a5d:410a:: with SMTP id l10mr11343394wrp.380.1584034392630; Thu, 12 Mar 2020 10:33:12 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id s28sm106147wrb.42.2020.03.12.10.33.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:33:12 -0700 (PDT) Date: Thu, 12 Mar 2020 17:33:11 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 5/6] mm, memcg: Prevent memory.swap.max load tearing Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The write side of this is xchg()/smp_mb(), so that's all good. Just a few sites missing a READ_ONCE. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e0ed790a2a8c..57048a38c75d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1511,7 +1511,7 @@ void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) if (cgroup_subsys_on_dfl(memory_cgrp_subsys)) pr_info("swap: usage %llukB, limit %llukB, failcnt %lu\n", K((u64)page_counter_read(&memcg->swap)), - K((u64)memcg->swap.max), memcg->swap.failcnt); + K((u64)READ_ONCE(memcg->swap.max)), memcg->swap.failcnt); else { pr_info("memory+swap: usage %llukB, limit %llukB, failcnt %lu\n", K((u64)page_counter_read(&memcg->memsw)), @@ -1544,7 +1544,7 @@ unsigned long mem_cgroup_get_max(struct mem_cgroup *memcg) unsigned long swap_max; memsw_max = memcg->memsw.max; - swap_max = memcg->swap.max; + swap_max = READ_ONCE(memcg->swap.max); swap_max = min(swap_max, (unsigned long)total_swap_pages); max = min(max + swap_max, memsw_max); } @@ -7025,7 +7025,8 @@ bool mem_cgroup_swap_full(struct page *page) return false; for (; memcg != root_mem_cgroup; memcg = parent_mem_cgroup(memcg)) - if (page_counter_read(&memcg->swap) * 2 >= memcg->swap.max) + if (page_counter_read(&memcg->swap) * 2 >= + READ_ONCE(memcg->swap.max)) return true; return false; From patchwork Thu Mar 12 17:33:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435139 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5163E6CA for ; Thu, 12 Mar 2020 17:33:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1DE982073E for ; Thu, 12 Mar 2020 17:33:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="qaWsULht" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1DE982073E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D93BB6B0075; Thu, 12 Mar 2020 13:33:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D444F6B0078; Thu, 12 Mar 2020 13:33:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C80736B007B; Thu, 12 Mar 2020 13:33:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0006.hostedemail.com [216.40.44.6]) by kanga.kvack.org (Postfix) with ESMTP id B0C546B0075 for ; Thu, 12 Mar 2020 13:33:18 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7216E180ACC54 for ; Thu, 12 Mar 2020 17:33:18 +0000 (UTC) X-FDA: 76587406476.02.bait69_39f0e08541115 X-Spam-Summary: 2,0,0,4ef3e5ba0e233619,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1534:1541:1593:1594:1595:1596:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3870:3871:5007:6261:6653:10004:10400:11026:11473:11658:11914:12043:12114:12296:12297:12438:12517:12519:12555:12895:13069:13161:13229:13311:13357:13439:13846:13895:14096:14097:14181:14394:14721:21080:21444:21451:21627:21990:30054:30064,0,RBL:209.85.221.67:@chrisdown.name:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: bait69_39f0e08541115 X-Filterd-Recvd-Size: 3939 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:33:17 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id 6so8569902wre.4 for ; Thu, 12 Mar 2020 10:33:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=NoiaqPMaLbyoL7WcV9VE9eKkmobOKnQ4L0s3bngIbSs=; b=qaWsULht3bWoY5iXQLEcDGdBTFV6hjN7lpV33RTCtB3xWc1PLX/WRxZQohX8Xrl/9w UmohlrmtFgTmdaoEpFrPYYLSu55234z8uvo9TnjcyiBJbYmm2fJpmw10ZPHUC3T9G4Z3 CvKX3yFDACoE48iNQsDgNcSOYv8fqeVuxR4Rk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=NoiaqPMaLbyoL7WcV9VE9eKkmobOKnQ4L0s3bngIbSs=; b=FRSrdXgOUCNXbtUR35y3q5FSKj5D3kvHvyj9oObDFt4pipgDsYjHpQnBz/MzTJvg/I 6OQjoeG88Z3qU2lvDH7/yyBaEE24DwT1FXfveY+CXsLk0RFJUJA5X+MpaTcb40E9qXB/ NelVIGC+nCOs/XCmiLsVhncQ90n2ABKL+wbxTz5D+qAAdMRM0n27Jc/A+tgAlUbNcGGL iXKpVU7OPB+MJThF4dRwxzo98TiF7f0iOSmGiR9ATe0mGlE1fNZOW0rcwAev45dJK1ip shjGpRiO0RDR830YzKlO8MlZUZfC8i06skQ2sP6brq+RcLcOHlXSHNQ9z5qc7krKlVhI 49HQ== X-Gm-Message-State: ANhLgQ1f6570FTBnZmNSMXIbSU+9FmsUdEVOasgTotpSBtaWXEeFrvCd hiice0aSLcSeoR+v15Qv4L7Cjg== X-Google-Smtp-Source: ADFU+vvYbyeKzXBtFBZm5+Qqw3wXcJUdTLliut8wUu0quaYaNk9IXP08ecLQIM8cttCro0Cmn5Mn+Q== X-Received: by 2002:a5d:63c7:: with SMTP id c7mr11956491wrw.384.1584034396977; Thu, 12 Mar 2020 10:33:16 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id o3sm14396506wme.36.2020.03.12.10.33.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:33:16 -0700 (PDT) Date: Thu, 12 Mar 2020 17:33:16 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 6/6] mm, memcg: Prevent mem_cgroup_protected store tearing Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000073, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The read side of this is all protected, but we can still tear if multiple iterations of mem_cgroup_protected are going at the same time. There's some intentional racing in mem_cgroup_protected which is ok, but load/store tearing should be avoided. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 57048a38c75d..e9af606238ab 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6301,8 +6301,8 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root, } exit: - memcg->memory.emin = emin; - memcg->memory.elow = elow; + WRITE_ONCE(memcg->memory.emin, emin); + WRITE_ONCE(memcg->memory.elow, elow); if (usage <= emin) return MEMCG_PROT_MIN;