From patchwork Thu Mar 12 17:32:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Down X-Patchwork-Id: 11435129 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB35214E5 for ; Thu, 12 Mar 2020 17:32:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E586206B1 for ; Thu, 12 Mar 2020 17:32:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chrisdown.name header.i=@chrisdown.name header.b="LB2Rpnhy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E586206B1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chrisdown.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CA9CE6B006E; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5A2F6B0070; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B703C6B0071; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0108.hostedemail.com [216.40.44.108]) by kanga.kvack.org (Postfix) with ESMTP id 9F2FA6B006E for ; Thu, 12 Mar 2020 13:32:54 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 63DD2499606 for ; Thu, 12 Mar 2020 17:32:54 +0000 (UTC) X-FDA: 76587405468.22.dog63_366bc9d6cfa04 X-Spam-Summary: 2,0,0,69d90c6b393dadc6,d41d8cd98f00b204,chris@chrisdown.name,,RULES_HIT:41:355:379:800:960:966:973:988:989:1260:1277:1312:1313:1314:1345:1359:1431:1437:1516:1518:1519:1535:1542:1593:1594:1595:1596:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2740:2898:2904:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3872:4321:4385:5007:6261:6653:8531:10004:10400:10450:10455:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:13439:13846:13895:14096:14097:14181:14394:14721:19904:19999:21080:21444:21451:21627:21740:21990:30054:30064:30070,0,RBL:209.85.221.65:@chrisdown.name:.lbl8.mailshell.net-66.100.201.201 62.2.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: dog63_366bc9d6cfa04 X-Filterd-Recvd-Size: 5797 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Mar 2020 17:32:53 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id n15so8534086wrw.13 for ; Thu, 12 Mar 2020 10:32:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisdown.name; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=CcUuUlC43ZZSpfh7MoEkxpHRz/YQaDQ0sa4yKp/08VA=; b=LB2RpnhyFSTH3hVgjWSNFLifh6wVwkygWTfWLNDnxQPIIBKr4BpapuRzF1lJYa0M15 UbSB1s+QSe7P8Bx+GLZMQ+cV7gO4cfbRWGjeLOLvLCaijAjS6Lpr8ZOYSHWG8HOqN/hu kT1oLw9Sjvc/qjL4Kh04/aFvpCsvKUXlmqg/Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=CcUuUlC43ZZSpfh7MoEkxpHRz/YQaDQ0sa4yKp/08VA=; b=pD2RFz/84x+8xNyTqOO+ntoGjloTLmyi55G4iZQ7mvnEDjMpWWauHu2BSCuZeEXBsy 0aLSI4b4FA39jG6ogHAPHxLgTnGjIHZImaihlHLpsflUfBDSreccAgt7vIoe3gctANCD RX7icDws3D553PA4SWaPjvYglPHTFVJXtbcf+G0vYqCujk6fP59cy2YwXZ9mIKJ6f2PA HGS7pEKRCt8ZqeeLkSS5nyPIafbAk+b3OZfyeSpQ40n8/iKKcW+j2k6heEKlT6g1dTwQ 4wgO87AuQr7vR0VkKzLiosHom8J2jhf03ADNuuF0V8RbYIG5Oby3i2Q/MfS9c56QeWtj nVFw== X-Gm-Message-State: ANhLgQ3fBMSIi+l3+B/wWH2IoN1BqLKdVS6P/z7fgg5IWM8GR1Gx3VY3 74Wp3F5muJ7Tn+tObIABU2cQCg== X-Google-Smtp-Source: ADFU+vtgf9rQPYcJ7fNM8yiJe0orpohUFakZwlPq0AegakMNuqTr8+u5nhAj4dJIgtmv3wka95/7/g== X-Received: by 2002:a05:6000:1008:: with SMTP id a8mr6809577wrx.8.1584034372767; Thu, 12 Mar 2020 10:32:52 -0700 (PDT) Received: from localhost ([89.32.122.5]) by smtp.gmail.com with ESMTPSA id z6sm20259371wru.15.2020.03.12.10.32.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Mar 2020 10:32:52 -0700 (PDT) Date: Thu, 12 Mar 2020 17:32:51 +0000 From: Chris Down To: Andrew Morton Cc: Johannes Weiner , Tejun Heo , Roman Gushchin , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/6] mm, memcg: Prevent memory.high load/store tearing Message-ID: <2f66f7038ed1d4688e59de72b627ae0ea52efa83.1584034301.git.chris@chrisdown.name> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A mem_cgroup's high attribute can be concurrently set at the same time as we are trying to read it -- for example, if we are in memory_high_write at the same time as we are trying to do high reclaim. Signed-off-by: Chris Down Cc: Andrew Morton Cc: Johannes Weiner Cc: Roman Gushchin Cc: Tejun Heo Cc: linux-mm@kvack.org Cc: cgroups@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: kernel-team@fb.com Acked-by: Michal Hocko --- mm/memcontrol.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 63bb6a2aab81..d32d3c0a16d4 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2228,7 +2228,7 @@ static void reclaim_high(struct mem_cgroup *memcg, gfp_t gfp_mask) { do { - if (page_counter_read(&memcg->memory) <= memcg->high) + if (page_counter_read(&memcg->memory) <= READ_ONCE(memcg->high)) continue; memcg_memory_event(memcg, MEMCG_HIGH); try_to_free_mem_cgroup_pages(memcg, nr_pages, gfp_mask, true); @@ -2545,7 +2545,7 @@ static int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, * reclaim, the cost of mismatch is negligible. */ do { - if (page_counter_read(&memcg->memory) > memcg->high) { + if (page_counter_read(&memcg->memory) > READ_ONCE(memcg->high)) { /* Don't bother a random interrupted task */ if (in_interrupt()) { schedule_work(&memcg->high_work); @@ -4257,7 +4257,8 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages, *pheadroom = PAGE_COUNTER_MAX; while ((parent = parent_mem_cgroup(memcg))) { - unsigned long ceiling = min(memcg->memory.max, memcg->high); + unsigned long ceiling = min(memcg->memory.max, + READ_ONCE(memcg->high)); unsigned long used = page_counter_read(&memcg->memory); *pheadroom = min(*pheadroom, ceiling - min(ceiling, used)); @@ -4978,7 +4979,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) if (!memcg) return ERR_PTR(error); - memcg->high = PAGE_COUNTER_MAX; + WRITE_ONCE(memcg->high, PAGE_COUNTER_MAX); memcg->soft_limit = PAGE_COUNTER_MAX; if (parent) { memcg->swappiness = mem_cgroup_swappiness(parent); @@ -5131,7 +5132,7 @@ static void mem_cgroup_css_reset(struct cgroup_subsys_state *css) page_counter_set_max(&memcg->tcpmem, PAGE_COUNTER_MAX); page_counter_set_min(&memcg->memory, 0); page_counter_set_low(&memcg->memory, 0); - memcg->high = PAGE_COUNTER_MAX; + WRITE_ONCE(memcg->high, PAGE_COUNTER_MAX); memcg->soft_limit = PAGE_COUNTER_MAX; memcg_wb_domain_size_changed(memcg); } @@ -5947,7 +5948,7 @@ static ssize_t memory_high_write(struct kernfs_open_file *of, if (err) return err; - memcg->high = high; + WRITE_ONCE(memcg->high, high); for (;;) { unsigned long nr_pages = page_counter_read(&memcg->memory);