From patchwork Wed Jan 9 12:20:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10753981 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A2C8746 for ; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 892CB28E8E for ; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 878CE28EBB; Wed, 9 Jan 2019 12:20:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D950E28EBA for ; Wed, 9 Jan 2019 12:20:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DBDB8E009F; Wed, 9 Jan 2019 07:20:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6CB108E0038; Wed, 9 Jan 2019 07:20:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 542F08E009F; Wed, 9 Jan 2019 07:20:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) by kanga.kvack.org (Postfix) with ESMTP id D9A838E0038 for ; Wed, 9 Jan 2019 07:20:38 -0500 (EST) Received: by mail-lj1-f197.google.com with SMTP id v24-v6so1779971ljj.10 for ; Wed, 09 Jan 2019 04:20:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=5tY899Xd2X+9Z3CRn+G6UgzyTSWMoq5hPLIK/1l/VBc=; b=mcEfSBwvNxvWgKvH0jBgezn/t0bE4bDdvPbQPA2F2EGC0zd8iBEnKKEfWd7Y9udpKe dMsGKbgcGglFuX8782V3EaJh2gpdsIf9Sh56dsps7Ipzseq9fr/OvLuueqa8H/wXQKgc YLdq5fhKvHQOaC6osELaJApN4dDExHz8HvN7ntAA+GgpZX6Jy5WFYxOgpzyQXpvO3ZLc Lb+m7t3oW6EzwZq1Atv85aC3YY1XnSnf8xR61Ua4IgT0j5Tvqbb6TNmfXwFTRvtcu8LG 89IJN8BxOZnyOk6skf9XpQI+7FAC2V9zonHXoFLk2gA8YQZ6pi10L4CCwfoE6lFnRm2A 9EVQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: AJcUukfgBmRdXTd+739CFY95SrTeUA+9BBRlAxsCjaSM4b/+BhKtE01A FrF8fHFm0ZzzD60gscWe27MKd0tZxZ0pxC6Xi3yq7MW+g2g4wKFy4FVnCrKZLuJUeyEj8qrRaG7 4w+EuDZxznAPLUsHXchRiQGtzk5Zft/xFU1CdqY2qKM0dAd0eA1lgUoGqaTWXlGv0xg== X-Received: by 2002:a2e:197:: with SMTP id f23-v6mr3309296lji.144.1547036438162; Wed, 09 Jan 2019 04:20:38 -0800 (PST) X-Google-Smtp-Source: ALg8bN4EKDkZuRzn+8Iz/majJKX712vWZq4aPRPBKasfRKTUusr8eQI3nrvMX6LRzVzNXH6GYQAJ X-Received: by 2002:a2e:197:: with SMTP id f23-v6mr3309247lji.144.1547036436943; Wed, 09 Jan 2019 04:20:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547036436; cv=none; d=google.com; s=arc-20160816; b=mr9thyfy2QNfe3CbiJtl7Q9pJtF9Kx7WUhgCxesxSunxXT/pyfIb7H1Rdoysq7qujt Bmh1zzOjyYFl5ibL2cgPwoM0R/r1xtqFO1m7oWbN3LgPeKT7/jT7GTSZGYGsAxwdTMUU ITj0+wr5eY262rBl/Tm09r1mDumY9tDvBjSGT9kWvQdHA8zsB5pUtO2LU55rsEJpE4gw 7HYRt5T73kTMO5qk1z3aeK4eKwRBalecFb9wiu8mgZgYu2W/aIYNp7rbrYNWGMVExjBD ZEwUc5YdL8zI1SjFF+eGrQzWZP6hiUqEePXNhGHyqMbuWnfpjWKgHRZT/TRBXzCL80BJ +b/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=5tY899Xd2X+9Z3CRn+G6UgzyTSWMoq5hPLIK/1l/VBc=; b=S+gUYJbyVZg4JzTkBRYYXGGu4E9L0Kj16kGPzPAgwBvGQW9mxnGN95AaoO51w52PrM D5zWiJWTng0WROmGYCRwTAB5Iv3FJe/8YXJayy8GzwCDFDdG853VR6e1Fc/NU3nB7W17 2JkoksrBuAPgiflk9yXSY+tGM4iQl2ECPBrw+xRHXyeCvKPNt4i7LtGKpXctXBY6cpJe d6KNh5LCtfqDsxcD6hirK9JSj/3QOtt93Di8+Vwf17997+B+WX8Pl86qjfdf+nillvhS iaWEjVMvs7l0XtRFO+pMYOpSNqNvMlRzzKnMcwV0WMqe5VSP1XLNqn1Id/RpxVWpPh7v IzFg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id v23-v6si63981269ljh.63.2019.01.09.04.20.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Jan 2019 04:20:36 -0800 (PST) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1ghCqO-0004gt-O2; Wed, 09 Jan 2019 15:20:24 +0300 Subject: [PATCH 1/3] mm: Uncharge and keep page in pagecache on memcg reclaim From: Kirill Tkhai To: akpm@linux-foundation.org, hannes@cmpxchg.org, josef@toxicpanda.com, jack@suse.cz, hughd@google.com, ktkhai@virtuozzo.com, darrick.wong@oracle.com, mhocko@suse.com, aryabinin@virtuozzo.com, guro@fb.com, mgorman@techsingularity.net, shakeelb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Wed, 09 Jan 2019 15:20:24 +0300 Message-ID: <154703642447.32690.5604527676583713589.stgit@localhost.localdomain> In-Reply-To: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> References: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch makes __remove_mapping() not remove a page from pagecache on memcg reclaim. After all mappings are removed and refcounter is freezed, we uncharge page memcg. Further putback_lru_page() places page into root_mem_cgroup, so it remains in pagecache till global reclaim. This gives memcg tasks extra possibility to obtain page from pagecache instead of launching IO. Next patch makes pagecache_get_page() to recharge a page in case of its memcg is NULL (i.e., on first access after uncharging). It looks to be the only function, which is used by filesystems to obtain a pagecache page. Here we introduce AS_KEEP_MEMCG_RECLAIM flag to mark the filesystems, which are reviewed, that they really follow this way. It has a sense to keep pages in __remove_mapping() only for them. Later, we remove this flags after all filesystems are reviewed. Signed-off-by: Kirill Tkhai --- include/linux/pagemap.h | 1 + mm/vmscan.c | 22 ++++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 1020e6f40880..1b880da85868 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -29,6 +29,7 @@ enum mapping_flags { AS_EXITING = 4, /* final truncate in progress */ /* writeback related tags are not used */ AS_NO_WRITEBACK_TAGS = 5, + AS_KEEP_MEMCG_RECLAIM = 6, }; /** diff --git a/mm/vmscan.c b/mm/vmscan.c index a714c4f800e9..7237603c8973 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -887,7 +887,7 @@ static pageout_t pageout(struct page *page, struct address_space *mapping, * gets returned with a refcount of 0. */ static int __remove_mapping(struct address_space *mapping, struct page *page, - bool reclaimed) + bool reclaimed, bool memcg_reclaim) { unsigned long flags; int refcount; @@ -963,7 +963,20 @@ static int __remove_mapping(struct address_space *mapping, struct page *page, if (reclaimed && page_is_file_cache(page) && !mapping_exiting(mapping) && !dax_mapping(mapping)) shadow = workingset_eviction(mapping, page); - __delete_from_page_cache(page, shadow); +#ifdef CONFIG_MEMCG + if (memcg_reclaim && + test_bit(AS_KEEP_MEMCG_RECLAIM, &mapping->flags)) { + /* + * Page is not dirty/writeback/mapped, so we may avoid + * taking mem_cgroup::move_lock for changing its memcg. + * See mem_cgroup_move_account() for details. + */ + mem_cgroup_uncharge(page); + page_ref_unfreeze(page, refcount); + goto cannot_free; + } else +#endif + __delete_from_page_cache(page, shadow); xa_unlock_irqrestore(&mapping->i_pages, flags); if (freepage != NULL) @@ -985,7 +998,7 @@ static int __remove_mapping(struct address_space *mapping, struct page *page, */ int remove_mapping(struct address_space *mapping, struct page *page) { - if (__remove_mapping(mapping, page, false)) { + if (__remove_mapping(mapping, page, false, false)) { /* * Unfreezing the refcount with 1 rather than 2 effectively * drops the pagecache ref for us without requiring another @@ -1458,7 +1471,8 @@ static unsigned long shrink_page_list(struct list_head *page_list, count_vm_event(PGLAZYFREED); count_memcg_page_event(page, PGLAZYFREED); - } else if (!mapping || !__remove_mapping(mapping, page, true)) + } else if (!mapping || !__remove_mapping(mapping, page, true, + !global_reclaim(sc))) goto keep_locked; unlock_page(page); From patchwork Wed Jan 9 12:20:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10753979 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 797B01850 for ; Wed, 9 Jan 2019 12:20:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68F4C28E97 for ; Wed, 9 Jan 2019 12:20:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 66F1A28EF7; Wed, 9 Jan 2019 12:20:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BC7C28EBB for ; Wed, 9 Jan 2019 12:20:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B4148E009E; Wed, 9 Jan 2019 07:20:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 73C3D8E0038; Wed, 9 Jan 2019 07:20:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62B2F8E009E; Wed, 9 Jan 2019 07:20:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) by kanga.kvack.org (Postfix) with ESMTP id EB4AE8E0038 for ; Wed, 9 Jan 2019 07:20:37 -0500 (EST) Received: by mail-lf1-f72.google.com with SMTP id m19so561225lfj.17 for ; Wed, 09 Jan 2019 04:20:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=DlQlWlVWuj5IakiQCuo6n4LBXun72uWkrZSTrhrcs3U=; b=jXBcAlr7vdtpnl9JB+oSn5lD2QSowl83EeHRY2R2x6LAV5JVDXKgd8wBIydPd1T+vS QE0SP4woFIaxNtto4SxazzBJOnomjQKzndG5Pb0qgD5kFZpZrSnEBHnEgh5c3hIC5j8T H0VzXD1ejmr9pfC5gDBj18EcPbiaGlhUvTuwgJKxqdXHOxziTuGKdW1ONTuLx85uJZet oW+ZzqobYy+u8lkI0M9/1i1IngHdc2xvBU5oDaAYDAMzyZ/8DxuljNqenrWTgCv6WV7l PW8QLYkcL+AZMLeXuNTvhbkSG/2tI0tG5U272u8/b+Hxla0GyGWvH10mWf+MfU0MMzQO NX3w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: AJcUukePMazWSu/STDco2GN232t0PlKEOJ8uiUSVpRIVIFRtuWWm7P2p LxYww8EiaUIKwsQgCRMSex3kcdcJOFTlNaIFidzW2ikgFHKdi0b6NssQPgZWyDJRaQWa2VgykI9 2qqWAWSnVAvg8bJCEZ/+hUe5ZxbTBg4XMoA1UN0L7Cl6EYNi4jeGWkymbgRTN2A8WgA== X-Received: by 2002:a2e:a289:: with SMTP id k9-v6mr3267522lja.24.1547036437214; Wed, 09 Jan 2019 04:20:37 -0800 (PST) X-Google-Smtp-Source: ALg8bN5wg76lhjKNqeRCDIxd6SfdwcHR5dUq1oQvHl+djSx7CbX/whpB0HvPQ+710rjUVPyfVhMY X-Received: by 2002:a2e:a289:: with SMTP id k9-v6mr3267477lja.24.1547036436124; Wed, 09 Jan 2019 04:20:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547036436; cv=none; d=google.com; s=arc-20160816; b=o1+4ERFIuauT9EqAcuEM2A9qggLxZZhKZh34zEwc6ZOcyb/zhH4C5ZAgoTjJ9TOsSO jmhte5RvXCT9uJu+HbAI2B8oh/AszrFzyhRERfRFh1D2GrX4FlDkWINCv1S303pXYUsO Uwelc6fgse0A2rpPnSpsTru/1C6FKQBnxIzUbqLYES4DKDuYAn9qLaFwhd7YajHQa4zP gUHW+yvxNshfj/aaJWosHOtb4eDwGIi5F83YFN08jHF7KT6Z65fDEgOeUI0oHbzEjFwW NsQjjtbEtPgiebOtRQwyMiLCgzV5SrnBrm3curZKRQkySzeGcEUmG1OD2/U2YmTqiEJ/ ehIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=DlQlWlVWuj5IakiQCuo6n4LBXun72uWkrZSTrhrcs3U=; b=b83lOkRH69EKkXdjla0vOoiobc1HCa8mrsiYnQtdCVuvWGX+5jTLdK96Vnvdz2BO4E Ycts31C2ksgQ6TfuEez3sSVxCoV8v1pGPDSG+BNC3C0ZZ1WpT0JcwY9KU+PUqlCCWgxC eRsCOp1rixPeZaX/q8GZ0/U4LWH6DT1g6nRVh4+IZFqffYStqAHY6XWJaIn+Vrpu4tEj hAPvsugekR1qxdlXIEldlrJCvA5Tnzz4UAKMP7Y/anhtM75Ld4Q7WtOQHUEswRyRJN5d aozDgpkfOTNj9PEhfvh7FwuROpIESnNpdHzcTK4bZ2x6lsJ1KpywYvYVQqVA6ZoInCaM UDhQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id p11-v6si61579253ljc.73.2019.01.09.04.20.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Jan 2019 04:20:36 -0800 (PST) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1ghCqU-0004h4-FM; Wed, 09 Jan 2019 15:20:30 +0300 Subject: [PATCH 2/3] mm: Recharge page memcg on first get from pagecache From: Kirill Tkhai To: akpm@linux-foundation.org, hannes@cmpxchg.org, josef@toxicpanda.com, jack@suse.cz, hughd@google.com, ktkhai@virtuozzo.com, darrick.wong@oracle.com, mhocko@suse.com, aryabinin@virtuozzo.com, guro@fb.com, mgorman@techsingularity.net, shakeelb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Wed, 09 Jan 2019 15:20:30 +0300 Message-ID: <154703642996.32690.7131436841852687919.stgit@localhost.localdomain> In-Reply-To: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> References: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch makes pagecache_get_page() to charge uncharged page into memcg of process, which accesses the page first. Page will be returned in case of it's charged only, so memcg tasks can't use pages, which was left by __remove_mapping(), without accounting them. In case of accounting is not possible, pages remain in pagecache, and further global reclaim will remove them (and this will be easily, since pages are not mapped by any task). Also, note that uncharged page can't be dirty or under writeback, since it was able to be isolated in __remove_mapping() earlier. Signed-off-by: Kirill Tkhai --- mm/filemap.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index 65c85c47bdb1..2603c44fc74a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1576,15 +1576,18 @@ EXPORT_SYMBOL(find_lock_entry); struct page *pagecache_get_page(struct address_space *mapping, pgoff_t offset, int fgp_flags, gfp_t gfp_mask) { + struct mem_cgroup *memcg; struct page *page; + bool drop_lock; repeat: + drop_lock = false; page = find_get_entry(mapping, offset); if (xa_is_value(page)) page = NULL; if (!page) goto no_page; - +lock: if (fgp_flags & FGP_LOCK) { if (fgp_flags & FGP_NOWAIT) { if (!trylock_page(page)) { @@ -1604,6 +1607,31 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t offset, VM_BUG_ON_PAGE(page->index != offset, page); } + if (!mem_cgroup_disabled() && !PageHuge(page) && + !page_memcg(page) && !page_mapped(page) && + test_bit(AS_KEEP_MEMCG_RECLAIM, &mapping->flags)) { + if (!(fgp_flags & FGP_LOCK)) { + drop_lock = true; + fgp_flags |= FGP_LOCK; + goto lock; + } + + if (!WARN_ON(PageDirty(page) || PageWriteback(page))) { + if (mem_cgroup_try_charge(page, current->mm, + gfp_mask, &memcg, false)) { + unlock_page(page); + put_page(page); + return NULL; + } + mem_cgroup_commit_charge(page, memcg, true, false); + if (!isolate_lru_page(page)) + putback_lru_page(page); + } + } + + if (drop_lock) + unlock_page(page); + if (fgp_flags & FGP_ACCESSED) mark_page_accessed(page); From patchwork Wed Jan 9 12:20:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10753987 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88FBD91E for ; Wed, 9 Jan 2019 12:20:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7802928ECC for ; Wed, 9 Jan 2019 12:20:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 75F1C28ED2; Wed, 9 Jan 2019 12:20:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB32E28ECC for ; Wed, 9 Jan 2019 12:20:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6F8C8E00A2; Wed, 9 Jan 2019 07:20:46 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C1DCC8E0038; Wed, 9 Jan 2019 07:20:46 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B34C58E00A2; Wed, 9 Jan 2019 07:20:46 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) by kanga.kvack.org (Postfix) with ESMTP id 4777E8E0038 for ; Wed, 9 Jan 2019 07:20:46 -0500 (EST) Received: by mail-lj1-f199.google.com with SMTP id f22-v6so1752695lja.7 for ; Wed, 09 Jan 2019 04:20:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=9S2UL6YPjRPiY8+WNR6DvKZzO3P0+ivl0h5qHhHkukY=; b=QziM+8bTGwOblifny1JIGjKwaqQogMFgCvIrW/eQHmhK2mxFghl+vXm2OKZv160uuk V7ngqjfLt7IdrQFneJGrDAOYXO+4NHWes/QxGUpthDxqbp0tVRptKvEq1Jb100U1eVD+ TmecwCXjsO3GfgfyJXrMaJNvgmSsQIlgb5cO6qir3/73ncrFoiUT915F7tBIK3hRqD0b VexQgR36LQriMe6NLVbBKoeTyI0r453Gm1AwhLyOhMArUTj7SVRDHoa8qXt2b3qir4xN iRRMAudZ4RjjnP7P1tncffl/BbQ0QeUcIufvzuynr9b4tUR5RdO5EAAwUn7tSkUnEBsP 06Qg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: AJcUukedcCgqrA95w7trm3EQEiuFt17s9O2dNH0NslOpPFOkS/BloyP0 0nUnqPmPL4EnEz3v/LNPk+ZfMn+Dmetsnf0GJqoEpyckruGUY1shisCSEuqgM7BSRH1XDM8pa3O d4Kz4E5btlS/kHvlSqsxY2VC6Gr45NQ/tMqmahBPqhGAbgaDpw8E6ITZc7Lx6tnJxzA== X-Received: by 2002:a2e:84ca:: with SMTP id q10-v6mr3244700ljh.65.1547036445712; Wed, 09 Jan 2019 04:20:45 -0800 (PST) X-Google-Smtp-Source: ALg8bN6JY0YvKOJ50TGT5n8hF3+S1y/0M70iQRe48wmnvZfUJH4VvOxzM1ZfLbvCzRrCP1aPNK4/ X-Received: by 2002:a2e:84ca:: with SMTP id q10-v6mr3244663ljh.65.1547036444641; Wed, 09 Jan 2019 04:20:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547036444; cv=none; d=google.com; s=arc-20160816; b=d3UIdjHtGOmwUtUnQEp8hik5ZfA13oxwGZE5IUM7sOutgkLoc3rxngUByJ0Ea83v5k OBqtulLdYNhHWI3aAV4dwgatF4AluLIqByNJkWUm85cbQEhhV7yaXASGDPBLpeUNSOoX JQAb12e0Sbgm6LYYpjFEMlx06njfjbxPymDtiLtmGoDlrHBQzRI2t+d4I4wXOgQsg8ON ffWGzJfkF4/vU8F5FdgRm1JEO6hX+0+LU5OixBq4wyGgPt0ul4F53GHxAy7g9HSoTi0y 88ICAb1J/l0LZIhZUmjeRP1g4vELice8cCm0JbQGo6llj6lVXKwcPyIR28SSIa1KCSRL fQlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=9S2UL6YPjRPiY8+WNR6DvKZzO3P0+ivl0h5qHhHkukY=; b=kZMWSUI9v5VvRMDxBTyohu/rrjwEaBcVzyr1ebxbYX3GXkn0RYgVSFSMJbpSzN2RWC tEPNY4ZZzoiiswMp/8Q3qKh3ZRXtuaEZa4x29t3jKfUNFREhFmvs63pqox1OJ0Gw8xBX aW/69Ii0wU21b2+CkubNFiCzVxBmn/XNxJqhwgJB7U5cK+v/O0WMPRPb7LD+LIkSuYvF h94+zuYLHSTC0FfyXZonS9YIVnea/kOy5D0rG9k+Rq54Rd+IoBClXSHmHfAfTpVWaEyV QvWvSu1KriiQxWaqdClF0Hng9rgXmOzcQU3s7MtDC9lzZiNeZJyTPRwTV32Y9saNtbSx d9TQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id u191-v6si73472439lja.171.2019.01.09.04.20.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 Jan 2019 04:20:44 -0800 (PST) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1ghCqZ-0004hI-Tg; Wed, 09 Jan 2019 15:20:35 +0300 Subject: [PATCH 3/3] mm: Pass FGP_NOWAIT in generic_file_buffered_read and enable ext4 From: Kirill Tkhai To: akpm@linux-foundation.org, hannes@cmpxchg.org, josef@toxicpanda.com, jack@suse.cz, hughd@google.com, ktkhai@virtuozzo.com, darrick.wong@oracle.com, mhocko@suse.com, aryabinin@virtuozzo.com, guro@fb.com, mgorman@techsingularity.net, shakeelb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Wed, 09 Jan 2019 15:20:35 +0300 Message-ID: <154703643564.32690.8416317230641240199.stgit@localhost.localdomain> In-Reply-To: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> References: <154703479840.32690.6504699919905946726.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP All page-obtaining functions, which are used by ext4, look to go thru pagecache_get_page() path, so all taken uncharged pages will be properly charged. Thus, we enable AS_KEEP_MEMCG_RECLAIM for ext4 regular files. Since memcg accounting requires page lock, and function generic_file_buffered_read() is the only of ext4-used functions, which does not care about FGP_NOWAIT, we make it use find_get_page_flags() and pass the flag. This allows pagecache_get_page() to use lock_page(), when it's possible. Signed-off-by: Kirill Tkhai --- fs/ext4/inode.c | 1 + mm/filemap.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index b1d7ddd70eee..2fc9e4a7c0db 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -5065,6 +5065,7 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino, inode->i_op = &ext4_file_inode_operations; inode->i_fop = &ext4_file_operations; ext4_set_aops(inode); + set_bit(AS_KEEP_MEMCG_RECLAIM, &inode->i_mapping->flags); } else if (S_ISDIR(inode->i_mode)) { inode->i_op = &ext4_dir_inode_operations; inode->i_fop = &ext4_dir_operations; diff --git a/mm/filemap.c b/mm/filemap.c index 2603c44fc74a..46922003811f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2061,6 +2061,11 @@ static void shrink_readahead_size_eio(struct file *filp, ra->ra_pages /= 4; } +static int kiocb_fgp_flags(struct kiocb *iocb) +{ + return (iocb->ki_flags & IOCB_NOWAIT) ? FGP_NOWAIT : 0; +} + /** * generic_file_buffered_read - generic file read routine * @iocb: the iocb to read @@ -2111,7 +2116,8 @@ static ssize_t generic_file_buffered_read(struct kiocb *iocb, goto out; } - page = find_get_page(mapping, index); + page = find_get_page_flags(mapping, index, + kiocb_fgp_flags(iocb)); if (!page) { if (iocb->ki_flags & IOCB_NOWAIT) goto would_block;