From patchwork Wed Aug 8 13:20:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10560025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B2F1139A for ; Wed, 8 Aug 2018 13:21:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14F782AC31 for ; Wed, 8 Aug 2018 13:21:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 120E82AC66; Wed, 8 Aug 2018 13:21:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18C1B2AC31 for ; Wed, 8 Aug 2018 13:21:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 273966B0007; Wed, 8 Aug 2018 09:21:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2206C6B0008; Wed, 8 Aug 2018 09:21:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09D7F6B000A; Wed, 8 Aug 2018 09:21:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f199.google.com (mail-qk0-f199.google.com [209.85.220.199]) by kanga.kvack.org (Postfix) with ESMTP id C77F26B0007 for ; Wed, 8 Aug 2018 09:21:08 -0400 (EDT) Received: by mail-qk0-f199.google.com with SMTP id e3-v6so2157940qkj.17 for ; Wed, 08 Aug 2018 06:21:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:subject:from:to:references :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding:spamdiagnosticoutput :spamdiagnosticmetadata; bh=PW0WOLFthiKNMQomc+zu4rJAI+6PA5dqVrsYRoLFDfY=; b=Z/FEI5wHk6MWcn3U4eD2fc4m3oP93wyRVR8bdIDN2w5BCs2AN30YXqMTKQcZQWpoyV P6xh5J5ZjsOGviT/4X/ZFE9CGmHcGBYNEOU/56uQfR6eS/NUwE+Sux3SbRjNdJui89MJ ei/PaZFk77gCJNKApFCN5VdHt0wGCNxYq0Xpzl27p6mD9QEHSY5VwWRxSsucn2c4IzVf RWSyHUmq6b39HAylXFU4bUdLwdVqxlvKD/EhBRoeCPw5s5EuRzmqBzxDdEXxf8nRpcU0 /QElIqWXoT2xv2TffyAPWLpOnnJE59fwoRSYZqtOhd/xEshi4Adv6wdBUkdaNDqc6d6l DLww== X-Gm-Message-State: AOUpUlH5VBdXPHdDqci8g9dzZBHRkElQWrlaFKozPbZ6r2nTFduV0d+Z 00PKZOa5RbDcIhEZl+Gxask8wyvmFcVZQxQ5qUsp7n3XCx8NNvRuTX3QdGosd7tsw2IMNtoZPsL rG/yxLL5HdkJpv+rzh/EQNGJmc3WdrflAdRFRI6iaFp6v8kOGT6uXzw9JOstzL+Wx3w== X-Received: by 2002:a37:21e6:: with SMTP id f99-v6mr2274538qki.206.1533734468540; Wed, 08 Aug 2018 06:21:08 -0700 (PDT) X-Google-Smtp-Source: AA+uWPy4njsH3rYW9u64g1TU9xEdx5XdaLa0JEZrZ0hcP/VIcyjCHnFr3B8DFtPis+CwcIea8Dqj X-Received: by 2002:a37:21e6:: with SMTP id f99-v6mr2274499qki.206.1533734467700; Wed, 08 Aug 2018 06:21:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533734467; cv=none; d=google.com; s=arc-20160816; b=jtZQtFtGy+4aMeYXVoMBT81HwJiCCuzLUZHxd4nOzhMUyBo4YCmk8JZ0814rqg/UgK mrDy1rle85rWFdlhOI5yJJIQ1Sb2N/ddSX51DQvFaOcXijU2RJi2GhTtmXPo2kDFi0Hn JTh+qjcdsfWei7xlUWRXL7IIq1dxO2s5W3ZjGELk+YqJhKCbbsfEGc56SzpBKTLAg2YF noVBQlBC0YLzNLB7FRU+sFzMlFf0OfLiJ+dpj1LwnXC3Pw/nPgsWAjhiD7I3u6jcF8Bx rBwlk0CnXRZQIwHT0q7gplh/OYrZwpgfTtVnEKC8/cBrNOxjxSeqeeklu2AE2MyARaVv iXFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput :content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:to:from:subject :dkim-signature:arc-authentication-results; bh=PW0WOLFthiKNMQomc+zu4rJAI+6PA5dqVrsYRoLFDfY=; b=p89f18Kvzjn1RIov0gGn6cpGiyu49M/D6HKhaEaSjmlxfMPJKFwQLBZqsBTxJHQxQJ l9azD2W63xwIFXmnmpO93Dx/W+otlv9Nnvx3wahkj01gRoW+8C4HfvRQMJFhWDRLbvVS DHVmqSpCdFRpObsjjAheFmns/ydEGDaZSLeldiuBlPwcY11g7wtzBe/B5T+7O302xZVo PTRAePrkKlehTy+oB92EAmeo8rUnL0sRJiH1Zzsr5q0CVzJ0X7msdeFKC+MGnTOuZOAl R01OgvUgZ3u/FBQt0ZtUemBydpMsm+uYAM3aTgBUgMsdDpDZIVf2KfyAoaXyNrn3U2ep SnQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector1 header.b=SUmTduri; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 40.107.1.90 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10090.outbound.protection.outlook.com. [40.107.1.90]) by mx.google.com with ESMTPS id n3-v6si3108682qvi.279.2018.08.08.06.21.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 08 Aug 2018 06:21:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 40.107.1.90 as permitted sender) client-ip=40.107.1.90; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=selector1 header.b=SUmTduri; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 40.107.1.90 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PW0WOLFthiKNMQomc+zu4rJAI+6PA5dqVrsYRoLFDfY=; b=SUmTduriy/ZWlX/dg0vUry7PJQpMQ02lRshhY2Ef9fZ1bna1cpgMAGgyRMh5TgNy2LigaxGeM889FPjj7w3BU2JfRoZMxL/I+yjhqZ1Ezz9wZSlcLZOGW2gnAYTC8Hs92Wv+ynlu1NV+3rQxBsdy184aEab5M3yID9xh73DZ0bQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ktkhai@virtuozzo.com; Received: from [172.16.25.169] (185.231.240.5) by HE1PR0801MB2026.eurprd08.prod.outlook.com (2603:10a6:3:50::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1017.15; Wed, 8 Aug 2018 13:20:59 +0000 Subject: [PATCH RFC v2 02/10] mm: Make shrink_slab() lockless From: Kirill Tkhai To: akpm@linux-foundation.org, gregkh@linuxfoundation.org, rafael@kernel.org, viro@zeniv.linux.org.uk, darrick.wong@oracle.com, paulmck@linux.vnet.ibm.com, josh@joshtriplett.org, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com, hughd@google.com, shuah@kernel.org, robh@kernel.org, ulf.hansson@linaro.org, aspriel@gmail.com, vivek.gautam@codeaurora.org, robin.murphy@arm.com, joe@perches.com, heikki.krogerus@linux.intel.com, sfr@canb.auug.org.au, vdavydov.dev@gmail.com, mhocko@suse.com, chris@chris-wilson.co.uk, penguin-kernel@I-love.SAKURA.ne.jp, aryabinin@virtuozzo.com, willy@infradead.org, ying.huang@intel.com, shakeelb@google.com, jbacik@fb.com, mingo@kernel.org, mhiramat@kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org References: <153365347929.19074.12509495712735843805.stgit@localhost.localdomain> <153365626605.19074.16202958374930777592.stgit@localhost.localdomain> <591d2063-0511-103d-bef6-dd35f55afe32@virtuozzo.com> Message-ID: <4ceb948c-7ce7-0db3-17d8-82ef1e6e47cc@virtuozzo.com> Date: Wed, 8 Aug 2018 16:20:54 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <591d2063-0511-103d-bef6-dd35f55afe32@virtuozzo.com> Content-Language: en-US X-Originating-IP: [185.231.240.5] X-ClientProxiedBy: DB6P193CA0016.EURP193.PROD.OUTLOOK.COM (2603:10a6:6:29::26) To HE1PR0801MB2026.eurprd08.prod.outlook.com (2603:10a6:3:50::15) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 84edffd3-7187-4dd2-4d2b-08d5fd31ca6e X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:HE1PR0801MB2026; X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB2026;3:w16ffdMt0MlHNui3JiMf/6e0xp5d68XA71Fa9BbmfF+k1BonhGpO4mryymUyJxVP3tag8Nry3oCbfN3VlYx2lwDrVJyyt4NoZawLpRqQXbrfpCzvrDi5hDqPUjwwFZ2ftht1eQK8f39wBNNkxP1obG4PfsS+TUSM8X9vbWlllZsnOEBL8LvsK6J4y+JCYlZ5kTvFy4cXEn3iuyYoV5E+rQU/q56J8YS5cdF1UNKWqnLIYosGG/68sR+W71pzrXDL;25:oM28E5BuuJD7q0KmIZcC0/keKpd1Ub1ylPH8jvC+RwoUvOFXU/FGWfDBGUCwWcX7Tea+nE5EdRMoTLhwj3m2ueNTZtFNQUXhH/ebRIrOnMYS8S0WaqT97DRkKDK6Kp1i5xiPlJztrgcYlVpwLNW2HWAEMQIb18hjJyuSuGaz/rmCpNDWkf3AOIw8wFbHU3Fa6rBFbO3q7rdkPvqSY33c3QfFi3sOqJlhKk2HRUD6hF3OXLLSMJ2NhRtLwXU0RZYLLCrWzz8CS/DaE5AkLTAvIxIcL8bPfglxItMjYU9tcS85mp0JpoF8O6SUs3iABhWmg5wHRUc10/xJK/xY8TgSCg==;31:qSFmQE59yEJozo7ZI+SoF1TjeV0Ge99Kmc2EvG7RqLgjBqddz22g1YIy10MQCsCdIIYGwRQbyY7DZTztzvaloPPyHeyLm3C3YJncHR+38tjXuI98/utoam01sA1Y0/IjOEbTsaSqQm5UQNHatULO5gU59NykqsDyzr0Nt3g1TfYoyYn29rmU/fX2R5vIrcf9nTCsp6uXFFCzthVgMzShoWo4C0eQLdXIElxBlRvPlB0= X-MS-TrafficTypeDiagnostic: HE1PR0801MB2026: X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB2026;20:uxSiRbuNq9g+t+GGB6dqbT7WCFaVjdi4FiBx7kgQIEmlewVJmB/UcLlcG9P6CXjiZfr28LPuk5c4i0XEo17jRISs/vNGtd5UlXNhHwOn3LEF8k/s1Jm0Ondxx8y5ixOqW4KQrRgAy/dF8ibeQ9S5QTS/TWIOjuaDVYM8cq/qTbqQZB+hEfU6Zzse2aXTRC6ZDNw8V3oUfn11thjTRWW+A4RNCktJXuwK5FY6WgNl0Etv6qirmjUDTnY7PFlT4sDkemO+GNNxsxH2qqkB1ac28wdV0xrJ1KW28nzSuA4VKgnxTIK9g2OTng3nMtz0Re6cCuuZoC8xTzGGUVrGtnNte9w8X2v/yDbVY5E0Rj/hhtz/adJDzte7l2zdp9U9yXLpUyFSWf2iLTJi8tpOrHh92omtnfnAfoMUO5Jhgnt1TbmB2nNq60PvArtLuoVTorS8E4to7/wxtM8vltg/u0nQRaYJV8Ji0hpGRfEsT53IP+UhW/YicMu0nGW3lZyie9z1;4:S/6o0zpfGWREuOGYy7IsOo7kpt7CpP/3ipPfCvMqt3x/MGK51GKX60YNdVd/IrIO+8viiUj8YoxODxcDDYMjxTLuhGhWM8VdtRnetZxB1sWAFs0CyXuhTlLKviEv9W7gHKmasb2obWr+Y44bdP+kPHi9bxFoE2jbzKhdSFPx3tQ81o6TwrNFS1QkhB3rFs+HJajTZ1BQRZGkU/3o6FJzT1A02d2vfVXAXx54CCAQGbsDBI2blBGLAXXj8ITsMBhjhu3VnQjXFA8SoHCxJsN7/Xdibv4PdGb0fxdXaPMOT2ESYzFu+PWZmFpWQVsKrb55 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(209352067349851); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040522)(2401047)(8121501046)(5005006)(3231311)(944501410)(52105095)(10201501046)(93006095)(93001095)(3002001)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(20161123558120)(20161123562045)(6072148)(201708071742011)(7699016);SRVR:HE1PR0801MB2026;BCL:0;PCL:0;RULEID:;SRVR:HE1PR0801MB2026; X-Forefront-PRVS: 07584EDBCD X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(6049001)(376002)(346002)(366004)(136003)(39850400004)(396003)(189003)(199004)(2906002)(39060400002)(386003)(6486002)(58126008)(26005)(478600001)(5660300001)(77096007)(16576012)(68736007)(305945005)(476003)(7416002)(36756003)(7736002)(7406005)(65826007)(53936002)(186003)(16526019)(956004)(31696002)(11346002)(2616005)(50466002)(86362001)(486006)(446003)(25786009)(230700001)(66066001)(14444005)(47776003)(52116002)(97736004)(3846002)(31686004)(64126003)(105586002)(23676004)(52146003)(2486003)(6116002)(106356001)(76176011)(65806001)(65956001)(316002)(6666003)(81166006)(8936002)(81156014)(8676002)(217873002)(921003)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:HE1PR0801MB2026;H:[172.16.25.169];FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BHE1PR0801MB2026=3B23=3A8YgZy?= =?utf-8?q?h0w/xaI3umRPYJQDzpF4DyYXM7WWWBQMIjVU2PvdgyvP+DzSPSDEMD+B1TF3PT9O8?= =?utf-8?q?jczclAyQQwRwV5kRx/CUQFU7TMs8gw6xGX9DjTAv+C5IfFBwaO7xmhuDAqlTZBpF0?= =?utf-8?q?07SA0dAWqwoeWkDfE8mvcOs1uyex2szHqUnMT1oT10ABBPfrmpgMcb/EHVY3RAe30?= =?utf-8?q?fpemC8XwmVflN5v5rQak9cT02CskzfGIHkUdpbxqJqRd/h4F2ztapHCfRcPeW8krC?= =?utf-8?q?ysIvzE2pIy/lmEsIwif/iJgprvbzVkuhiqFxPaitfyHK98KkdmgILVSORjEZTfJLy?= =?utf-8?q?TArZb/xzrL35vXNs6+qdyi2Mxns7oZNH+fgWj5TPlTpZc7yks1o4zdZAbhqNBwfqr?= =?utf-8?q?C+wNRh1d9DGRiZ7+wk2f81RH/R99O1dY6pXwyOCLkroDVGm3urk5gjE9ysOQyIQjD?= =?utf-8?q?CHvg2EwaUcvANkJBqE+Jw4o6GYY16g/rLloMQxDVeG0jPcrPyWTOEIsQGVDMtgDeB?= =?utf-8?q?LkmKWGAHbozR+nlgqLzkKTsfo8L6Nm/0HFzBVd3XOT34tZjOsQYuQAql9G3UlTX3d?= =?utf-8?q?4LyTLjKRg2X4YpWT5ra4aF2q9fajBCLmapNRR3dw94W8EMjN5VQjjVLGFkMDa3mwa?= =?utf-8?q?ZAs4n066xcy7sAdQAfgn5pDoI5w7AVlFSDbUQ1QY3h70Uj6Rcu7ouEPqTPSeGWTIp?= =?utf-8?q?NuCzBUtziOlZsx2BaTbPx7oiOW0eJLNpSxOx6rAqh1Cr20cLof4L2doYGrOy+NNkl?= =?utf-8?q?sFhGdxA25bPRfe5gxyaye12DYPTUq53b28XUWhGAOJzxuuT2FO24TES6gcTWj270G?= =?utf-8?q?ADac0xQvI/9QcUZ0qe/9S8dctUozf4H1+/ex2Fv+eCwL++14lBT7/Wu1kujqwFVCf?= =?utf-8?q?sp1/QvKaO0a+3Fww4hjuZYPa+UyUewHCRd5drZtuMSpxWUWwZMTKLuGithEREKBi8?= =?utf-8?q?HyMwxK+FckhWgqSkfkFoHiwmGFhofNSkFOIAqvy4nadIVYGycQHHyVSV/TPJ9nSmd?= =?utf-8?q?OJX1B8jnGmEcknr1ib5VnXiTCtlFjJTHV+dqicKs4yjXTZBlD47fB/70iRNjkhmyf?= =?utf-8?q?OVSPTt0tBfnnJEmUu21wa8Vn+JmD8hEm8gx2ljbNauyTU+8j6Shn8H4r102AECgNV?= =?utf-8?q?PrhJkDtnBNPBB8wt3JRB/U2gcbalu2TgZm0LYdv8PkE5o2tTBXH9xKjwbVI0mw2qw?= =?utf-8?q?HyEvU3II/KPcYO62FB1FKnuJn2h0JtxhkTlHYWuZIS6UG7jSTNsYSvucaz/g5hlAq?= =?utf-8?q?cfTd6aPmCWsuyhW9SXh1cLcW7Bc3b1rbuIsT2QzFB6fPbk2NS66UgX/2N0IvMMC06?= =?utf-8?q?XLwJ4w55u08Hzxd0zEn4Q6qFqXG/NgZDybdUkyoD/FBsq90B54xNBTzSrMalqVNSW?= =?utf-8?q?YKIZh4e12KD7Arc4=3D?= X-Microsoft-Antispam-Message-Info: QoOx+IN5vc8CkkwLStJKHimWnvrAqWUyLNW8fipTabNPyRrspvD3z+v6HYnX9iXUC3WyO/0vGCoOuw754pDIS3PQoPXn7A/piXAPhQAUZiIAgrGHiirE5xlAKD2w28AmHD690E83CcZ95A0+HTl2HPgU3reGztMmcqJEfQe12drpaJFW+hng0UDg+3opxPjqEB4whkz7HVegut7IMR/kvv/khYHgBcMOFN7Z7avOKebHcJnNrzPcBm0RIck9Jhg1uti1kH4k323vPCDqvCYoz7IuBJVWHNP+hLRmp/cAzggO9IXV6bl8YvT5tslrBX12JwNE/RKmBgjgEkATewqaOt6DFdP6v/e4G/oEYwf2Cg0= X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB2026;6:wA3s3743zdOAnYioCNHsBKaS+6NYIy8dnUqh6m+NLtxavQpJPvxME3bZSH3Y8WChqW+WyRmXTJF/X2IHoPAFMkKK2BjuKbv0e3YcHsj+xXFZrzH7gBJXtC9/d/bQNQZ9ifFsP8djDj/wlCNO7flqWlzIL5QnSjMAmFWYWAS/RlarxoQurEJmKHocHxYS6SU+6UEX3QzlvUMIS8ilzHqqqO2Pw4SoaN5T6FrpiAk2OX37nwwTGDWiV4FwgZopaZDYjpoQjTxxxUtRQiI+5oGOC0Gl3KEg62nYtv0ddssir7pC4BNzjyOUl5TxaOPHJ0D0+nDRmw0XbVGo+d/M/qQVEuFyDpnlR19uUmhSLqLFQyXFNJsevvCoN4rOGn6kPh0c1sO5LVWjC9h1SbxoGgKEOuZIXVlWaSt+423+dmivSQE0KP82GlhKkzpZjzVuBdRBuwsi7UoHwhimrtRrkryDjQ==;5:+LwSBVsO7JsfcuELBqb0Kgd3GgVQGa7iQjUeZWBq0fDg1x6t+NIjmr/Gg000FE9SnrW/SlewU9J55LTJx9mEkyr1ISXSlCFYwHqGM3ykYEUUAI4Pp6Ej0Sci3JLItkhYjYkxSqMxfaQClaChhK3thgdleHBwzzfCyqU4OSdtBCk=;7:D+kKWM/4zt8A58Cm5KPsdPrBnGP7T5btQZ+nFZKYnpcZZATC+9N+1aWo5MhUiFl0BeEopQWnF1j4146H3cksh3+K5UF107EerfCOR4+kHtUFX4+zLeyF24OS2EbYhYZb2HK69laJ5RzS/wio7w+vlC40BMT/SpMzLBN8d/Yh1vzpvu2qTCAYz0ZxgLsCokLbny9HfXxKtbEWDXwZRZwIfPthk1ZOYEgGhEpoItplIktMOvowvMkbqAIumkGv1Sla SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;HE1PR0801MB2026;20:CFid3tb4mVyCAr34YSE+4DykIZZ4dpEHhEkObqiBLZ6AWIGhb7uDbhQrx1Jo4o8GcVer6ugOaCxUG5KWUr1UWlPp4pr7KrO602j5blcd16M7QhSTazvqMOGfh28VyZkbbtADHjf6KZORnB6lSpgR7aPdwRYW001RVe4anM8nWjo= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2018 13:20:59.1586 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84edffd3-7187-4dd2-4d2b-08d5fd31ca6e X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB2026 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP [Added two more places needed srcu_dereference(). All ->shrinker_map dereferences must be under SRCU, and this v2 adds missed in previous] The patch makes shrinker list and shrinker_idr SRCU-safe for readers. This requires synchronize_srcu() on finalize stage unregistering stage, which waits till all parallel shrink_slab() are finished Note, that patch removes rwsem_is_contended() checks from the code, and this does not result in delays during registration, since there is no waiting at all. Unregistration case may be optimized by splitting unregister_shrinker() in tho stages, and this is made in next patches. Also, keep in mind, that in case of SRCU is not allowed to make unconditional (which is done in previous patch), it is possible to use percpu_rw_semaphore instead of it. percpu_down_read() will be used in shrink_slab_memcg() and in shrink_slab(), and consecutive calls percpu_down_write(percpu_rwsem); percpu_up_write(percpu_rwsem); will be used instead of synchronize_srcu(). Signed-off-by: Kirill Tkhai diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 9443cafd1969..94b44662f430 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -82,6 +82,8 @@ struct shrinker { #define SHRINKER_NUMA_AWARE (1 << 0) #define SHRINKER_MEMCG_AWARE (1 << 1) +extern struct srcu_struct shrinker_srcu; + extern int prealloc_shrinker(struct shrinker *shrinker); extern void register_shrinker_prepared(struct shrinker *shrinker); extern int register_shrinker(struct shrinker *shrinker); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4e3c1315b1de..ed40eb4b8300 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -332,8 +332,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, lockdep_assert_held(&memcg_shrinker_map_mutex); for_each_node(nid) { - old = rcu_dereference_protected( - mem_cgroup_nodeinfo(memcg, nid)->shrinker_map, true); + old = srcu_dereference_check( + mem_cgroup_nodeinfo(memcg, nid)->shrinker_map, + &shrinker_srcu, true); /* Not yet online memcg */ if (!old) return 0; @@ -347,7 +348,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, memset((void *)new->map + old_size, 0, size - old_size); rcu_assign_pointer(memcg->nodeinfo[nid]->shrinker_map, new); - call_rcu(&old->rcu, memcg_free_shrinker_map_rcu); + call_srcu(&shrinker_srcu, &old->rcu, memcg_free_shrinker_map_rcu); } return 0; @@ -364,7 +365,8 @@ static void memcg_free_shrinker_maps(struct mem_cgroup *memcg) for_each_node(nid) { pn = mem_cgroup_nodeinfo(memcg, nid); - map = rcu_dereference_protected(pn->shrinker_map, true); + map = srcu_dereference_check(pn->shrinker_map, + &shrinker_srcu, true); if (map) kvfree(map); rcu_assign_pointer(pn->shrinker_map, NULL); @@ -427,13 +429,15 @@ void memcg_set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) { if (shrinker_id >= 0 && memcg && !mem_cgroup_is_root(memcg)) { struct memcg_shrinker_map *map; + int srcu_id; - rcu_read_lock(); - map = rcu_dereference(memcg->nodeinfo[nid]->shrinker_map); + srcu_id = srcu_read_lock(&shrinker_srcu); + map = srcu_dereference(memcg->nodeinfo[nid]->shrinker_map, + &shrinker_srcu); /* Pairs with smp mb in shrink_slab() */ smp_mb__before_atomic(); set_bit(shrinker_id, map->map); - rcu_read_unlock(); + srcu_read_unlock(&shrinker_srcu, srcu_id); } } diff --git a/mm/vmscan.c b/mm/vmscan.c index da135e1acd94..acb087f3ac35 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -168,6 +168,7 @@ unsigned long vm_total_pages; static LIST_HEAD(shrinker_list); static DECLARE_RWSEM(shrinker_rwsem); +DEFINE_SRCU(shrinker_srcu); #ifdef CONFIG_MEMCG_KMEM @@ -192,7 +193,6 @@ static int prealloc_memcg_shrinker(struct shrinker *shrinker) int id, ret = -ENOMEM; down_write(&shrinker_rwsem); - /* This may call shrinker, so it must use down_read_trylock() */ id = idr_alloc(&shrinker_idr, SHRINKER_REGISTERING, 0, 0, GFP_KERNEL); if (id < 0) goto unlock; @@ -406,7 +406,7 @@ void free_prealloced_shrinker(struct shrinker *shrinker) void register_shrinker_prepared(struct shrinker *shrinker) { down_write(&shrinker_rwsem); - list_add_tail(&shrinker->list, &shrinker_list); + list_add_tail_rcu(&shrinker->list, &shrinker_list); idr_replace(&shrinker_idr, shrinker, shrinker->id); up_write(&shrinker_rwsem); } @@ -432,8 +432,11 @@ void unregister_shrinker(struct shrinker *shrinker) if (shrinker->flags & SHRINKER_MEMCG_AWARE) unregister_memcg_shrinker(shrinker); down_write(&shrinker_rwsem); - list_del(&shrinker->list); + list_del_rcu(&shrinker->list); up_write(&shrinker_rwsem); + + synchronize_srcu(&shrinker_srcu); + kfree(shrinker->nr_deferred); shrinker->nr_deferred = NULL; } @@ -567,16 +570,14 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, { struct memcg_shrinker_map *map; unsigned long freed = 0; - int ret, i; + int ret, i, srcu_id; if (!memcg_kmem_enabled() || !mem_cgroup_online(memcg)) return 0; - if (!down_read_trylock(&shrinker_rwsem)) - return 0; - - map = rcu_dereference_protected(memcg->nodeinfo[nid]->shrinker_map, - true); + srcu_id = srcu_read_lock(&shrinker_srcu); + map = srcu_dereference(memcg->nodeinfo[nid]->shrinker_map, + &shrinker_srcu); if (unlikely(!map)) goto unlock; @@ -621,14 +622,9 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, memcg_set_shrinker_bit(memcg, nid, i); } freed += ret; - - if (rwsem_is_contended(&shrinker_rwsem)) { - freed = freed ? : 1; - break; - } } unlock: - up_read(&shrinker_rwsem); + srcu_read_unlock(&shrinker_srcu, srcu_id); return freed; } #else /* CONFIG_MEMCG_KMEM */ @@ -665,15 +661,13 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, { struct shrinker *shrinker; unsigned long freed = 0; - int ret; + int srcu_id, ret; if (!mem_cgroup_is_root(memcg)) return shrink_slab_memcg(gfp_mask, nid, memcg, priority); - if (!down_read_trylock(&shrinker_rwsem)) - goto out; - - list_for_each_entry(shrinker, &shrinker_list, list) { + srcu_id = srcu_read_lock(&shrinker_srcu); + list_for_each_entry_rcu(shrinker, &shrinker_list, list) { struct shrink_control sc = { .gfp_mask = gfp_mask, .nid = nid, @@ -684,19 +678,9 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, if (ret == SHRINK_EMPTY) ret = 0; freed += ret; - /* - * Bail out if someone want to register a new shrinker to - * prevent the regsitration from being stalled for long periods - * by parallel ongoing shrinking. - */ - if (rwsem_is_contended(&shrinker_rwsem)) { - freed = freed ? : 1; - break; - } } + srcu_read_unlock(&shrinker_srcu, srcu_id); - up_read(&shrinker_rwsem); -out: cond_resched(); return freed; }