From patchwork Fri Sep 8 14:53:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13377567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B733EE8011 for ; Fri, 8 Sep 2023 14:53:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A1EC6B00D2; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 477386B00CE; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C8A56B00CF; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 16D316B00CC for ; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E0EDAA103B for ; Fri, 8 Sep 2023 14:53:22 +0000 (UTC) X-FDA: 81213723444.29.1A56C4D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf23.hostedemail.com (Postfix) with ESMTP id 0026214001E for ; Fri, 8 Sep 2023 14:53:20 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KUyJUvRQ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="fZHG/B1Y"; spf=pass (imf23.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694184801; a=rsa-sha256; cv=none; b=hyrgXVKMerDNqz4OCX0y3WIyR96UsyNMg0U6c1hMsLtML8pRmPqTo2DEwjqAUL8VRfkxfX JncAggVYIWHrmnQKylftM1q1OeEKw3kxEMLmMOsO+dbUtb2NZGu8l8q5SBk9MMsMiQymhT Fdkdy6hfFAntVXSPG+UNbwMC+Wy+R80= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=KUyJUvRQ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="fZHG/B1Y"; spf=pass (imf23.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694184801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9/ffVZ2jgkNFPV8v9GjIklo4R9arDXW9nEkTxNB0K6c=; b=bmbqKpt6bKus3ChEa/XkA5koOd5GVQzq84icDNM5asD83NYnzFvqBrAlwpSdsPQ0BOUy1S S2MGSYAcW+4/nG9OOObNd2crflafrzQB1U6lrbUX3YqSpbXftdFPP/8izSCavYlmzq8Alt tGgYyxpXozwDgOxu4Vzqb6hC3tkAoeU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 065781FEFE; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/ffVZ2jgkNFPV8v9GjIklo4R9arDXW9nEkTxNB0K6c=; b=KUyJUvRQS4gA2ke4GVINk61D6KwZrbVETPHCWg2keeQHFwJDBjRWoICJcRSmEkd1dxkjj5 TweCBVVTjorbVIzvDPI61iMA74jg4u+dwy5WGvFONU/ZC9FZihSGpc9w87jcACXu3wiaDg SwKeICwK+C9hN+SMhocIUAFapCddKZ0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/ffVZ2jgkNFPV8v9GjIklo4R9arDXW9nEkTxNB0K6c=; b=fZHG/B1YFdRe43J1fIyQGYzL7Ak+D5TYIdHhViuuB1fMP+TMnern1sOdtMTiv+4XRfZu7K CUoHOR832W1ZyoDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D27E713357; Fri, 8 Sep 2023 14:53:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id SDC+Ml41+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:18 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 1/4] mm/slub: simplify the last resort slab order calculation Date: Fri, 8 Sep 2023 16:53:04 +0200 Message-ID: <20230908145302.30320-7-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 0026214001E X-Stat-Signature: i3ipgfeob4mkt4hd57t439brnru3oytr X-Rspam-User: X-HE-Tag: 1694184800-877943 X-HE-Meta: U2FsdGVkX18NgWXNRyzgoFtf+2uzSGgf56Djxvty0nyFAJWiRFW9a4kFZtiq0Yr4Mrr1IQ3LefJbEE9sLYgIAbkJy8AGeyQO7nN8cPaSmszmWQXVMSSwydahzj+/2nKnsMuCD1gsVJlSU6/iwAdPqg4KazVHdnSMlvkPN0f7IKgZSFoWg9lL3cFJUUXBF1Vv3NB7nsOdZp5ynJsS+jFe3EujCpCKjufADExEz9xMfkknz6l6oGnuosAbXqt8TiOABrAbMkGtjGwMt1o+z+uYxcj1rUcOeapTMHwwqy6MSfDNJWUFVAVtnJlIuzVhhfdI8zmc/At/RuA50fOhAZ2mrRzm2t1kXF2xEdJ/UhTqvF/zaSVYliUBPgjGiHHC8jjNL8l9YLkMy+VNcI3TL8/VPWCwqmFMY0iH0/vsOzA465OtTeO5ONgSpUj/Cpr0iIzhq6u0i+zkc/0UQe8Oj2KVbp98s3h/gZX3nRI0zbUiay3mYhFwV2ZmTtFxukemf0BAh692jKun9UIcoHM6gV1TgZ5zTUmpitZ4YSMlmZnrDsNvNZCsVfqBDz2eN73c5o8dUy/cD23Bp8DKUKHlSnC9Ij2phSv+ZkfDjELkEpLkgz6RRXRhZF3IGxFNgRuPLQXtxbrHgFS8WWfDAXNlMJyBmxtIwbSONjoFVk2+5Ow3T9G2arfk6ZREMxcawVLRVuJtMEtqkInfz3wJEh7KS1y6rkb9drMnEteejtAkASiYL2oM34qmFt5JN7Hr2M/5kgsOBLPlpon9V5uc5xWp/bMN5O+NGCzCsXcQXpBqa/TegfhCUcrJlYNb1h5URYHX9OI2il0Z6f9OKyVljZaVyodrq9lBLr1WBSespDuoD3ETV3zTrrKXvEGRjJRDCGQ9ZoZYT6apZ18lUoup252Y7AhlEErqhyEDop+6zwRzka2g80/Abn0Fd5RA9Xh1nNIv7uh8qdIe9XCnALUdxK29Lqe rGK3bI5X FTSFSdP5UaVEsNV9WdYAXBi0FPNkw7IEKahcObMhKel7/XOUxzta9ZP4ssxmb/pzknDTO9MwZf6qBjkhJIma8wxiFOVpidWPVXzlLoHHGQXipa74SqYKgknGHLKD2RHv8g2VHgY0GOvGr2iRPMWu0PkLwmObFzC7nKimBk/TniJdZEpx/rw2BHGcc85AbCBcA/fhN9j2WBza2NJV7UUTb0BLC/KysjJmAH2rID1ot92IAxWn29opHSHx4lR2g76aOCw56 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If calculate_order() can't fit even a single large object within slub_max_order, it will try using the smallest necessary order that may exceed slub_max_order but not MAX_ORDER. Currently this is done with a call to calc_slab_order() which is unecessary. We can simply use get_order(size). No functional change. Signed-off-by: Vlastimil Babka --- mm/slub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slub.c b/mm/slub.c index f7940048138c..c6e694cb17b9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4193,7 +4193,7 @@ static inline int calculate_order(unsigned int size) /* * Doh this slab cannot be placed using slub_max_order. */ - order = calc_slab_order(size, 1, MAX_ORDER, 1); + order = get_order(size); if (order <= MAX_ORDER) return order; return -ENOSYS; From patchwork Fri Sep 8 14:53:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13377568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A750EE8012 for ; Fri, 8 Sep 2023 14:53:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D5276B00CF; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 75C536B00CC; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 433316B00CD; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 211096B00CD for ; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id ED0EC1CAD63 for ; Fri, 8 Sep 2023 14:53:22 +0000 (UTC) X-FDA: 81213723444.27.7E4F476 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf15.hostedemail.com (Postfix) with ESMTP id 1BD8BA0003 for ; Fri, 8 Sep 2023 14:53:20 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=YuP4yGAg; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=zdKRoEqR; dmarc=none; spf=pass (imf15.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694184801; a=rsa-sha256; cv=none; b=ytcU5Z5Sg5OtOGKEdkv2FMMLu7u5zLjNECCs3I4YbFya5HqXDQwX75ihqP06iOEeCKFW5q cIrcWJxOaSp3kv7ObPunSZ7ZzqeS8k7Ypggz0IFiaweThY0n1eWxdYDedEDIrcvYorAbKY eOM/0H+MgfxNJre1nb0nwu9sokY6Y3o= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=YuP4yGAg; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=zdKRoEqR; dmarc=none; spf=pass (imf15.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694184801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1Z1sKJpFTH+GkYm0khClKXyg32JfDaf6DR+O1U32B1U=; b=oP7vwWkD9GqZqZdK/8GIiSMX734dAp7ixRPKkAOyxx2mlMrZsJSQJnX9FhDmE2ssu8puV4 fOvvKOvUlpfeXRqKgIr1g8CW28Iv/WIKdpUOBHioLo+OaAo89eeOvkmGb6jb22Ao7M1kV9 9k77OsU7YlhYHspLnrGnnzU1fJKf61k= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2FCAB21D25; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1Z1sKJpFTH+GkYm0khClKXyg32JfDaf6DR+O1U32B1U=; b=YuP4yGAgh8IJXJqsJ73xpH1c8mWzwQKxk19hlkWByx+xqNYN38WeOb48/N6bxU0xCx8maW W1jKRpnjfatbKB8YELQlCeuLU2hRZzscE/fnXEfChJl//ljQGtUhCDbG9stCFPzEOpYgEa m9vCuphfho6ykKwxQy2FsDwCQfrC7nE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1Z1sKJpFTH+GkYm0khClKXyg32JfDaf6DR+O1U32B1U=; b=zdKRoEqR5QeAuWFifH3Asdg2OEmp22MdGD1d+kXPIOxuQfreIS8fFSyp4hCLbK4CVYH8py dB2y1nrKP4MuEvDg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0857E131FD; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id AMVrAV81+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 2/4] mm/slub: remove min_objects loop from calculate_order() Date: Fri, 8 Sep 2023 16:53:05 +0200 Message-ID: <20230908145302.30320-8-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 1BD8BA0003 X-Stat-Signature: tj6b6nzch6s835jh8eugyjbue5h8ods7 X-HE-Tag: 1694184800-960893 X-HE-Meta: U2FsdGVkX19ArIKbzdS9+yGwJoyri4jyOQYjwOQxSkUVL/L4PI1fCeiWmRfibN47FqmBKn+EWNGcAnw8GofrOOH3LS4UYiZNGJkKC+eGGh0u1IJtJ/deqMHok3gN9T3bP97ATgxnq7cwQMmGeMActjiPQy4QJ4Jzx4WxSZnaL7sOd30Dn+hYucKiIyburiRXgt7rBW+c2hyv7trmeTdWpbnLWeLBq724UTr/NDzUf535GWyTBLOF9K4Xxsh48nkt+mBTIb8pkIBRUcQYocBiL+OT/Mp/lT6/4r30y69ckI5T1+ajqgkd3eriWSkLvkLGLMFr9onokG5fg2EL4iKO5oG7Ogg2az/PQQKYmgKkG67OwNEcprjHwiHWoFlS/kJs44RKU3/7oq8BLaBwheYU6AWe6I0rmj3uSupCdF4u1cilzOHxJIL1ntf3khrGFiEZuR2Xy44LdV7egAFrj6sHCJ8Rno46k1TsTfeF0ayNjG+TbIcNjeoDLqFDAWj2Il9LryxnTqLROgTZPis15l3L2vDE7RWQncJLCpnW87L9iQzWdwnOyfaDNh1TR09WMboZ+Vu3GL5elOVr/MtcVs0OXGGnztGe91mvued2s/8rNeq7D+tlh3fyWZZzmW0HpFQOYDjbXBpUSzkPM5PRqLCcqh1wzk+4wVGDx6BUE4/pF2NkNZwOEre5aN91HzaMhHKFZssbIWKqf6KAWnmi7ZhxVnL6Ssbvjfn08q5OQwhs6ziqd+iGgzPluYRzUxQHVeRCYj8YmsDEA/FnP+dWo/xgebFT2zPfvlDp2Z2XSbB7Gxl+hbVzElFXQvaeEHJHMpjJN/9DA8/qPauMrw+p7Pu6FS8nNgyENsDk0BVRieFKyKlpejf+BSpaeFXp9Gp5t/8hlpUnUCUqt3o/BdUJ/QO92dyrfzPrx9UZebvCxjCsTQyu/yrA1EV40NZUjKUhpboPU6vupdbp9i802gSydlO DFx9lLcn kYaWw5PapLq2BgKpWEJXvkFoj9695QSkR/fyF4D+IYVuUxV7eaAkfG5Um6ZRiG4LO9hzBiMpoRR44//wLOaqVX3fNn2D+v9+ESSv4nGvJx5lKBxFPlAu2l/I/x3TQZhh9B0XDVlnV5pW5fkpIqe36gSY/lpFCLEvYVJKerTf6zL31kI9G00R5vxf1li0GnsiphllhrgmNFYDWJVnzpRxA60ETQaIkp6c8d4jSqXMeR6dI1Btxs9nfmD7kFqjQoGlQ4OfCa/i529wRZUOMZQwarsn9Yxd6h7DOP9EOXKXfCL7TRdUs/dmvMevwRg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: calculate_order() currently has two nested loops. The inner one that gradually modifies the acceptable waste from 1/16 up to 1/4, and the outer one that decreases min_objects down to 2. Upon closer inspection, the outer loop is unnecessary. Decreasing min_objects could have in theory two effects to make the inner loop and its call to calc_slab_order() succeed where a previous iteration with higher min_objects would not: - it could cause the min_objects-derived min_order to fit within slub_max_order. But min_objects is already pre-capped to max_objects that's derived from slub_max_order above the loops, so every iteration tries at least slub_max_order in calc_slab_order() - it could cause calc_slab_order() to be called with lower min_objects thus potentially lower min_order in its loop. This would make a difference if the lower order could cause the fractional waste test to succeed where a higher order has already failed with same fract_leftover in the previous iteration with a higher min_order. But that's not possible, because increasing the order can only result in lower (or same) fractional waste. If we increase the slab size 2 times, we will fit at least 2 times the number of objects (thus same fraction of waste), or it will allow us to fit one more object (lower fraction of waste). For more confidence I have tried adding a printk to notify when decreasing min_objects resulted in a success, and simulated calculations for a range of object sizes, nr_cpus and page_sizes. As expected, the printk never triggered. Thus remove the outer loop and adjust comments accordingly. There's almost no functional change except a weird corner case when slub_min_objects=1 on boot command line would cause the whole two nested loops to be skipped before this patch. Now it would try to find the best layout as usual, resulting in potentially higher orderthat minimizes waste. This is not wrong and will be further expanded by the next patch. Signed-off-by: Vlastimil Babka --- mm/slub.c | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index c6e694cb17b9..5c287d96b212 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4141,14 +4141,6 @@ static inline int calculate_order(unsigned int size) unsigned int max_objects; unsigned int nr_cpus; - /* - * Attempt to find best configuration for a slab. This - * works by first attempting to generate a layout with - * the best configuration and backing off gradually. - * - * First we increase the acceptable waste in a slab. Then - * we reduce the minimum objects required in a slab. - */ min_objects = slub_min_objects; if (!min_objects) { /* @@ -4168,18 +4160,24 @@ static inline int calculate_order(unsigned int size) max_objects = order_objects(slub_max_order, size); min_objects = min(min_objects, max_objects); - while (min_objects > 1) { - unsigned int fraction; - - fraction = 16; - while (fraction >= 4) { - order = calc_slab_order(size, min_objects, - slub_max_order, fraction); - if (order <= slub_max_order) - return order; - fraction /= 2; - } - min_objects--; + /* + * Attempt to find best configuration for a slab. This works by first + * attempting to generate a layout with the best possible configuration and + * backing off gradually. + * + * We start with accepting at most 1/16 waste and try to find the + * smallest order from min_objects-derived/slub_min_order up to + * slub_max_order that will satisfy the constraint. Note that increasing + * the order can only result in same or less fractional waste, not more. + * + * If that fails, we increase the acceptable fraction of waste and try + * again. + */ + for (unsigned int fraction = 16; fraction >= 4; fraction /= 2) { + order = calc_slab_order(size, min_objects, slub_max_order, + fraction); + if (order <= slub_max_order) + return order; } /* From patchwork Fri Sep 8 14:53:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13377570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF55AEE8013 for ; Fri, 8 Sep 2023 14:53:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D07D86B00CE; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CDC76B00D0; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D21B6B00CE; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3D85E6B00D1 for ; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 14D90A02CB for ; Fri, 8 Sep 2023 14:53:23 +0000 (UTC) X-FDA: 81213723486.28.ECE5107 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf05.hostedemail.com (Postfix) with ESMTP id 025E8100024 for ; Fri, 8 Sep 2023 14:53:20 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=yaoSVUcf; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=1F8uln1O; spf=pass (imf05.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694184801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=xV/340XFDJY0fsDFCw6gYFD8f/zU5cFRFugaOzcpZrFUWVW0yBrniBkcvPJBiRJbh4R1qK YLtuvvjnlU7TsZ5kDnNle222HKcAMWiMzgCX49Lq6IvBIhroyoew0BQuVTI6AeB45MuW2J tp2cSnV31b0PXp80xMKzxo05XCHAYtQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694184801; a=rsa-sha256; cv=none; b=4cPVS4lKAJmj0x8bH4JVaMvSKQR2bb2wlDrAnjv/rrs4djJt0LeASssUyEXcFKlV9VZZN3 pVCqfivC6nhyYucQJIxzPxcrVO0kDTWfEIclSpM/hu+ua8EJTouxCnWILSTvH921jjjBL4 vSlEZwVPAcThHOuO3xN7qTIpbMX7Fm8= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=yaoSVUcf; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=1F8uln1O; spf=pass (imf05.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 57A6A21D26; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=yaoSVUcftdz9Z0pvCocl71s6a42GoguCtjRHXa0NVRtJyYOG6A+9KfluK9Nx3m9FsnxsoH nQhFs/wVHJNXuBKUYjlgy0YSvjKAfZz5BhBOE/d/6MtSEdQP0EBpF8i9JPxbt4GAX6uJVi y0ZD68CA0KFlChqiBb0LB0e5BbPQwtk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=1F8uln1OGPghmt/eKsFza2ILUJbA9ByJ69bS4pA/H55d4XBFwWCspcGJ6dS96kWwjv3Hpo 6+lv3dA372hQ5VCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3169A13357; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id wGFkC181+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 3/4] mm/slub: attempt to find layouts up to 1/2 waste in calculate_order() Date: Fri, 8 Sep 2023 16:53:06 +0200 Message-ID: <20230908145302.30320-9-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 X-Rspamd-Queue-Id: 025E8100024 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 1jpdh6fzh8dba15woxchi5dhdzzh7fxf X-HE-Tag: 1694184800-245353 X-HE-Meta: U2FsdGVkX1/FH8OLWtQ3yK7GQN0RbhJBF4HiI2PelISVEQN/jXRFtaxz6XKMzdTQ6+PGTzk9XFNVsdLYuFI7IPHGJERCp3fSHX93Lc/IjSexqvcXjAzSQjjTMU+7cPbwHaPTNbz9+vcipxuiWhI8rbFWC4FoPa/K55LPImdskJBr4qnWtzY0u1numIH2EkFCavi5e063fWT/SRJdKPnvGqlczEpMQe31wW4DQHKSvo2MLws9iWad72PYbnuQ3XDknWwAL9sEW5+ZStwKlWHqttH+nJLPW70b935gpzB0P5HedlzT5PTCR9bxcHVpBi9b56AsVfzOF+YhnR0nr9fPdUJG/p2ElUog+Jfu8qnXEgi81wMMWL5LdleKIckxi4/qTkoGropgS0WSW85fa60nyAQLpxT/Bq/Ur/krjwynKVa/4wOU8Gxg0co/tpS9zEthG90+L6J2FFPHfg1HKltBlUtr61kD4aFHVXccpC8ttK+iHUHJuib6GZSVnBf/jtLyfz79IJOxt6vcUyIhLpSwmhl6F7ycNgNu4EaaFvWR5Y/qZOzbIjzb/SEeA3CaI3+ViS8QOUsqr++zTFFTfzPFivuMCPE4tq1nCsWUgu4oaRbjpZqlvnOa+L4oE5fMthgvlN+1Om2Io/o4fpjErBfWSaNQ3qdRVKXy36Cl1/8cPdAyRIIz+uhlm33T+sq0hSWCtgUs6kH4dnJXyoH/ESWFyDTy8Roi3kU6yQdme9FuhwlQT149Q398OSm/73tZaGPaqHaSp8/yV8H3djRQvPdvmMnhAAuBzIViGvVkGjBfK9MlxYN04aL/7+QEBRDqDhQZfr4F+6KO88MN7LgXH8aUjLe92PNd4P7ZqJAEhWerdSLCudFkxcDk+eR1bK5DlA19HbrZ7L7pdH66leQEPSwqY9ChDKTKfYybq063FytEEqUXZ+gGeoosBeJll2ibeZ02bFe7CAEuOKDnWpDaT8C QabBFWz+ EljxDjNgSiMjiT7e7kyeDzUcwUFW92xOf8Jh+FWefhOjKuT/45Vrev5Epm2gVWwIH85/e0HRPdnoQsp6CTQKw2Fmo1X/sISCjJJiyW9L7S1Km2ESxVI/ypHUhZgLFkZxL1oveMIm/+ZqbACRYDC6JDbU2yF1sFHla8TL3dXL0KmPMnoHXmq98aTiqF5vHPfr0IFlVEZoMroc/NFulYE3y6LRBie4qgMiFumD5gK1QWE87Wbns60w7iK1zVDPdd/libRlZKlhzoVEnHwJtqj6xgfsLXIWoryGqUkA//NMZMXrcnR/YKl1eiektTw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The main loop in calculate_order() currently tries to find an order with at most 1/4 waste. If that's impossible (for particular large object sizes), there's a fallback that will try to place one object within slab_max_order. If we expand the loop boundary to also allow up to 1/2 waste as the last resort, we can remove the fallback and simplify the code, as the loop will find an order for such sizes as well. Note we don't need to allow more than 1/2 waste as that will never happen - calc_slab_order() would calculate more objects to fit, reducing waste below 1/2. Sucessfully finding an order in the loop (compared to the fallback) will also have the benefit in trying to satisfy min_objects, because the fallback was passing 1. Thus the resulting slab orders might be larger (not because it would improve waste, but to reduce pressure on shared locks), which is one of the goals of calculate_order(). For example, with nr_cpus=1 and 4kB PAGE_SIZE, slub_max_order=3, before the patch we would get the following orders for these object sizes: 2056 to 10920 - order-3 as selected by the loop 10928 to 12280 - order-2 due to fallback, as <1/4 waste is not possible 12288 to 32768 - order-3 as <1/4 waste is again possible After the patch: 2056 to 32768 - order-3, because even in the range of 10928 to 12280 we try to satisfy the calculated min_objects. As a result the code is simpler and gives more consistent results. Signed-off-by: Vlastimil Babka Reviewed-by: Feng Tang --- mm/slub.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 5c287d96b212..f04eb029d85a 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4171,23 +4171,17 @@ static inline int calculate_order(unsigned int size) * the order can only result in same or less fractional waste, not more. * * If that fails, we increase the acceptable fraction of waste and try - * again. + * again. The last iteration with fraction of 1/2 would effectively + * accept any waste and give us the order determined by min_objects, as + * long as at least single object fits within slub_max_order. */ - for (unsigned int fraction = 16; fraction >= 4; fraction /= 2) { + for (unsigned int fraction = 16; fraction > 1; fraction /= 2) { order = calc_slab_order(size, min_objects, slub_max_order, fraction); if (order <= slub_max_order) return order; } - /* - * We were unable to place multiple objects in a slab. Now - * lets see if we can place a single object there. - */ - order = calc_slab_order(size, 1, slub_max_order, 1); - if (order <= slub_max_order) - return order; - /* * Doh this slab cannot be placed using slub_max_order. */ From patchwork Fri Sep 8 14:53:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13377571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BCA6EE8014 for ; Fri, 8 Sep 2023 14:53:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E15A6B00CD; Fri, 8 Sep 2023 10:53:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 191356B00D0; Fri, 8 Sep 2023 10:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E887A6B00D3; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B581A6B00CD for ; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8E65FA107C for ; Fri, 8 Sep 2023 14:53:23 +0000 (UTC) X-FDA: 81213723486.24.BBBDBFA Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf04.hostedemail.com (Postfix) with ESMTP id 6E17640025 for ; Fri, 8 Sep 2023 14:53:21 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=AErap95n; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=PBQchTCP; dmarc=none; spf=pass (imf04.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694184801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TBS/rfQAa9CuGAOXLs/AG66+kZ+35/pxxXJ2H3zd1io=; b=lvf5ajyMo7BqqNIx2w4A9ObEPTisnrNxjraZbpk7ZOxfQkGFxBlAbYIm61tnQLkxofi+gn cQE6I8JL72iK5+8grUkHP/MusIVb6wSiLIDIyB6ORmmmv5+frAkPctgUnEZXSTs0ESIjAk or5LPjJ39OVHMdnbWXl8dc4UIv9+TQE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=AErap95n; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=PBQchTCP; dmarc=none; spf=pass (imf04.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694184801; a=rsa-sha256; cv=none; b=5YN3wRz+HwnqR2ZsNGDmXGyrDOm8oENhfG7NZwsCCzZV2e59yEO6qaHkxpqgwYqY0IYFzm Nd1uDxIv5ae1PkOVHPZoOK/eh/lROmWsY+DSGwZog79kTqp5rUCU/auszSZ2gTuZy9ztaO ZdRKjoMMrFPobwC1tKjqzzC5CXTPYyQ= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 825EA1FEFF; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TBS/rfQAa9CuGAOXLs/AG66+kZ+35/pxxXJ2H3zd1io=; b=AErap95nCfprazdQmp9OXffnXycQVwIbGZpHDVA0rP1sASRm6I/cAKuQEAWyQwn3yPrIhj n2us9Yv71zG8n60xarSDoURo97o0UJr6tPpUw1eQl5dNdrsDvGofv7aMgtveGRu7mN2InU ErsEOQVo1bhO94LeTmRRs1dVrdIzL/Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TBS/rfQAa9CuGAOXLs/AG66+kZ+35/pxxXJ2H3zd1io=; b=PBQchTCPYcBZU1rMZMkGvd3jPhMJtIy0CS2wp+p1ullQX6NqzbDhx7UDZYJDCdQdtJKh3v ubVT/MasEOcFelDA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 59E68131FD; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id iIFAFV81+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 4/4] mm/slub: refactor calculate_order() and calc_slab_order() Date: Fri, 8 Sep 2023 16:53:07 +0200 Message-ID: <20230908145302.30320-10-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6E17640025 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: h3xtkx14cqfh7rqjf8imhrtadw15qsqi X-HE-Tag: 1694184801-541081 X-HE-Meta: U2FsdGVkX1+cbTcoqeu9x5QTEGtO5kNSg5nd9RUP0gESnHTGre9zO8NtFGrr2+N4QY+30Wq2+VYy4OiFvy1GEN2nWysQ7zWxym68Q8Kpwfn8Yi09U6m5p3lT5vgXpqu+PycRRFk/WoNb4JQWQfyraY+o/bF3xJ+KGW+Gr49sSwophKholO84OPCfy0QvAgVWDY5inR5UjRPwRc37W7Qq/IeiGGnFiKUQ69jKWWreDL2AE1UhhnM3LugupjgeJEJKuznu/eTOzJV0cfdLYwVZBTF/jMvSyfoIuOe5hj7PdvIIguA0pHzTNSvy7chAyKD8dD70EcmE++tBvd4TZthYnQgTQhr9g2fqmNsklCbhtysDs/OeAfoPJwE3aArhum5ma/6921gfwhSAxBnekJahpb6MRAaWog37ffrTk5Ad0Zdao38WJIsHH1FV0VkJgIGDGrlgMW92catTfb1Y8Rw1ARJORJ3XbP8ZMGxmsADJws+ZEiAwTINpJ6J1cqGgylHq6EcqReyXqWai0+SBen1SC27v9Er5l/3j1bS7McFIQsWuq5DvWO5hPo9/vZ96IUOAgxPs98LKVgen6HFYarEOnJfwCRHH37l9oodJjM4uJktGvhKZiAZeFlAgVDHD5+brre/NSdBmniplgP93uAIJK7h+Vt+XLoa91Ha4K/wawcO/C9+SNwQb5xMkmtmGEDFt3vBFoCjp81HMlJNySy8wJLlTHnDB+px/iOJ+NX5MHvlAe3aNmeeCPrF1mmEyOe5ZMUMEhoWqaGfO7ZLZmNz3iAxgNi+w0bH8Bg4A8vPrz5ClM+qMGRgzOoEwOrrp07bxu9Vvn+hs3QFu7xk4xMdhqgSnfJpo+nzCEtR5KQzcZorCxagm7kQgJZgLfqG312fvmwhDXehMABwl9jV1e3cfus2HpmoA0lL+khUyyZKYGdAzFKE4qFKyLgKfHWUsObgcHJmbi5ocrGOad2NnfgT iz9Jnmb1 lSFbdiqtEiywOvpI2LGGt91nq/DEKxTyMydjoG7JO3utfFhoGeqN0mqOfjSEB8x12p47czSoH0Jw8FeIhR8+gcmwHa+UXYD7/RCyIeBlsdm3zaKT0h88fccgphIDTl7jX9igc6tsr1OUaSSc0BPzIGVo3pEkBxw86K+ECfGKSWem/ExK1rMa4IA83NOcyR0DavdRhnyIhP3mQTMeNwD1giEqAWH+V66sprAqtojprDF3+Mz3UQIL0H+QFgciVQprZ6Z8NaLusPzKPAgHZwgK+/3FdMvPn+zu5lf9vdaXBNxPyvVvHEzo2SDAiGQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After the previous cleanups, we can now move some code from calc_slab_order() to calculate_order() so it's executed just once, and do some more cleanups. - move the min_order and MAX_OBJS_PER_PAGE evaluation to calc_slab_order(). - change calc_slab_order() parameter min_objects to min_order Also make MAX_OBJS_PER_PAGE check more robust by considering also min_objects in addition to slub_min_order. Otherwise this is not a functional change. Signed-off-by: Vlastimil Babka Reviewed-by: Feng Tang --- mm/slub.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index f04eb029d85a..1c91f72c7239 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4110,17 +4110,12 @@ static unsigned int slub_min_objects; * the smallest order which will fit the object. */ static inline unsigned int calc_slab_order(unsigned int size, - unsigned int min_objects, unsigned int max_order, + unsigned int min_order, unsigned int max_order, unsigned int fract_leftover) { - unsigned int min_order = slub_min_order; unsigned int order; - if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE) - return get_order(size * MAX_OBJS_PER_PAGE) - 1; - - for (order = max(min_order, (unsigned int)get_order(min_objects * size)); - order <= max_order; order++) { + for (order = min_order; order <= max_order; order++) { unsigned int slab_size = (unsigned int)PAGE_SIZE << order; unsigned int rem; @@ -4139,7 +4134,7 @@ static inline int calculate_order(unsigned int size) unsigned int order; unsigned int min_objects; unsigned int max_objects; - unsigned int nr_cpus; + unsigned int min_order; min_objects = slub_min_objects; if (!min_objects) { @@ -4152,7 +4147,7 @@ static inline int calculate_order(unsigned int size) * order on systems that appear larger than they are, and too * low order on systems that appear smaller than they are. */ - nr_cpus = num_present_cpus(); + unsigned int nr_cpus = num_present_cpus(); if (nr_cpus <= 1) nr_cpus = nr_cpu_ids; min_objects = 4 * (fls(nr_cpus) + 1); @@ -4160,6 +4155,10 @@ static inline int calculate_order(unsigned int size) max_objects = order_objects(slub_max_order, size); min_objects = min(min_objects, max_objects); + min_order = max(slub_min_order, (unsigned int)get_order(min_objects * size)); + if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE) + return get_order(size * MAX_OBJS_PER_PAGE) - 1; + /* * Attempt to find best configuration for a slab. This works by first * attempting to generate a layout with the best possible configuration and @@ -4176,7 +4175,7 @@ static inline int calculate_order(unsigned int size) * long as at least single object fits within slub_max_order. */ for (unsigned int fraction = 16; fraction > 1; fraction /= 2) { - order = calc_slab_order(size, min_objects, slub_max_order, + order = calc_slab_order(size, min_order, slub_max_order, fraction); if (order <= slub_max_order) return order;