From patchwork Fri Apr 9 07:17:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12193205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36D81C43462 for ; Fri, 9 Apr 2021 07:17:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 93C1D61177 for ; Fri, 9 Apr 2021 07:17:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93C1D61177 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 185146B006E; Fri, 9 Apr 2021 03:17:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 10C286B0070; Fri, 9 Apr 2021 03:17:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D92C36B0071; Fri, 9 Apr 2021 03:17:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id B75156B006E for ; Fri, 9 Apr 2021 03:17:30 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7CED71830EE7D for ; Fri, 9 Apr 2021 07:17:30 +0000 (UTC) X-FDA: 78011973060.09.7D6BC3F Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by imf23.hostedemail.com (Postfix) with ESMTP id CBB17A0003A0 for ; Fri, 9 Apr 2021 07:17:28 +0000 (UTC) Received: by mail-ej1-f52.google.com with SMTP id u17so7065005ejk.2 for ; Fri, 09 Apr 2021 00:17:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=MAgKFkUbajfTwkiBbjHCwLbY8LqpQ8PBmh7VA/cwU0E=; b=igEspD4oLTpM4o2jCwSGcYwjzgvaFDiBYrZde/yigoCOzASZ6Q36kFdOLTPk9+sEKA At9Rg6okYKLJenOHCVlutmLzuLAhyDZfgGBYJ66Oj8C6MPsS8wita034AoWkMFIgC25Z 7hsoj7iGl8V5wzeL/wT3tqHTfrZTHkZwk5UBViKyHTywBmZFq0frm8mI3J2xlddZBmoE s8q8GSI2WepiZ10MAzwDGwDMtYCoPpkyTnDhEtIa1GyGDEw6k37Rs+HJ9yff6WL6sw1O 4WqcJX+pPtmQilHkwSI8hffyk+1BcLWllcyd2nRmfoTOw/Abay7/0UE6y0hP+L+s4Qlb YxuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=MAgKFkUbajfTwkiBbjHCwLbY8LqpQ8PBmh7VA/cwU0E=; b=AArSiaZ9geZ0P7m9kkRJNNw1gGNjbEGbAP6EDXtoiGRcCuymWOSCcAOe0cGknmgvpH xTyrq78R9V4js1Jg3g07iyAyf9TJC0fknsDIuWlaHUSdXW8g/IYPF8aAEhoygGPhBynn 3wkgNaeF1PlpD7FjYjvyFc0BaLh5nralz1uB5rHPIh77Mzz3lB7trz8Sb9ih79AmeGZA Nmq9LeXkGAi2L9eAXJMiBDeyzyQiL3YUSRCgRVBmzSoUxWhNtRsIsQLSGuyCV9P7bt5o FDC9fhqYprazNUpHMKKm/N8etW6xMHxl73TuUY+W+XlYu3Bx3wOlBz84ZfzP6VMUOUIm FKow== X-Gm-Message-State: AOAM532RsZaR9tTRvB/Dk/xJQXlB2Fw5BVborsmmr5JNqtYJJKrWlPyj 3jmxSsgnmpWBpiBU1m8AGew= X-Google-Smtp-Source: ABdhPJzQ2tCFOmAsi20Gal6ccZzgbiWTy3xs28H9Fe7zY+J/NCEjtr1WZzzVEoA6J7yNPFrn15rtkw== X-Received: by 2002:a17:906:5618:: with SMTP id f24mr15226911ejq.414.1617952646960; Fri, 09 Apr 2021 00:17:26 -0700 (PDT) Received: from abel.fritz.box ([2a02:908:1252:fb60:216f:9340:7a2b:4f98]) by smtp.gmail.com with ESMTPSA id w1sm905369edt.89.2021.04.09.00.17.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Apr 2021 00:17:26 -0700 (PDT) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: ray.huang@amd.com, daniel@ffwll.ch, akpm@linux-foundation.org Subject: [PATCH 1/2] mm/vmscan: add sync_shrinkers function Date: Fri, 9 Apr 2021 09:17:24 +0200 Message-Id: <20210409071725.1532-1-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Stat-Signature: nbrd87m734d5o8gqz45sm45s3tzwcj5r X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: CBB17A0003A0 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=mail-ej1-f52.google.com; client-ip=209.85.218.52 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617952648-268764 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To be able to switch to a spinlock and reduce lock contention in the TTM shrinker we don't want to hold a mutex while unmapping and freeing pages from the pool. But then we somehow need to prevent a race between (for example) the shrinker trying to free pages and hotplug trying to remove the device which those pages belong to. Taking and releasing the shrinker semaphore on the write side after unmapping and freeing all pages should make sure that no shrinker is running in paralell any more. Signed-off-by: Christian König --- include/linux/shrinker.h | 1 + mm/vmscan.c | 10 ++++++++++ 2 files changed, 11 insertions(+) diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 0f80123650e2..6b75dc372fce 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -92,4 +92,5 @@ extern void register_shrinker_prepared(struct shrinker *shrinker); extern int register_shrinker(struct shrinker *shrinker); extern void unregister_shrinker(struct shrinker *shrinker); extern void free_prealloced_shrinker(struct shrinker *shrinker); +extern void sync_shrinkers(void); #endif diff --git a/mm/vmscan.c b/mm/vmscan.c index 562e87cbd7a1..46cd9c215d73 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -408,6 +408,16 @@ void unregister_shrinker(struct shrinker *shrinker) } EXPORT_SYMBOL(unregister_shrinker); +/** + * sync_shrinker - Wait for all running shrinkers to complete. + */ +void sync_shrinkers(void) +{ + down_write(&shrinker_rwsem); + up_write(&shrinker_rwsem); +} +EXPORT_SYMBOL(sync_shrinkers); + #define SHRINK_BATCH 128 static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, From patchwork Fri Apr 9 07:17:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 12193203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 327A1C433ED for ; Fri, 9 Apr 2021 07:17:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ABD1B61179 for ; Fri, 9 Apr 2021 07:17:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABD1B61179 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BBA0E6B006C; Fri, 9 Apr 2021 03:17:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B6A176B006E; Fri, 9 Apr 2021 03:17:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0D456B0070; Fri, 9 Apr 2021 03:17:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id 858AE6B006C for ; Fri, 9 Apr 2021 03:17:29 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 42D3918333BF7 for ; Fri, 9 Apr 2021 07:17:29 +0000 (UTC) X-FDA: 78011973018.12.85914C0 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by imf26.hostedemail.com (Postfix) with ESMTP id EF5BA40002D8 for ; Fri, 9 Apr 2021 07:17:25 +0000 (UTC) Received: by mail-ed1-f49.google.com with SMTP id m3so5342812edv.5 for ; Fri, 09 Apr 2021 00:17:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gbjvidtHezwGj4SvL2Dc9DE4DP5+DorOADCoNSePk0o=; b=r5imc+noURb/sXKErdXVdbUdGts2IWxjfj5so6adBwjlq4IAcXJeNbVW/IydyIy8OE /ZvbzxhoSjZIJV8g3P3QepUFZSpKDDI+DbQJb2QHBnYlNkP6yByWo0arkonzdJRLYO6w juj+aswm7fZYfMB9yeZooUD2jpvciU2nweFoe3A2EeoCHr52pdDRQMTK1XzGU5Ntj+/e 3gApRInc3hLv+qTx0pwI91QTLvAJ7RJYti7IJSVTrBM09paQbZwEKVSDb+s4dA0fqkRg 1oUeeSfJJzvJQ3eLQwuWZFgsq895QmaCLfL080HYqTo8gZBp3teFwrPqQz55ckn8ZuEA riEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gbjvidtHezwGj4SvL2Dc9DE4DP5+DorOADCoNSePk0o=; b=YZ/ykXUaNNoGJYUxUM/nXZwt6QqTXdoD5U+MBFAgOs+nob0KHku/Nv1MmKdvzmDjCc vUUc4GIrpaaQAeoQ10f9g3H0f8MwHq7zxTkFS+y3fUuyDZGGBZu19Y9YyR3dsqTW4LJF ONom1TDTFN0p120VL/nncxTIF50RHLKVsNSNl5h5OL4XCJexlgKUw+jS5/P7UT45unyM i8yWAISZOEms30UNa06241KfG8ZxkPhUD7egn/3slwT2iFeYzzB3J7MYsIgb4YsmBwba y3ViI3SsoVgojHG4L8yaCvb6YkZtVtnsHKWLfKmUonvxP4sJt94Fh8wjsBGHRYP3Qz5D 96+w== X-Gm-Message-State: AOAM531BhGggzYjQa/3/NqFM8xnzpFRfWCJJPPS3eBkym9iLysM3QIXs sTSgWc7kmiuhk0dGZ5fhdGo= X-Google-Smtp-Source: ABdhPJwY0JBCnC6/OSxNGZjnksFU03VbvOIrPRv9gylJl43S3ie4aCAulRekCGDv/19tTtCYZA7oFg== X-Received: by 2002:a05:6402:2786:: with SMTP id b6mr16140557ede.310.1617952647643; Fri, 09 Apr 2021 00:17:27 -0700 (PDT) Received: from abel.fritz.box ([2a02:908:1252:fb60:216f:9340:7a2b:4f98]) by smtp.gmail.com with ESMTPSA id w1sm905369edt.89.2021.04.09.00.17.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Apr 2021 00:17:27 -0700 (PDT) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: ray.huang@amd.com, daniel@ffwll.ch, akpm@linux-foundation.org Subject: [PATCH 2/2] drm/ttm: optimize the pool shrinker a bit Date: Fri, 9 Apr 2021 09:17:25 +0200 Message-Id: <20210409071725.1532-2-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210409071725.1532-1-christian.koenig@amd.com> References: <20210409071725.1532-1-christian.koenig@amd.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: EF5BA40002D8 X-Stat-Signature: jbq3k5nbhwgnkt8h45riehsh3rf7dtb7 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail-ed1-f49.google.com; client-ip=209.85.208.49 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617952645-99535 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Switch back to using a spinlock again by moving the IOMMU unmap outside of the locked region. Signed-off-by: Christian König --- drivers/gpu/drm/ttm/ttm_pool.c | 40 +++++++++++++++------------------- 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index cb38b1a17b09..a8b4abe687ce 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -70,7 +70,7 @@ static struct ttm_pool_type global_uncached[MAX_ORDER]; static struct ttm_pool_type global_dma32_write_combined[MAX_ORDER]; static struct ttm_pool_type global_dma32_uncached[MAX_ORDER]; -static struct mutex shrinker_lock; +static spinlock_t shrinker_lock; static struct list_head shrinker_list; static struct shrinker mm_shrinker; @@ -263,9 +263,9 @@ static void ttm_pool_type_init(struct ttm_pool_type *pt, struct ttm_pool *pool, spin_lock_init(&pt->lock); INIT_LIST_HEAD(&pt->pages); - mutex_lock(&shrinker_lock); + spin_lock(&shrinker_lock); list_add_tail(&pt->shrinker_list, &shrinker_list); - mutex_unlock(&shrinker_lock); + spin_unlock(&shrinker_lock); } /* Remove a pool_type from the global shrinker list and free all pages */ @@ -273,9 +273,9 @@ static void ttm_pool_type_fini(struct ttm_pool_type *pt) { struct page *p; - mutex_lock(&shrinker_lock); + spin_lock(&shrinker_lock); list_del(&pt->shrinker_list); - mutex_unlock(&shrinker_lock); + spin_unlock(&shrinker_lock); while ((p = ttm_pool_type_take(pt))) ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); @@ -313,24 +313,19 @@ static struct ttm_pool_type *ttm_pool_select_type(struct ttm_pool *pool, static unsigned int ttm_pool_shrink(void) { struct ttm_pool_type *pt; - unsigned int num_freed; struct page *p; - mutex_lock(&shrinker_lock); + spin_lock(&shrinker_lock); pt = list_first_entry(&shrinker_list, typeof(*pt), shrinker_list); + list_move_tail(&pt->shrinker_list, &shrinker_list); + spin_unlock(&shrinker_lock); p = ttm_pool_type_take(pt); - if (p) { - ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); - num_freed = 1 << pt->order; - } else { - num_freed = 0; - } - - list_move_tail(&pt->shrinker_list, &shrinker_list); - mutex_unlock(&shrinker_lock); + if (!p) + return 0; - return num_freed; + ttm_pool_free_page(pt->pool, pt->caching, pt->order, p); + return 1 << pt->order; } /* Return the allocation order based for a page */ @@ -530,6 +525,7 @@ void ttm_pool_fini(struct ttm_pool *pool) for (j = 0; j < MAX_ORDER; ++j) ttm_pool_type_fini(&pool->caching[i].orders[j]); } + sync_shrinkers(); } /* As long as pages are available make sure to release at least one */ @@ -604,7 +600,7 @@ static int ttm_pool_debugfs_globals_show(struct seq_file *m, void *data) { ttm_pool_debugfs_header(m); - mutex_lock(&shrinker_lock); + spin_lock(&shrinker_lock); seq_puts(m, "wc\t:"); ttm_pool_debugfs_orders(global_write_combined, m); seq_puts(m, "uc\t:"); @@ -613,7 +609,7 @@ static int ttm_pool_debugfs_globals_show(struct seq_file *m, void *data) ttm_pool_debugfs_orders(global_dma32_write_combined, m); seq_puts(m, "uc 32\t:"); ttm_pool_debugfs_orders(global_dma32_uncached, m); - mutex_unlock(&shrinker_lock); + spin_unlock(&shrinker_lock); ttm_pool_debugfs_footer(m); @@ -640,7 +636,7 @@ int ttm_pool_debugfs(struct ttm_pool *pool, struct seq_file *m) ttm_pool_debugfs_header(m); - mutex_lock(&shrinker_lock); + spin_lock(&shrinker_lock); for (i = 0; i < TTM_NUM_CACHING_TYPES; ++i) { seq_puts(m, "DMA "); switch (i) { @@ -656,7 +652,7 @@ int ttm_pool_debugfs(struct ttm_pool *pool, struct seq_file *m) } ttm_pool_debugfs_orders(pool->caching[i].orders, m); } - mutex_unlock(&shrinker_lock); + spin_unlock(&shrinker_lock); ttm_pool_debugfs_footer(m); return 0; @@ -693,7 +689,7 @@ int ttm_pool_mgr_init(unsigned long num_pages) if (!page_pool_size) page_pool_size = num_pages; - mutex_init(&shrinker_lock); + spin_lock_init(&shrinker_lock); INIT_LIST_HEAD(&shrinker_list); for (i = 0; i < MAX_ORDER; ++i) {