From patchwork Thu Nov 22 19:54:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694775 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 25DA214BD for ; Thu, 22 Nov 2018 19:56:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17C4B2B010 for ; Thu, 22 Nov 2018 19:56:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0BB282B136; Thu, 22 Nov 2018 19:56:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 583282B010 for ; Thu, 22 Nov 2018 19:56:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66E856B2CF4; Thu, 22 Nov 2018 14:56:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 61F126B2CF5; Thu, 22 Nov 2018 14:56:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5360C6B2CF6; Thu, 22 Nov 2018 14:56:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 145346B2CF4 for ; Thu, 22 Nov 2018 14:56:12 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id g188so3004749pgc.22 for ; Thu, 22 Nov 2018 11:56:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=UijWGe/GnKfoHQDoEehyKyKkU+M7UHhoyt00RkEYD64=; b=BtuflwsdJ0GOjpOYGZtzzE84RpCNRQMRoc5GRCzkeVj2clToSdI6vkzA57D9+11d77 7H3oW8NNivUugz/vUp2zzVmehhpYGXDyXcz0Qs7FvyaiQ3XGX9FxtFzlcn20rmpdcXWu 0XBAXpdxSO2i5gVWX/hI5gTGKZfKY7qCVfeK2GEEX3CJu1z6G/KGV/ezkYcHUfGwJzlz g9LthXOCmqJvpKCbxWpVBtr8THf1lWXXE7PgdQq/pO3lHyCutsajwnH4vQFNA5+VMc02 Xv81JK3qDD4tuP3XEwqnun6LQKKiHp6sbDaSEjz2/J940ZIcm71tJoJbPjV3MyvjUQV7 H0Iw== X-Gm-Message-State: AA+aEWYwQJEy8iobLsuWdDv/02Kmsm/8rvlYtte7jG+3pzLql9119DYM ShUo5M4E7Ulv7U5QuvZhGypUbtRx+Rv8gTMDrGnODqbedjKHRMJVmH24GHZ4EH3MzTci5T9twRT DRUimvcO+hBqOdZz7vsp0znaqQTnwR2hcmCHwTL3d9QDx+pbaO3P/mVnv7UHRiIAZKg== X-Received: by 2002:a63:4101:: with SMTP id o1mr11108089pga.447.1542916571726; Thu, 22 Nov 2018 11:56:11 -0800 (PST) X-Google-Smtp-Source: AFSGD/VflqIxIk0banC7un6ljEqROf364HCn4CJo7/6gXzeJsAqGdez0z0LiReCfuniGHw5vrN3Q X-Received: by 2002:a63:4101:: with SMTP id o1mr11108049pga.447.1542916570867; Thu, 22 Nov 2018 11:56:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916570; cv=none; d=google.com; s=arc-20160816; b=t+B1M7TgmqifFjLbQLKwGd7tDufsofsrejTOAvdteOHtTHKbzhYqtj/Y0+FWQmK5hN NBUr+ijfJRI5MnfeKv2FJr3VCYworjZooMfxfJ8GXuWZQ7wzoq5Fhm9dGSD3nnz7kO+V YMEWa4inssl+ecPMPf1O5jGn0xcsxhJ2ZNffBnGetFU7QHLucu/KLqMh6GmASwrTO6Si iPMWJYItzbKCQTj/6ZYvA9cn5xcE4vo/Z/gTQqR3AzppvQiUuTD5P4Z2B5vZXAPlNGsj o1LMG//ugxJlwXi8Sp69xO/d9IcXp7hHu4ZQhQQFBhDnaBS1iQ/N9qKZc+Q7YMH3LmjD mHcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UijWGe/GnKfoHQDoEehyKyKkU+M7UHhoyt00RkEYD64=; b=naSotb08SBRplpHY6dvz8hUmBTt/vQskRCqiNPEpN2uVpOmr71KI55lwopTWdZFu9R IMsZ/MbZCbFEUafMZs61jEIpVc4hAvTqqWktbv21NSkHMPEPFgvwBPn53xttcidd5Jw4 2qRAYKvbjRkZxV2sbTEDv8mp5ngJRbwt44yKbYESdS2ovxhSLgYOVC3TY5wGclItfJM/ 6i/14RZ/CBTLsP4ZCPs3iwHPyqBu9sMXCsFbsiTj7Y1Sk08CIq9M6fRHJRuhJLxQUhvV BjSX9o0iA2+0v3b7l2NVUmqHaQ4rw64tJc+yLB+t81kfoTFElxePpAleuRcnUb/+nFuk QvYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=QSXAzck7; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id t3si32217804pgl.108.2018.11.22.11.56.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:56:10 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=QSXAzck7; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0376A20864; Thu, 22 Nov 2018 19:56:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916570; bh=XGApC75tsE0a/HLhg1A1seQEfABEBPrukrY9pXv8FnQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QSXAzck7SCNmk0Ba/jkQZsrCvJDwaTZDxYouqqrW/d5eZQTn8O3/TERXw7bCXnxLG 7hSaAmHEg4Wi4An7ej+nUG9HVmjNFLA3s+/rJ78nh/VdLwThu/x/bicRmu9WNdmGFY iDzEILqHqy/we4Q7U3lc0hrsVBvIovXnyHqGZCVQ= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Vitaly Wool , Vitaly Wool , Jongseok Kim , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.14 19/21] z3fold: fix possible reclaim races Date: Thu, 22 Nov 2018 14:54:50 -0500 Message-Id: <20181122195452.13520-19-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195452.13520-1-sashal@kernel.org> References: <20181122195452.13520-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Vitaly Wool [ Upstream commit ca0246bb97c23da9d267c2107c07fb77e38205c9 ] Reclaim and free can race on an object which is basically fine but in order for reclaim to be able to map "freed" object we need to encode object length in the handle. handle_to_chunks() is then introduced to extract object length from a handle and use it during mapping. Moreover, to avoid racing on a z3fold "headless" page release, we should not try to free that page in z3fold_free() if the reclaim bit is set. Also, in the unlikely case of trying to reclaim a page being freed, we should not proceed with that page. While at it, fix the page accounting in reclaim function. This patch supersedes "[PATCH] z3fold: fix reclaim lock-ups". Link: http://lkml.kernel.org/r/20181105162225.74e8837d03583a9b707cf559@gmail.com Signed-off-by: Vitaly Wool Signed-off-by: Jongseok Kim Reported-by-by: Jongseok Kim Reviewed-by: Snild Dolkow Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/z3fold.c | 101 ++++++++++++++++++++++++++++++++-------------------- 1 file changed, 62 insertions(+), 39 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index f33403d718ac..2813cdfa46b9 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -99,6 +99,7 @@ struct z3fold_header { #define NCHUNKS ((PAGE_SIZE - ZHDR_SIZE_ALIGNED) >> CHUNK_SHIFT) #define BUDDY_MASK (0x3) +#define BUDDY_SHIFT 2 /** * struct z3fold_pool - stores metadata for each z3fold pool @@ -145,7 +146,7 @@ enum z3fold_page_flags { MIDDLE_CHUNK_MAPPED, NEEDS_COMPACTING, PAGE_STALE, - UNDER_RECLAIM + PAGE_CLAIMED, /* by either reclaim or free */ }; /***************** @@ -174,7 +175,7 @@ static struct z3fold_header *init_z3fold_page(struct page *page, clear_bit(MIDDLE_CHUNK_MAPPED, &page->private); clear_bit(NEEDS_COMPACTING, &page->private); clear_bit(PAGE_STALE, &page->private); - clear_bit(UNDER_RECLAIM, &page->private); + clear_bit(PAGE_CLAIMED, &page->private); spin_lock_init(&zhdr->page_lock); kref_init(&zhdr->refcount); @@ -223,8 +224,11 @@ static unsigned long encode_handle(struct z3fold_header *zhdr, enum buddy bud) unsigned long handle; handle = (unsigned long)zhdr; - if (bud != HEADLESS) - handle += (bud + zhdr->first_num) & BUDDY_MASK; + if (bud != HEADLESS) { + handle |= (bud + zhdr->first_num) & BUDDY_MASK; + if (bud == LAST) + handle |= (zhdr->last_chunks << BUDDY_SHIFT); + } return handle; } @@ -234,6 +238,12 @@ static struct z3fold_header *handle_to_z3fold_header(unsigned long handle) return (struct z3fold_header *)(handle & PAGE_MASK); } +/* only for LAST bud, returns zero otherwise */ +static unsigned short handle_to_chunks(unsigned long handle) +{ + return (handle & ~PAGE_MASK) >> BUDDY_SHIFT; +} + /* * (handle & BUDDY_MASK) < zhdr->first_num is possible in encode_handle * but that doesn't matter. because the masking will result in the @@ -717,37 +727,39 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) page = virt_to_page(zhdr); if (test_bit(PAGE_HEADLESS, &page->private)) { - /* HEADLESS page stored */ - bud = HEADLESS; - } else { - z3fold_page_lock(zhdr); - bud = handle_to_buddy(handle); - - switch (bud) { - case FIRST: - zhdr->first_chunks = 0; - break; - case MIDDLE: - zhdr->middle_chunks = 0; - zhdr->start_middle = 0; - break; - case LAST: - zhdr->last_chunks = 0; - break; - default: - pr_err("%s: unknown bud %d\n", __func__, bud); - WARN_ON(1); - z3fold_page_unlock(zhdr); - return; + /* if a headless page is under reclaim, just leave. + * NB: we use test_and_set_bit for a reason: if the bit + * has not been set before, we release this page + * immediately so we don't care about its value any more. + */ + if (!test_and_set_bit(PAGE_CLAIMED, &page->private)) { + spin_lock(&pool->lock); + list_del(&page->lru); + spin_unlock(&pool->lock); + free_z3fold_page(page); + atomic64_dec(&pool->pages_nr); } + return; } - if (bud == HEADLESS) { - spin_lock(&pool->lock); - list_del(&page->lru); - spin_unlock(&pool->lock); - free_z3fold_page(page); - atomic64_dec(&pool->pages_nr); + /* Non-headless case */ + z3fold_page_lock(zhdr); + bud = handle_to_buddy(handle); + + switch (bud) { + case FIRST: + zhdr->first_chunks = 0; + break; + case MIDDLE: + zhdr->middle_chunks = 0; + break; + case LAST: + zhdr->last_chunks = 0; + break; + default: + pr_err("%s: unknown bud %d\n", __func__, bud); + WARN_ON(1); + z3fold_page_unlock(zhdr); return; } @@ -755,7 +767,7 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) atomic64_dec(&pool->pages_nr); return; } - if (test_bit(UNDER_RECLAIM, &page->private)) { + if (test_bit(PAGE_CLAIMED, &page->private)) { z3fold_page_unlock(zhdr); return; } @@ -833,20 +845,30 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) } list_for_each_prev(pos, &pool->lru) { page = list_entry(pos, struct page, lru); + + /* this bit could have been set by free, in which case + * we pass over to the next page in the pool. + */ + if (test_and_set_bit(PAGE_CLAIMED, &page->private)) + continue; + + zhdr = page_address(page); if (test_bit(PAGE_HEADLESS, &page->private)) - /* candidate found */ break; - zhdr = page_address(page); - if (!z3fold_page_trylock(zhdr)) + if (!z3fold_page_trylock(zhdr)) { + zhdr = NULL; continue; /* can't evict at this point */ + } kref_get(&zhdr->refcount); list_del_init(&zhdr->buddy); zhdr->cpu = -1; - set_bit(UNDER_RECLAIM, &page->private); break; } + if (!zhdr) + break; + list_del_init(&page->lru); spin_unlock(&pool->lock); @@ -895,6 +917,7 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) if (test_bit(PAGE_HEADLESS, &page->private)) { if (ret == 0) { free_z3fold_page(page); + atomic64_dec(&pool->pages_nr); return 0; } spin_lock(&pool->lock); @@ -902,7 +925,7 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) spin_unlock(&pool->lock); } else { z3fold_page_lock(zhdr); - clear_bit(UNDER_RECLAIM, &page->private); + clear_bit(PAGE_CLAIMED, &page->private); if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) { atomic64_dec(&pool->pages_nr); @@ -961,7 +984,7 @@ static void *z3fold_map(struct z3fold_pool *pool, unsigned long handle) set_bit(MIDDLE_CHUNK_MAPPED, &page->private); break; case LAST: - addr += PAGE_SIZE - (zhdr->last_chunks << CHUNK_SHIFT); + addr += PAGE_SIZE - (handle_to_chunks(handle) << CHUNK_SHIFT); break; default: pr_err("unknown buddy id %d\n", buddy); From patchwork Thu Nov 22 19:54:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694777 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A2311709 for ; Thu, 22 Nov 2018 19:56:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D6AB2B010 for ; Thu, 22 Nov 2018 19:56:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 418E92B136; Thu, 22 Nov 2018 19:56:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C02E12B010 for ; Thu, 22 Nov 2018 19:56:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E0766B2CF6; Thu, 22 Nov 2018 14:56:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8912E6B2CF7; Thu, 22 Nov 2018 14:56:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 757D76B2CF8; Thu, 22 Nov 2018 14:56:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 370376B2CF6 for ; Thu, 22 Nov 2018 14:56:15 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id 4so14572653plc.5 for ; Thu, 22 Nov 2018 11:56:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=JneqEw+CGPE7jVMbc6OB15LiQoIX8Fiy0lae51kex8I=; b=i6CUubwVXn9wd4j9IzH8iT1fxZ5Z5gbwcFtAO4eqbCbTzqDSNfXbZCdMDQE4aHyrFT HslTbjAfM20cuwvI66FIDzqPImcMTRtRBMCcbOtW3WFNXe+RlffUAUybMqeQbBCMA3oZ EX/MZXoCUXw9MFDMzsy+90NcjBIWovC0wSbXKkoQsWAlQS15pA+HkR7jhaaQjslTrVUh j/uPJLA0T/qJcu7J0E/ZSuqOmIW/Iu4qOQV6h1AX5Y8QzXvHNR5j1YxYKYI3BfZ/qJbx 8Ij2qdbZQQE67StGkGoRg8k8JrvRRw4Og6HRxeHaDP3DiVRwc+fqT6xNKjPTG4vl1mkn afDQ== X-Gm-Message-State: AA+aEWa+7qJAG33Hmlhd0pZCysW83w0gwDZf5NhG59xHvVnUgl5dXvcI MmhoFsrZtQwrwopTP2wWcnQOfQMruRT3j/4A4gxJJyRZ6hCUvWU3zMPzv53UUrQkxpE4RaHgsgw tnLzSmkuYkoaUuR8LHnGZokMkm5bmYRsiL24p7Z8n0WAG7wFpHmg7QBzm049TGWBRtA== X-Received: by 2002:a63:fc05:: with SMTP id j5mr11433202pgi.434.1542916574846; Thu, 22 Nov 2018 11:56:14 -0800 (PST) X-Google-Smtp-Source: AFSGD/UERgqbq66ftwaKC6ZacSBRCYGjYrLG1YDKYZxOcdc/jEaSWR3l//IlocEu+bfQvyC1scvQ X-Received: by 2002:a63:fc05:: with SMTP id j5mr11433182pgi.434.1542916574296; Thu, 22 Nov 2018 11:56:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916574; cv=none; d=google.com; s=arc-20160816; b=Bh9P0i6AlSf3dnxMbdiY3LzK/+Q2znozjuTzHctMujfnOChdvklzyxYbb99dGYEG35 trvq1r0L8ochZgmMCWVikfU2Eurx4NX87jzYv+UsjHcbZTLEmSW2zWQ2eFF/P2WZ3T3z lr3kqKXIrTxRKHWwPP0XRzLbiWTyEpUrxDw+zXmg9w9Xrv6MGIfIKD4WvxrRieBRxfAb gLyhEp7RL9d9Mym7avIorIoeyOoIxVhyuo5Jp+pLAp1JB0gCyEKzYPI+Rf+wcw9q7TQQ lEli25hU1VUM4q7LXADbvp55oaDHez2L0YCFnC45r1cKwPACgLhZQFNKmkggslnjYnMd g0eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JneqEw+CGPE7jVMbc6OB15LiQoIX8Fiy0lae51kex8I=; b=CG8yONC//A/TsQM38Ic04vihK0dfyEQxirQgvAEeGg19FuD8gt0LK68LIu0oOz9da4 6w65/GT9HeWdR5nGXWd0FMXD6GBGs41JXSbnuJrumPp2PZjtm7xArUQG5oKQfjFGJcgd /XpLDf0RWS99bspObiQU671fDV1ZMQt1qkES4KtHePknVfjOrLgMNW4dxp7idduL70c6 w08PHvMWenz7pDJgrKmHRPzTgpMrWiJXAzO60ApFQSV1kz6/lcI+bGYZCX4qILtIckuX hJUCcvsjicirNT9ILko8ookKmR3/IQbDSRHfbi43fx7xADEh9Mb5DgPQ8SFEsHEd0bnK z2dQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=sUYDgL1E; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id z10si37404161pfm.37.2018.11.22.11.56.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:56:14 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=sUYDgL1E; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 390B120672; Thu, 22 Nov 2018 19:56:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916574; bh=S7vU8rSHrCZb3a0fzkTucu2Jn6JD/fp3nZZwX1T/5sE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sUYDgL1Ekh56fMazDOnErhYvy5hwvruEbEEuy7UZ53V0KLLelPUz8TeVJGgk5WQwn Sd41JXowZFhhrkJ0xEJtx37Qw5y0NU9dvT3ZfGLZD7t/D87LL9zjOzCJ111g0ykAoe Sdb/evuoj4Q0ppEa57o+BFyaWgl8dPdrdIQRq6V4= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Yufen Yu , Al Viro , Hugh Dickins , William Kucharski , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.14 20/21] tmpfs: make lseek(SEEK_DATA/SEK_HOLE) return ENXIO with a negative offset Date: Thu, 22 Nov 2018 14:54:51 -0500 Message-Id: <20181122195452.13520-20-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195452.13520-1-sashal@kernel.org> References: <20181122195452.13520-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Yufen Yu [ Upstream commit 1a413646931cb14442065cfc17561e50f5b5bb44 ] Other filesystems such as ext4, f2fs and ubifs all return ENXIO when lseek (SEEK_DATA or SEEK_HOLE) requests a negative offset. man 2 lseek says : EINVAL whence is not valid. Or: the resulting file offset would be : negative, or beyond the end of a seekable device. : : ENXIO whence is SEEK_DATA or SEEK_HOLE, and the file offset is beyond : the end of the file. Make tmpfs return ENXIO under these circumstances as well. After this, tmpfs also passes xfstests's generic/448. [akpm@linux-foundation.org: rewrite changelog] Link: http://lkml.kernel.org/r/1540434176-14349-1-git-send-email-yuyufen@huawei.com Signed-off-by: Yufen Yu Reviewed-by: Andrew Morton Cc: Al Viro Cc: Hugh Dickins Cc: William Kucharski Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/shmem.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index ea786a504e1b..fa08f56fd5e5 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2590,9 +2590,7 @@ static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) inode_lock(inode); /* We're holding i_mutex so we can access i_size directly */ - if (offset < 0) - offset = -EINVAL; - else if (offset >= inode->i_size) + if (offset < 0 || offset >= inode->i_size) offset = -ENXIO; else { start = offset >> PAGE_SHIFT; From patchwork Thu Nov 22 19:54:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 10694779 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 515CF1709 for ; Thu, 22 Nov 2018 19:56:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F7F62B010 for ; Thu, 22 Nov 2018 19:56:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2FBC02B136; Thu, 22 Nov 2018 19:56:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79EAB2B010 for ; Thu, 22 Nov 2018 19:56:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75A4C6B2CF8; Thu, 22 Nov 2018 14:56:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 709566B2CF9; Thu, 22 Nov 2018 14:56:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F9E16B2CFA; Thu, 22 Nov 2018 14:56:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by kanga.kvack.org (Postfix) with ESMTP id 1E6096B2CF8 for ; Thu, 22 Nov 2018 14:56:22 -0500 (EST) Received: by mail-pg1-f200.google.com with SMTP id l131so3057426pga.2 for ; Thu, 22 Nov 2018 11:56:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=HJQM6B1JeeKuxQG1shlJz1zUHOF3lDSR6SD7TN4qYbg=; b=M+G3wnyRQxecOJpd6R/bG0goWwj7UGGJID73+4DakESC+FetERm7p1CfQ4Lf88ot6F IG5ehipmrvKuHHWizbDjo0xlFNofuTzOxIooFOo3qZEVzOZyB5Bnms3QDxgWbNRBfqwq tjist4YabXh5Yq4dQdNZG3+NAQDT3vLKgCsUTddGxCk/AauUiANog9JQMohP0BQjxFli tGpcd0Upo/r173ekFD6kOSc+etwl5jg0/gguIzlSiXBPowgOxqJiIJFaGcAwBqhIMf5f 1Y9QDq1XtsG1fCYZv3+diw9tS4p1NG4Itio3V/Nm/4voDYOOjFGfPNtkEw5IjU6quEGh m0PQ== X-Gm-Message-State: AA+aEWYWNvWDjU4wkz8kTU686sUZKokSdWakRQhGML8KBsnKNpMA5cJ7 xGe033zVNE1DIhPTe2VAS1xZTZKEFQNkE4sYN//rO6wyGgqdW9VrxHKbSHXXAfnWagVGsh45oLF 1BnYg3FIowDmQ+lCBJHrXJqVyTxEjpaJaeWUpDPMVI/0iUiW/G5L6PMBzRUoS2+KJiQ== X-Received: by 2002:a17:902:43e4:: with SMTP id j91mr12202572pld.147.1542916581781; Thu, 22 Nov 2018 11:56:21 -0800 (PST) X-Google-Smtp-Source: AFSGD/UVqis0q1CyxvLADeTXzCC9ta1j04b0O4kfx/XQOzaSfHy0m9GyX60ylOBE4AU25cw9eeYK X-Received: by 2002:a17:902:43e4:: with SMTP id j91mr12202545pld.147.1542916581091; Thu, 22 Nov 2018 11:56:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542916581; cv=none; d=google.com; s=arc-20160816; b=VUQqPbNYKPD5297g3BVmwTx5TQAkEfdzogM2BRgkMSAjY+1o+sPiwSlkf9cKPinVIN utj1ZuStXIlfHpL6qE7X/V73DrEjxiEnZ95HN+DzSu20J6NdkZKorCf5gGAr9/ilXmdk /Eosd/5Mu7Ho4KDBzvePDzH+crqu6SsbfnMA8GXD3gagb4NcW0k/J1Q0ccTbFKuj0RXQ AflbjxUUFA3QUqcOdbRDzqTJNcJOpP3y2M36qQ6ZSG1QF5govTp+i7GKrPkETgFKPNY+ pSD40oRLBTFtme+r1CsAYPNW44BTbH7VVyLqWCS3fd90TSE8xbHJJ8YrUommaemWknfW mhhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HJQM6B1JeeKuxQG1shlJz1zUHOF3lDSR6SD7TN4qYbg=; b=eWoomqDGViCQk3us+iMMVSF73UrvNcNgSCpq6vJNIK7x+SDT18cjc4w+H0pJwW3f3v EBOnQFod6yr4YQSXQh9/0PkqHNMfj4klH+KA/1yeo/42Sm0mY8aSR8xMMu4MD3D4ONmZ Iw9O+xDZWbI6tTAkY/EN2ZiIhhTcTcuxUQpRGX+KPQ9Dud5HS29G9Z7y3cPm3aglgqSv F5rgocJ5FxduYLSPI+T8Qy8j6MK/lkQO4s9zapgKoV1mj/4yS3t7VNGGK+uxajgYPX6U rGklWXen0Qtj47Lm8+NCktFYcIc9ujur+T67CQiL1izuWAPCqf4kzfHgLnwM6trtwYl2 saOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=zCSP0z9j; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail.kernel.org (mail.kernel.org. [198.145.29.99]) by mx.google.com with ESMTPS id f61si13275929plb.51.2018.11.22.11.56.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 11:56:21 -0800 (PST) Received-SPF: pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) client-ip=198.145.29.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=zCSP0z9j; spf=pass (google.com: domain of sashal@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=sashal@kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sasha-vm.mshome.net (unknown [37.142.5.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 937E320864; Thu, 22 Nov 2018 19:56:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1542916580; bh=sJFYhQVjF2sT8uCmoRl4SvhCC5gDGR0+SXDcj4wN0eU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zCSP0z9j3E0W9ASuOAWGqF2sNF54i0qoC6KOowReJDAEzb9ZY+p9G3OsTRNHTQdOO 1nHV54W22o3adXqFEaRBpliRVehnifor+RVnfGVMpbJRRM8eGHWj6O573SR54LBPgj j1UhyGC9FxLSlfKlgof2/QK/1sStNAerDjWBrdAw= From: Sasha Levin To: stable@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Michal Hocko , Balbir Singh , Mel Gorman , Pavel Tatashin , Oscar Salvador , Mike Rapoport , Aaron Lu , Joonsoo Kim , Byoungyoung Lee , "Dae R. Jeong" , Andrew Morton , Linus Torvalds , Sasha Levin , linux-mm@kvack.org Subject: [PATCH AUTOSEL 4.14 21/21] mm, page_alloc: check for max order in hot path Date: Thu, 22 Nov 2018 14:54:52 -0500 Message-Id: <20181122195452.13520-21-sashal@kernel.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181122195452.13520-1-sashal@kernel.org> References: <20181122195452.13520-1-sashal@kernel.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko [ Upstream commit c63ae43ba53bc432b414fd73dd5f4b01fcb1ab43 ] Konstantin has noticed that kvmalloc might trigger the following warning: WARNING: CPU: 0 PID: 6676 at mm/vmstat.c:986 __fragmentation_index+0x54/0x60 [...] Call Trace: fragmentation_index+0x76/0x90 compaction_suitable+0x4f/0xf0 shrink_node+0x295/0x310 node_reclaim+0x205/0x250 get_page_from_freelist+0x649/0xad0 __alloc_pages_nodemask+0x12a/0x2a0 kmalloc_large_node+0x47/0x90 __kmalloc_node+0x22b/0x2e0 kvmalloc_node+0x3e/0x70 xt_alloc_table_info+0x3a/0x80 [x_tables] do_ip6t_set_ctl+0xcd/0x1c0 [ip6_tables] nf_setsockopt+0x44/0x60 SyS_setsockopt+0x6f/0xc0 do_syscall_64+0x67/0x120 entry_SYSCALL_64_after_hwframe+0x3d/0xa2 the problem is that we only check for an out of bound order in the slow path and the node reclaim might happen from the fast path already. This is fixable by making sure that kvmalloc doesn't ever use kmalloc for requests that are larger than KMALLOC_MAX_SIZE but this also shows that the code is rather fragile. A recent UBSAN report just underlines that by the following report UBSAN: Undefined behaviour in mm/page_alloc.c:3117:19 shift exponent 51 is too large for 32-bit type 'int' CPU: 0 PID: 6520 Comm: syz-executor1 Not tainted 4.19.0-rc2 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0xd2/0x148 lib/dump_stack.c:113 ubsan_epilogue+0x12/0x94 lib/ubsan.c:159 __ubsan_handle_shift_out_of_bounds+0x2b6/0x30b lib/ubsan.c:425 __zone_watermark_ok+0x2c7/0x400 mm/page_alloc.c:3117 zone_watermark_fast mm/page_alloc.c:3216 [inline] get_page_from_freelist+0xc49/0x44c0 mm/page_alloc.c:3300 __alloc_pages_nodemask+0x21e/0x640 mm/page_alloc.c:4370 alloc_pages_current+0xcc/0x210 mm/mempolicy.c:2093 alloc_pages include/linux/gfp.h:509 [inline] __get_free_pages+0x12/0x60 mm/page_alloc.c:4414 dma_mem_alloc+0x36/0x50 arch/x86/include/asm/floppy.h:156 raw_cmd_copyin drivers/block/floppy.c:3159 [inline] raw_cmd_ioctl drivers/block/floppy.c:3206 [inline] fd_locked_ioctl+0xa00/0x2c10 drivers/block/floppy.c:3544 fd_ioctl+0x40/0x60 drivers/block/floppy.c:3571 __blkdev_driver_ioctl block/ioctl.c:303 [inline] blkdev_ioctl+0xb3c/0x1a30 block/ioctl.c:601 block_ioctl+0x105/0x150 fs/block_dev.c:1883 vfs_ioctl fs/ioctl.c:46 [inline] do_vfs_ioctl+0x1c0/0x1150 fs/ioctl.c:687 ksys_ioctl+0x9e/0xb0 fs/ioctl.c:702 __do_sys_ioctl fs/ioctl.c:709 [inline] __se_sys_ioctl fs/ioctl.c:707 [inline] __x64_sys_ioctl+0x7e/0xc0 fs/ioctl.c:707 do_syscall_64+0xc4/0x510 arch/x86/entry/common.c:290 entry_SYSCALL_64_after_hwframe+0x49/0xbe Note that this is not a kvmalloc path. It is just that the fast path really depends on having sanitzed order as well. Therefore move the order check to the fast path. Link: http://lkml.kernel.org/r/20181113094305.GM15120@dhcp22.suse.cz Signed-off-by: Michal Hocko Reported-by: Konstantin Khlebnikov Reported-by: Kyungtae Kim Acked-by: Vlastimil Babka Cc: Balbir Singh Cc: Mel Gorman Cc: Pavel Tatashin Cc: Oscar Salvador Cc: Mike Rapoport Cc: Aaron Lu Cc: Joonsoo Kim Cc: Byoungyoung Lee Cc: "Dae R. Jeong" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin --- mm/page_alloc.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a604b5da6755..2074f424dabf 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3867,17 +3867,6 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, unsigned int cpuset_mems_cookie; int reserve_flags; - /* - * In the slowpath, we sanity check order to avoid ever trying to - * reclaim >= MAX_ORDER areas which will never succeed. Callers may - * be using allocators in order of preference for an area that is - * too large. - */ - if (order >= MAX_ORDER) { - WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN)); - return NULL; - } - /* * We also sanity check to catch abuse of atomic reserves being used by * callers that are not in atomic context. @@ -4179,6 +4168,15 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, gfp_t alloc_mask; /* The gfp_t that was actually used for allocation */ struct alloc_context ac = { }; + /* + * There are several places where we assume that the order value is sane + * so bail out early if the request is out of bound. + */ + if (unlikely(order >= MAX_ORDER)) { + WARN_ON_ONCE(!(gfp_mask & __GFP_NOWARN)); + return NULL; + } + gfp_mask &= gfp_allowed_mask; alloc_mask = gfp_mask; if (!prepare_alloc_pages(gfp_mask, order, preferred_nid, nodemask, &ac, &alloc_mask, &alloc_flags))