From patchwork Thu Oct 25 09:28:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Wool X-Patchwork-Id: 10655635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4BFBE109C for ; Thu, 25 Oct 2018 09:28:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A8512AFF8 for ; Thu, 25 Oct 2018 09:28:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2F20C2B006; Thu, 25 Oct 2018 09:28:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C9BD2AFF8 for ; Thu, 25 Oct 2018 09:28:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 756256B0278; Thu, 25 Oct 2018 05:28:27 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6DC356B027A; Thu, 25 Oct 2018 05:28:27 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57F616B027B; Thu, 25 Oct 2018 05:28:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f69.google.com (mail-lf1-f69.google.com [209.85.167.69]) by kanga.kvack.org (Postfix) with ESMTP id DA3E06B0278 for ; Thu, 25 Oct 2018 05:28:26 -0400 (EDT) Received: by mail-lf1-f69.google.com with SMTP id y12-v6so966438lfh.16 for ; Thu, 25 Oct 2018 02:28:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :message-id:mime-version:content-transfer-encoding; bh=RP9WX0lcf21kEhw4xJOYo/rCCLPzKEozfMJGHBeKxJA=; b=oAr3FitGGK0GMmUxhJtQJ2rHbDsr9JXWNoBUP0cU35qFnIa4clRzBBxsayLRQ7xDri 1DF79yaMOsaHRXGKnX+JKhPxwnkqJ+KN0OaKTJ/ue+LijqZje7eFX/mV8F3jdItef7op DRT81uAS1ndWD53l57SPlqDReQObuH7TuZjeTB9vxGMZ3pxbjaOeTA+nQvxCQI+N3Zal AtFy0kaUJooJgGN6z/42p3ekFOkcKSosU/MVKLDvt1wJ6fMjQ72RyrK/wBu3Rnc1iLcv HVm57tSS7G+Ix9f+YkJ6IpMwrGcBtaQV451Iu6+JFW2hMxyhjXPvP2gqM9xtFk+rRzSg wSVw== X-Gm-Message-State: AGRZ1gIgfscUt+X0ZYi6v7o+Gvov1y6GpSAhbgccjo6huw4o48ZFQm+t 0vtmuY1pmU2cx/uMRrcaylIXCojz8lgDaPW4/eTr7vi7JKa7wxXVBRubL2m8HpZEtZFcfyrvhDa T+LR9GBsred+ELWHGor0yOphl+vUPQBT7iT8VApGR+b34yHGYfmnYZhkwr5xJcjSftKrss8YD1m 0w5I0c6Q2+/RpYsVTGiYe9+8SK5lwiY+DZU88FrAm/7qZPrVK4Irt0oLMKZzm426dDaJ+ZGYQu3 eHVLmPYLfx3PsMtxeBadzMjjY5oCWc62Wx3Ki8Vfmm1HV6Q+ZXnzr3CRDa6oUnkCYiRte64pfrn Sbqet1kriZPZ+gnetCyCf0eg4wPW5wxoKV50rE1tOHgOYB7ANQjYxQ2Tei4+mTQXLghHnmUsXCD z X-Received: by 2002:ac2:4159:: with SMTP id c25-v6mr665784lfi.19.1540459705818; Thu, 25 Oct 2018 02:28:25 -0700 (PDT) X-Received: by 2002:ac2:4159:: with SMTP id c25-v6mr665725lfi.19.1540459704570; Thu, 25 Oct 2018 02:28:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540459704; cv=none; d=google.com; s=arc-20160816; b=H+034015hnGz1aSp0F0IeZYOUC3yNzKQZjvVO+qjZyndFHTAxcn7baXa0EUVq9a4N/ nPrekVspvFx7lS95PvQ7iwtHSiEfCHkaDjRf8U8GnJ/K3RWAVgYrqUgoZRy6Bal6bFiL 07gUvchKrSNFmLaxaiRTyik/LR/5hJkOUAQsauwwdQYQzO16/79jzgnynvk6yGO2wiZ8 TFisWUeHj0TU6w7JTkm1SSazqa3uqaunsAn4g8nzoJBtwKjLCrJzFSDEt47uObByOXZ5 f2LW5JWn5bYBfJQMfQ3E1khUiL/SgEeUIoAOOHhg5iZZbcytRBJoiTE3QtXrBKAHwXeo ruVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:subject:cc:to :from:date:dkim-signature; bh=RP9WX0lcf21kEhw4xJOYo/rCCLPzKEozfMJGHBeKxJA=; b=Rkvs09HWmRlXC9s/slWneF+91xzH1NKgSq1fWerK9SrrvCb4skns7CNmQoDJVu5vvc bd6LfCYy3n/TNLYE21jAXZKcOPRFZRY/SeBOZqqURaGh4IdZb6R4Ri5bDsYlPICAc1U9 fa2jJy0hnvvQLS7hvEqnpNJrz1WffudayK3BoESALmQfqyefPFBi32jpwN1BocXM0W5Z YfDxfugByv7lOgM9QdXyS80eQ7LNXvu+XRKM8seRnDL19WUreX7ccD3ucJdWg2siQR+i LjyZ0L9/kaBxZTYzPY/ktBmygwneJX2KQ9pDLIRmox6BdrgXH1JgiStsP4umpgb+0zs2 j7jg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=n0Cow3Eb; spf=pass (google.com: domain of vitalywool@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=vitalywool@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id z12-v6sor4788094ljb.6.2018.10.25.02.28.24 for (Google Transport Security); Thu, 25 Oct 2018 02:28:24 -0700 (PDT) Received-SPF: pass (google.com: domain of vitalywool@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=n0Cow3Eb; spf=pass (google.com: domain of vitalywool@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=vitalywool@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:mime-version :content-transfer-encoding; bh=RP9WX0lcf21kEhw4xJOYo/rCCLPzKEozfMJGHBeKxJA=; b=n0Cow3EbQWr38gbmgHRdD/1pwb3U+WmuFRaj1fhZK+PmLYJQ0C2jUpXpt6zP3PNeqk NGgI/lB+MPIGEzT3uxKv2F8oehgoMvtPL/V7nIp3R1MC7ZQQ/IufpOnuCFM50DHHvPp7 BCN0HTcCcGgJcH58+q9Bszd5ZlHnUcwgY9txWafRwAtcRWI0WtQBADewuvMdVxMyvs9m f2NfMNOgH7wIOgU0EqkXwqrsd4Np1XMpfMn8gCwz0dJBMchHJTrBGz2VP/ypSC660qvE slJ6VN9QZz1vlEIDMx9B+bh2hGz15BbLvsflffwpNSBxgD+dUlT99ISNxUnyeoI37CR+ yBgA== X-Google-Smtp-Source: AJdET5dJftHSGcy/IZOHtRCQ1KVaDTBtMrnpMf6kJYVAThQkGkfbYgMBvCrWg8qi7lwCgJR5b+726A== X-Received: by 2002:a2e:9a93:: with SMTP id p19-v6mr682232lji.87.1540459703591; Thu, 25 Oct 2018 02:28:23 -0700 (PDT) Received: from seldlx21914.corpusers.net ([37.139.156.40]) by smtp.gmail.com with ESMTPSA id n195-v6sm1170285lfb.26.2018.10.25.02.28.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Oct 2018 02:28:22 -0700 (PDT) Date: Thu, 25 Oct 2018 11:28:21 +0200 From: Vitaly Wool To: Linux-MM , linux-kernel@vger.kernel.org Cc: Andrew Morton , Oleksiy.Avramchenko@sony.com, Guenter Roeck Subject: [PATCH] z3fold: encode object length in the handle Message-Id: <20181025112821.0924423fb9ecc7918896ec2b@gmail.com> X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.30; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Reclaim and free can race on an object (which is basically ok) but in order for reclaim to be able to map "freed" object we need to encode object length in the handle. handle_to_chunks() is thus introduced to extract object length from a handle and use it during mapping of the last object we couldn't correctly map before. Signed-off-by: Vitaly Wool --- mm/z3fold.c | 48 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 35 insertions(+), 13 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 4b366d181f35..86359b565d45 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -99,6 +99,7 @@ struct z3fold_header { #define NCHUNKS ((PAGE_SIZE - ZHDR_SIZE_ALIGNED) >> CHUNK_SHIFT) #define BUDDY_MASK (0x3) +#define BUDDY_SHIFT 2 /** * struct z3fold_pool - stores metadata for each z3fold pool @@ -223,8 +224,17 @@ static unsigned long encode_handle(struct z3fold_header *zhdr, enum buddy bud) unsigned long handle; handle = (unsigned long)zhdr; - if (bud != HEADLESS) - handle += (bud + zhdr->first_num) & BUDDY_MASK; + if (bud != HEADLESS) { + unsigned short num_chunks = zhdr->first_chunks; + + if (bud == MIDDLE) + num_chunks = zhdr->middle_chunks; + if (bud == LAST) + num_chunks = zhdr->last_chunks; + + handle |= (bud + zhdr->first_num) & BUDDY_MASK; + handle |= (num_chunks << BUDDY_SHIFT); + } return handle; } @@ -234,6 +244,11 @@ static struct z3fold_header *handle_to_z3fold_header(unsigned long handle) return (struct z3fold_header *)(handle & PAGE_MASK); } +static unsigned short handle_to_chunks(unsigned long handle) +{ + return (handle & ~PAGE_MASK) >> BUDDY_SHIFT; +} + /* * (handle & BUDDY_MASK) < zhdr->first_num is possible in encode_handle * but that doesn't matter. because the masking will result in the @@ -732,7 +747,6 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) break; case MIDDLE: zhdr->middle_chunks = 0; - zhdr->start_middle = 0; break; case LAST: zhdr->last_chunks = 0; @@ -746,11 +760,14 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) } if (bud == HEADLESS) { - spin_lock(&pool->lock); - list_del(&page->lru); - spin_unlock(&pool->lock); - free_z3fold_page(page); - atomic64_dec(&pool->pages_nr); + /* if a headless page is under reclaim, just leave */ + if (!test_bit(UNDER_RECLAIM, &page->private)) { + spin_lock(&pool->lock); + list_del(&page->lru); + spin_unlock(&pool->lock); + free_z3fold_page(page); + atomic64_dec(&pool->pages_nr); + } return; } @@ -836,20 +853,24 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) } list_for_each_prev(pos, &pool->lru) { page = list_entry(pos, struct page, lru); + zhdr = page_address(page); if (test_bit(PAGE_HEADLESS, &page->private)) - /* candidate found */ break; - zhdr = page_address(page); - if (!z3fold_page_trylock(zhdr)) + if (!z3fold_page_trylock(zhdr)) { + zhdr = NULL; continue; /* can't evict at this point */ + } kref_get(&zhdr->refcount); list_del_init(&zhdr->buddy); zhdr->cpu = -1; - set_bit(UNDER_RECLAIM, &page->private); break; } + if (!zhdr) + break; + + set_bit(UNDER_RECLAIM, &page->private); list_del_init(&page->lru); spin_unlock(&pool->lock); @@ -898,6 +919,7 @@ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) if (test_bit(PAGE_HEADLESS, &page->private)) { if (ret == 0) { free_z3fold_page(page); + atomic64_dec(&pool->pages_nr); return 0; } spin_lock(&pool->lock); @@ -964,7 +986,7 @@ static void *z3fold_map(struct z3fold_pool *pool, unsigned long handle) set_bit(MIDDLE_CHUNK_MAPPED, &page->private); break; case LAST: - addr += PAGE_SIZE - (zhdr->last_chunks << CHUNK_SHIFT); + addr += PAGE_SIZE - (handle_to_chunks(handle) << CHUNK_SHIFT); break; default: pr_err("unknown buddy id %d\n", buddy);