From patchwork Wed Oct 16 15:41:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13838600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10A09D2A529 for ; Wed, 16 Oct 2024 15:42:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 929276B008C; Wed, 16 Oct 2024 11:42:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D9976B0092; Wed, 16 Oct 2024 11:42:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77CF16B0093; Wed, 16 Oct 2024 11:42:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4EBF16B008C for ; Wed, 16 Oct 2024 11:42:04 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C4598C05B8 for ; Wed, 16 Oct 2024 15:41:52 +0000 (UTC) X-FDA: 82679880948.08.F324F05 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf27.hostedemail.com (Postfix) with ESMTP id A2D924000E for ; Wed, 16 Oct 2024 15:41:52 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Ub8EXjkf; spf=pass (imf27.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729093248; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g4wtmCSOL20OH1846ssdNZijDKNqe+VpFdtBicgceqw=; b=jN9NqUA9s33g0us4qyj5GTRkm0EARj4CfANzu+LMdMxDtKHuVC1GnzXHmJEoAtgYrNEfrx raIh1qECNrJEPYQxKoc0coI9BGgXbT/3DAXFo0CGanaMYa771T1Grvc4lDS6Vvo10gJGur DUEY19zb6e77isMY9cxHFW+eeFmyfM4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Ub8EXjkf; spf=pass (imf27.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729093248; a=rsa-sha256; cv=none; b=xsPBOk2d3YvOSTFo5fIIz1vwujn3tOO8yi82nQSTnzW7x6t0PxbWicnG/5gp3SW25xOxnx h+23HD8INaM7/m6Ly0waiWJzjqdPH3FYWrNalgwc6NhJoDAmJ7CIzUhVpqxWnVCJSmJAtl cY704aBornik/OO6laXQrTEGK2KXKog= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729093321; x=1760629321; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1yohiX8Mia2wqWhVgM9lhouT0M30SjIN3hnB+Hq2y3E=; b=Ub8EXjkfy0zoiUfR4IL2j6LuO2K/NIcQskBfFi9G71uMIEV8nP2o4kkY 5HpQ9qjyqu6eqtoZvYdptPiVpKvaY90mJUeyQh6N1z79YtcbOGsaHwuQh 8Yl2vWAlFaoNCpqBk30bUJh+NNjhxgD8b2Y+RaKBi0iHK3c9DcePgZzmd 8nmVpArYT+DqEmkeXQVQ8n+mCfL02qI0G+SjIl2AlPRPG0OGgBwGv8K1X M3lGunRt16VLDRbIChIKv+dDt3NSOVcVzUQwDj6/yMsHdjJc/hxZ6FQrz I4EGp8YWH7aVe39taDis7/HvG/MQF3hpe61aWxZXGGohkz89+v1tYV/m+ Q==; X-CSE-ConnectionGUID: CD92HxzGTGKOKl9e2x9ZvQ== X-CSE-MsgGUID: hqJ99KywQWeOCR9wSSCHyw== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="46021350" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="46021350" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Oct 2024 08:42:01 -0700 X-CSE-ConnectionGUID: +cn8A2teRjqp+D+KyviP2Q== X-CSE-MsgGUID: 8UoO9yu/TrqGqxq22dttoQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,208,1725346800"; d="scan'208";a="109018906" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by fmviesa001.fm.intel.com with ESMTP; 16 Oct 2024 08:41:57 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Danilo Krummrich , Narasimhan.V@amd.com Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 1/3] mm/slub: Consider kfence case for get_orig_size() Date: Wed, 16 Oct 2024 23:41:50 +0800 Message-Id: <20241016154152.1376492-2-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241016154152.1376492-1-feng.tang@intel.com> References: <20241016154152.1376492-1-feng.tang@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A2D924000E X-Stat-Signature: 474i11b1tokzdz4skpsxknsewyn1yskn X-HE-Tag: 1729093312-157077 X-HE-Meta: U2FsdGVkX1/MTiwue8o4EY9JGB3dsLGwhG8nQcxKZIo0z9jRYfU6TNP3j+WTPolYBh9exJ4aBDDzKGSO0uJtZqAuaWADQGvzZso9KHXS0wpqWOhvQPJWmYqaY/zb/ac9eRSmM2tqYe5xRLk3BZatVNGvevT6E2f0KWzEggkiYwq6OMOlrx3gzjgdAkwO54O4d1GGOUSvgOqMUCRAq9GaXuvhoCn54Dm4k3BqkPU05NeiD5rEthAlbQEYoTobLSDBbL9ipFZYDYWzpZJU9yEZUOOK+fgOZWVQDB+jpOBKLgM+XyQK15QlgbSULhTzMyv+up80wR2GFRCBl4SNL9XarXOaBIldLyeTK90Kg55yuMzFuTZLOONJ2HQx8dNHdzfY2aF3lNaeW/jgcBTguVXJs2jaWmCK+4FR9IUhghYPgDrfoJsUfdjGA8EZhE1zeIP4oJ5Fq3WVFP7MlwjwpbiGQE9rf2Xb87XCFU6RXHKtx5SyJ3XvL4pnafKuesUv6z2Rns5C0+j8WtgjgzxiZU0ledqEhvvryAy5xCkS9O82CsGUDI0OVKj/SuUPPKEICTXgyMxmH1VOW5bW2SqPkQhIi8FgF69JjaDFFNnhAAs7HCYlksXnyMXnF4fjHhojbaeYs1BYjAI0KPJ0ESEynGZt/ZzW4lLJXw7OrZgDOpULYWBZrcN5bNtgrHDcBqkqOosLvwHVy0YC9y0lXPy5GJPkyr32/dvD/VksW+U+w9gmgiIuzl7Y22mMDr/8sKrrK0CqLM5YcxsQ6iRUScZCveQeyfnzVhSS3QMQgkGJQRXphJarMUfW6tm1qTUGWDsKldXHB9xM0m12sIHZAvr4H1Zq+AFEOUE7kmFJ8pKetyitfj/VJaYhOiyjUcJ1HudRXH1NlldhUe1qLzwMrAmi4taB7eCrdNy5p+lOIM5FFhneEDaqCMGTSD6oYxJM3j2LtUw9/FsHkvIvb8WtJ+kFLFB jTEFU/ag Vz6cSJvD2Zv8XS9a9wvJgyRqqhf5a6MKn/ryG2b9oI6cFxuo8lVKjZGo1ie3bNS/IPduM/8QOyplY+/XIenCmjof8VmP+MozF//N2Q/vfd2AcNNuRVPHjuJ7rHsKX19xZgs45Xz1AoqIYSxLLxnZSt48sfJbcagLpomN4euen9ef0kDCBLWILxXIgRdUYABQkDFvHTMeXqJB0y5BPg/+KSzf3BBetej42pbcdK3HWLVLQCU4JNXMG466Xv5Xc78L/l6kMol71doiVOOR/65rC+n7mZ/+Uedu0k8u5+Zt4fEyeCDo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When 'orig_size' of kmalloc object is enabled by debug option, it should either contains the actual requested size or the cache's 'object_size'. But it's not true if that object is a kfence-allocated one, and the data at 'orig_size' offset of metadata could be zero or other values. This is not a big issue for current 'orig_size' usage, as init_object() and check_object() during alloc/free process will be skipped for kfence addresses. But it could cause trouble for other usage in future. Use the existing kfence helper kfence_ksize() which can return the real original request size. Signed-off-by: Feng Tang Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/slub.c b/mm/slub.c index af9a80071fe0..1d348899f7a3 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -768,6 +768,9 @@ static inline unsigned int get_orig_size(struct kmem_cache *s, void *object) { void *p = kasan_reset_tag(object); + if (is_kfence_address(object)) + return kfence_ksize(object); + if (!slub_debug_orig_size(s)) return s->object_size; From patchwork Wed Oct 16 15:41:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13838601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1CCFD2A529 for ; Wed, 16 Oct 2024 15:42:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72DA76B0093; Wed, 16 Oct 2024 11:42:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DCD66B0095; Wed, 16 Oct 2024 11:42:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5575A6B0096; Wed, 16 Oct 2024 11:42:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 344086B0093 for ; Wed, 16 Oct 2024 11:42:08 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BFE8A406CA for ; Wed, 16 Oct 2024 15:42:01 +0000 (UTC) X-FDA: 82679881116.07.37A7F8E Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf27.hostedemail.com (Postfix) with ESMTP id D86F040002 for ; Wed, 16 Oct 2024 15:41:56 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PcxNTrAX; spf=pass (imf27.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729093252; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SAsqbT7F8ZH/bzSMugdy64lPBN/0mAPzdY7m+7h1gzw=; b=kROBWEDiqUiSn1bEm9EksVcwuX1TXrnI5ZDbyl7qeqBP2kyRsoIlaskDTuuuZpGeS/lx/e FgteXZdEywZsxMvz0BXmRamJIKS62DHCRa7Vhok+ICkVFL3pzOh58PvPsO9bkKBrTOvHBA JxxPJB5QdCoQjnNDxexQzkP4ZOguPfI= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=PcxNTrAX; spf=pass (imf27.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729093252; a=rsa-sha256; cv=none; b=z/DdtE4aV+0kR+JsW4PCxTc7195zb2smMvmpbPbnW86l7XU4gAknHZU+5h3SgaZfEdoJ/B UGv+HIzjqtdPHrXZe2E2KssRpBJz8XBX7Z9beIzRqOLaIW2FxzJmDcM1TgXLGWKOMxGSPJ 8U+25y4AQevcUEm8wX7t43Bw2pJDTHI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729093325; x=1760629325; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SfN+aazMvPNd5suMLtjlSQgXeOWhjVA+dg6w5Bfb7iQ=; b=PcxNTrAXTTSab0pUUj3y5GT7I7A8AFE6pLpp6f0dtJoWHUolLmDhgUW6 KHNdwzZVInOo9Vg2bS9sei8FEq+wTXvb83OUoUQvphESeGpB1Kru8Npl/ UPM0NHgjgH3G863f0f1qBuF7UeLlR+gCAUCnjOP/TWjIQ4hy46XH4kKqV 9ZkYJCmxP6CwTVcUcBM7xsmEp30mEFTX2XOSoHpAD+SpHM+hw2wOSjQqx k6M5xMOteHAxBT1drvNm4/eJMcq1dH3Yuh2u7RroO5sVZAeUOq+QL6OCH vCtU1ZUx/VDWuha/Y1N5yA6A77R1tv1PoUxZBERCkttRx8KbHuiR9ynyN A==; X-CSE-ConnectionGUID: CFpYAv0IR9O330Lgl690Uw== X-CSE-MsgGUID: qvtO0ceKRmOMqbCrSF7O+g== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="46021370" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="46021370" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Oct 2024 08:42:05 -0700 X-CSE-ConnectionGUID: UUGX2kglTD2IGECAAw+E6g== X-CSE-MsgGUID: 51KYtXJaTQ6dVWcB49DHcg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,208,1725346800"; d="scan'208";a="109018920" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by fmviesa001.fm.intel.com with ESMTP; 16 Oct 2024 08:42:01 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Danilo Krummrich , Narasimhan.V@amd.com Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 2/3] mm/slub: Improve redzone check and zeroing for krealloc() Date: Wed, 16 Oct 2024 23:41:51 +0800 Message-Id: <20241016154152.1376492-3-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241016154152.1376492-1-feng.tang@intel.com> References: <20241016154152.1376492-1-feng.tang@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: D86F040002 X-Stat-Signature: 5n3e7bhdbnj5hhuybar3miti67dfoofc X-HE-Tag: 1729093316-628493 X-HE-Meta: U2FsdGVkX1+mhi8XzUkLhzbTSTIUihtJByUaUErNRAf0rNKTP1sJgLEBWri5ZJaFtWfQVq+q3fKNyrW5QNBIFlcKR0W+rRHVGj9LkeMLZRwG9ARGiiwOTziootE7z/2DsWhB3fsHZPeYLH7a0ov1l9Zz4SC86QFkxE/qJ5oIy4lPKESpvoYiZ6dQ8aaGN7Q3O8uwMY3T0kb6eSu3aggWweN5PdgHWPxL4JNx3PRQqMLmKTaHWE2/GODujbXD3bMTfAwMXatzLIb6Xo4byWsuEtr5iRme5p4noy70mBT1lYgjyGvdVtFCDibfVmE2c0ANy3GqRjZmHaPhnmd3Nf87wKsrlcdjiuvNPPvI+rZ0AlPZWhaIbjqyPqcw0G0MAJBAD591d2y1lEJRYGzx2xO8z1sdSIEZmEzrDZ5ivQpzXKdZiWXxxs+3Chago93cnGg4LvOThqVx3YYmUqkDlfWUTCsTJGKG+M35xKJGwtLlKvvV/ZnOjZHAgSCRJ6yO6if6JvHucL7PY19NRNB9gsEi6gTIaEze6+KOQdbW75CMHLkWYSm7e0A23aZsEv++CtXJYKhIFDImjwaZW+t0C/L5DGXr+BYvBTrBJLgw4Kqiqa4AmS+F7aNMGeFqyGmDkCblLFAzCWgoEuGcckCbkfkqvZzMJ2QbZNdyVCisRXylD4nXi+0Kz7W/L9UaWAhJUDmaYuveSC8n/Arri6+AApiZouCo3v0BHFb+8+jtBlLTESLTke82GN40utwxC8T7EBUXNjEKHCQAJfypkK9o0vD384bRez2Xi+ECaSxINlrM5aO0kgY+9beK4iHcgmD+p/Ni9eDq0ME0G6XmYchsYz1a+2oPoemzpPrOcODDVNsscggGY1msiEb2oxWuvTQTvO2JO8Sbk0VA3v7S6RNm2L9IEbdZSECrKYdJ3q09Lkyw8AP/h7P6RitYi24CsKJNGA1vrBqnjqmSrkDVtaM80i8 R6kDrJQo yRXRWvG5hFCIxGbSKoaWnc0E6MrD4Hi0HsLaaBJKMByknKVGjoYt3YMFmxpMvQLFLZB4raalL/8PQwqxF0/JW9xRTLJTRNiWVA97WCfXOTPFpSBqDkyDsIOFADyz0BfgyfIMqJ6VqLMdAH+GnrOOFZDDnUl1O8kvMshgp1bNNxFkT9EizX2PQPNDWwYt/JYCbJRjhZj7E3rVHKO9p9hZmfq4H6xB7FRlgFq6HhlcVNWzapDqN++4KyuOs6Zx0XnrghTv5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For current krealloc(), one problem is its caller doesn't pass the old request size, say the object is 64 bytes kmalloc one, but caller may only requested 48 bytes. Then when krealloc() shrinks or grows in the same object, or allocate a new bigger object, it lacks this 'original size' information to do accurate data preserving or zeroing (when __GFP_ZERO is set). Thus with slub debug redzone and object tracking enabled, parts of the object after krealloc() might contain redzone data instead of zeroes, which is violating the __GFP_ZERO guarantees. Good thing is in this case, kmalloc caches do have this 'orig_size' feature. So solve the problem by utilize 'org_size' to do accurate data zeroing and preserving. [Thanks to syzbot and V, Narasimhan for discovering kfence and big kmalloc related issues in early patch version] Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 84 +++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 60 insertions(+), 24 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 1d348899f7a3..958f7af79fad 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4718,34 +4718,66 @@ static __always_inline __realloc_size(2) void * __do_krealloc(const void *p, size_t new_size, gfp_t flags) { void *ret; - size_t ks; - - /* Check for double-free before calling ksize. */ - if (likely(!ZERO_OR_NULL_PTR(p))) { - if (!kasan_check_byte(p)) - return NULL; - ks = ksize(p); - } else - ks = 0; - - /* If the object still fits, repoison it precisely. */ - if (ks >= new_size) { - /* Zero out spare memory. */ - if (want_init_on_alloc(flags)) { - kasan_disable_current(); + size_t ks = 0; + int orig_size = 0; + struct kmem_cache *s = NULL; + + /* Check for double-free. */ + if (unlikely(ZERO_OR_NULL_PTR(p))) + goto alloc_new; + + if (!kasan_check_byte(p)) + return NULL; + + if (is_kfence_address(p)) { + ks = orig_size = kfence_ksize(p); + } else { + struct folio *folio; + + folio = virt_to_folio(p); + if (unlikely(!folio_test_slab(folio))) { + /* Big kmalloc object */ + WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE); + WARN_ON(p != folio_address(folio)); + ks = folio_size(folio); + } else { + s = folio_slab(folio)->slab_cache; + orig_size = get_orig_size(s, (void *)p); + ks = s->object_size; + } + } + + /* If the old object doesn't fit, allocate a bigger one */ + if (new_size > ks) + goto alloc_new; + + /* Zero out spare memory. */ + if (want_init_on_alloc(flags)) { + kasan_disable_current(); + if (orig_size && orig_size < new_size) + memset((void *)p + orig_size, 0, new_size - orig_size); + else memset((void *)p + new_size, 0, ks - new_size); - kasan_enable_current(); - } + kasan_enable_current(); + } - p = kasan_krealloc((void *)p, new_size, flags); - return (void *)p; + /* Setup kmalloc redzone when needed */ + if (s && slub_debug_orig_size(s)) { + set_orig_size(s, (void *)p, new_size); + if (s->flags & SLAB_RED_ZONE && new_size < ks) + memset_no_sanitize_memory((void *)p + new_size, + SLUB_RED_ACTIVE, ks - new_size); } + p = kasan_krealloc((void *)p, new_size, flags); + return (void *)p; + +alloc_new: ret = kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _RET_IP_); if (ret && p) { /* Disable KASAN checks as the object's redzone is accessed. */ kasan_disable_current(); - memcpy(ret, kasan_reset_tag(p), ks); + memcpy(ret, kasan_reset_tag(p), orig_size ?: ks); kasan_enable_current(); } @@ -4766,16 +4798,20 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that * __GFP_ZERO is not fully honored by this API. * - * This is the case, since krealloc() only knows about the bucket size of an - * allocation (but not the exact size it was allocated with) and hence - * implements the following semantics for shrinking and growing buffers with - * __GFP_ZERO. + * When slub_debug_orig_size() is off, krealloc() only knows about the bucket + * size of an allocation (but not the exact size it was allocated with) and + * hence implements the following semantics for shrinking and growing buffers + * with __GFP_ZERO. * * new bucket * 0 size size * |--------|----------------| * | keep | zero | * + * Otherwise, the original allocation size 'orig_size' could be used to + * precisely clear the requested size, and the new size will also be stored + * as the new 'orig_size'. + * * In any case, the contents of the object pointed to are preserved up to the * lesser of the new and old sizes. * From patchwork Wed Oct 16 15:41:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13838602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC2A8D2A529 for ; Wed, 16 Oct 2024 15:42:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D23D6B0096; Wed, 16 Oct 2024 11:42:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 384726B0098; Wed, 16 Oct 2024 11:42:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FC066B0099; Wed, 16 Oct 2024 11:42:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 014716B0096 for ; Wed, 16 Oct 2024 11:42:11 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CB7721A0609 for ; Wed, 16 Oct 2024 15:41:53 +0000 (UTC) X-FDA: 82679881494.04.A7CD92C Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf27.hostedemail.com (Postfix) with ESMTP id C78C84000C for ; Wed, 16 Oct 2024 15:42:00 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=K25Pibhp; spf=pass (imf27.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729093256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2hm99wYh6qNVY2KuNWC++KmNGsWyV5y7wgmQq9+ww+Y=; b=qmQ8Zih8nqfjbg+j3CDgsBtyRvYBmveFgzq+Qt6U6BQ2qFlKJhWTDRp7J1k/VQ9pwTHfzq WyQTXJ4J9gC3DVvSdsxi/h/2hGnzEWnoaccHWvqjCkFldPP+fxU10URGbRXabMMBEDseL9 FT+D4mRwUEbaUYwbJ1hA07QHa39GxsY= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=K25Pibhp; spf=pass (imf27.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729093256; a=rsa-sha256; cv=none; b=Y7yL1+IcBuToBPhtwL2I30CBV5l7KPY1McKxTBwvM95s5iByz77ArxhEW2t1X5CmlkXJ34 VV1JwNjJM4BSmVJEwk5HqjZbF9R0UPMxWAH/J3xYtbj+Qz1BMftWL54PqopCNGKG9QI4+v +9z6673RnX/PIOUh2Futv3lZou71bFY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729093329; x=1760629329; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u/Fj3XtocN55VN0j0tMVNeViXP1RJmwNkXYG9TfzlL0=; b=K25Pibhp4GE8dytnYTGvKogRYWLWL4+ZgWw/f3POTG6kniBowz46nvd5 vc3yXHG+CndMOXWZuFXFszDZ1hT1ZYRjpjaQgR/YwcnsMkIUH5uTwdzLD +mgj+ApnCaDRiglfkaEsYr2w03c7iEnmyBiOL2/w5MBjHRiHU05STPCOD /YZGuC9y+H2eSvu6sEP4TLtZ4b4fYuvd9y2x7G3nCbrfDnMkR/R0rC9/K YiJ3948WOMcdwwbHXnawu59NnQEMOlbPhm9bdUUx5xs9aYFz8gTdXCErD LCLf/Wmuslb2HeJr4IpdrmEOTedLfee/rJYKaLQuizGIlORLTUuOgVcZK A==; X-CSE-ConnectionGUID: 1xU+ZU0NRDeYzKlj46jLJA== X-CSE-MsgGUID: jRO3YD8rSDq1Glv+iB8ErA== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="46021388" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="46021388" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Oct 2024 08:42:09 -0700 X-CSE-ConnectionGUID: r7upwpckRCmJoHJMMwP35w== X-CSE-MsgGUID: iXHmcnIjQDGr0gnwyXsyLQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,208,1725346800"; d="scan'208";a="109018940" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by fmviesa001.fm.intel.com with ESMTP; 16 Oct 2024 08:42:05 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Danilo Krummrich , Narasimhan.V@amd.com Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 3/3] mm/slub, kunit: Add testcase for krealloc redzone and zeroing Date: Wed, 16 Oct 2024 23:41:52 +0800 Message-Id: <20241016154152.1376492-4-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241016154152.1376492-1-feng.tang@intel.com> References: <20241016154152.1376492-1-feng.tang@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: C78C84000C X-Stat-Signature: cuf6pgu3yoc69drccqfmbdpypii8bkic X-HE-Tag: 1729093320-10225 X-HE-Meta: U2FsdGVkX19FAfs/beaLcYIbxn/LLI+zlU5Caj7OP3uuItgjNKM6l5LupK/Q7t2cdaMYBqhMeQwLqtOpuqpZHVImrvT/dKeKaNOAxMOgGng8HBkF+hvklk+jgRRDKVg2q7sC21lPG+0Y/LA+GYCtE7S4tlbI1bqEbffRFwatur7ytYQJcIcesKH5FZmoA8/Vzg0lVmwPzwBeKSEoa5ZjB4LKgeBETiNPJZ5nd/nDgpX5TzjcW2BhGKiggEmKgBIWRhwJUzzwNS0PebJNA7q4y8vMEwB9aIysA2w7Agoh8QQNlkVwxKGzX0O8giJzXFIIbOcLmax5gV/er4w0MSbR1/nyN72qnSMIX9QDfqz9dQUeCU2yAXLgl1ZrPyjzfT7JuiGfif2Gr8TZU2AnuvOkewCR63QBkO/MLUNMfqVTjU7npDXtsybs9ntgY+OKyzgDBbd6XVJrmR2vtgLvN6iBq81GNrOErJnX+Js3UmZI/mn+fU149H9mlkvRSrwk3/+G/cemBDE10fW96PzhYx/LohCfw/tZrqj5lYW6kxwtSzsyG/3MzgoW1YVzlnbqd7hupxOA9UICJUMP2Ft36PAeDl1cbsdEXLWwNO0IaV3LCl4XJ9ZMcpZcJ+C2f9Tem/46HWrK6u/C18pIwbzSbAKalN6D2klAZoEvUbV7SjMkx3x5k1DVJcCoyisNj9mbLermZj1w07Y+1PS7SNmN2Ley24HB4pJ0Ht9gDsIj5WSlruTvL2qe/iKVH7A/PvYHBwxYNpU5CqXTv7FOwVaHtDjxSzlT/3mKfoqYuTDVlUks+nNVJ8/LBQGgvpqGrJM2NedUcDqpBR8wnrs03vbqvHUBP/Gpto2H4pGyt7V4WBRpAuUUlCOPVx7s4p2mv3IoObyzlFgEUNdHe9pZPcPROuPjFZqWYQQfRiKssCVjVr1oGG1nupz6hiLOxQYr0EbnV+DeSCnN5IYEQmOJI6yIkSu VmnnnJRc 5/sE4NsNvwnCIIATvRsc9eHYkYQ6sZtUla9KJh0sjk1ek2paAobMBiv6Kr7RpCVpjPY/KkYCgmIiJH1wrvd3yZmMeSqXdiIteHgtxEJqVY3UTN3JMjW3B6/OLMgJucNGntxC5WVRAdRIc9+syIhfa4N4t+28NfMkBPbxvvCge5RBHqb2pd1KhanLV+s94J0Sw1Vb2BBX+7hctcvTsqdDhgnUrX7jKQAuOlE/uDqsMb98b8eLqhxTCEkxJj9HD84p7pS6TDw6sHew2HVu7gQhkMks2YA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Danilo Krummrich raised issue about krealloc+GFP_ZERO [1], and Vlastimil suggested to add some test case which can sanity test the kmalloc-redzone and zeroing by utilizing the kmalloc's 'orig_size' debug feature. It covers the grow and shrink case of krealloc() re-using current kmalloc object, and the case of re-allocating a new bigger object. [1]. https://lore.kernel.org/lkml/20240812223707.32049-1-dakr@kernel.org/ Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang --- lib/slub_kunit.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/lib/slub_kunit.c b/lib/slub_kunit.c index 80e39f003344..3cd1cc667988 100644 --- a/lib/slub_kunit.c +++ b/lib/slub_kunit.c @@ -192,6 +192,47 @@ static void test_leak_destroy(struct kunit *test) KUNIT_EXPECT_EQ(test, 2, slab_errors); } +static void test_krealloc_redzone_zeroing(struct kunit *test) +{ + u8 *p; + int i; + struct kmem_cache *s = test_kmem_cache_create("TestSlub_krealloc", 64, + SLAB_KMALLOC|SLAB_STORE_USER|SLAB_RED_ZONE); + + p = __kmalloc_cache_noprof(s, GFP_KERNEL, 48); + memset(p, 0xff, 48); + + kasan_disable_current(); + OPTIMIZER_HIDE_VAR(p); + + /* Test shrink */ + p = krealloc(p, 40, GFP_KERNEL | __GFP_ZERO); + for (i = 40; i < 64; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + + /* Test grow within the same 64B kmalloc object */ + p = krealloc(p, 56, GFP_KERNEL | __GFP_ZERO); + for (i = 40; i < 56; i++) + KUNIT_EXPECT_EQ(test, p[i], 0); + for (i = 56; i < 64; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + + validate_slab_cache(s); + KUNIT_EXPECT_EQ(test, 0, slab_errors); + + memset(p, 0xff, 56); + /* Test grow with allocating a bigger 128B object */ + p = krealloc(p, 112, GFP_KERNEL | __GFP_ZERO); + for (i = 0; i < 56; i++) + KUNIT_EXPECT_EQ(test, p[i], 0xff); + for (i = 56; i < 112; i++) + KUNIT_EXPECT_EQ(test, p[i], 0); + + kfree(p); + kasan_enable_current(); + kmem_cache_destroy(s); +} + static int test_init(struct kunit *test) { slab_errors = 0; @@ -214,6 +255,7 @@ static struct kunit_case test_cases[] = { KUNIT_CASE(test_kmalloc_redzone_access), KUNIT_CASE(test_kfree_rcu), KUNIT_CASE(test_leak_destroy), + KUNIT_CASE(test_krealloc_redzone_zeroing), {} };