From patchwork Mon Mar 3 05:37:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peilin Ye X-Patchwork-Id: 13998214 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CA251E5716 for ; Mon, 3 Mar 2025 05:37:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740980276; cv=none; b=BSNK2GNeilZjP679I7TpJsUgBYuZ2DUB6UVpLtgb8/D7ES9tWTemJhyJGLemLY7W7vyBW99xG9UFH+2yPKFzvkwQOj/nFZJL/V8/eW7qHgb8GXWMOBvIm004VGHabqGkl8AiZZ04UQkLqgDUZ1IM3wQ4kgRVuwVySHeDy5pfFfo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740980276; c=relaxed/simple; bh=3s+Yrmj5yMBXmNPTntJWviC8ExUfdF4PHTRQyWTWZps=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qskbJKhqxmmQLmD8PVsCSECsgcqzlZ4N7rPy1lm62SQk4MpuNbolz4vvMF78iSswBVGBa/t2eDynoseboztH9dqvkjmZ1ISRjYgown2UjHteMVeifIkq8CUBe9QYp1BSIGzZenyMYYSANzpWz6idLjx5WndqcFL1rYJR7F6O5V8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yepeilin.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=HZqvjses; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yepeilin.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="HZqvjses" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2f816a85facso7923659a91.3 for ; Sun, 02 Mar 2025 21:37:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740980274; x=1741585074; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6BuDMmX0jHHwSlxRNuzBpxukuM3wSJDBUPrtSODIeAg=; b=HZqvjseskg0PGTGeauAj4lVw95AVXYhELnObHgsk7ipCB1+ZD8bBY3UTS2aTt/FK+X yNuKXAUuIKBFXx56Kf1+LhX5lvr2goXMDxHzNbep26Vy3Xi8GKG852TQrP+YhqjuaH3/ x/fA/tcTOzBd5aqBka/puhRAwiTY6wLGr1vH3p1/rywwaooZ0vHKctuIVCBRf4sZcnqA CP4nuUBZRWNT4N09l8ZhT+0mJBOu/yfHsZz1n/dTZSGCxQWdoJlYaZkhsMMJI4kFsiFJ TWOl++/jJKZ5iylVP7U82UeGvGeNYqXG5EdW2y2xo5ZANjYtJ1nPlymIRRyNOoWo9Cpd PehQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740980274; x=1741585074; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6BuDMmX0jHHwSlxRNuzBpxukuM3wSJDBUPrtSODIeAg=; b=u6BpcpWqVaPJkIbjW8oS2L/+Z66f3a/n3l3Wk9Bfn0yII5pEj6/Q4eh11ZNnImAXff 71sKxnTXWLtSPswsr7phG7FmrLzrDnB/LlpDChspGCK5Twa0ZNiTSmJaPGYJamyL0/DO sGRss6wp5j0tffs/XxGys0lGj2DcPTlZnBF0Armt9C1r1hl+tj8yzymfLLnO2xvJ4qmZ 55EvJ7crQcMQSOhxPyqCi82cUFPGNzWyNN5Ca0gut1ULrP9XLgDQqAcBOOHaJzV1NzQr +NPMM53fszjwKBiejW84mRVntakWO73i66rBnVU+IHnNWO6YEg7/iUQ/pPpsOC5qFdeG ioyQ== X-Gm-Message-State: AOJu0YwOmeiaFLR70YsNWNiSNM8EbwquH6xdkmLHvGj4rpJdGgKKsd8s AvZejTHXRswGrlvTZ6yH8LHGlM4VSauGSo6jwYcgLz33UMDXkbIoucKApSZqp1woPjBsq4GJ0t2 n+9Y4p5SCo8o4FcNgNmuCx3zDZmwkjteh5KsyNhtjFMBTzAlMqM3kWpnWiah35fNxFxvPub+Vy0 bwGikHdlY5gFL2Fx2tYc1H8vTvmmTS1Z/w63qqjSI= X-Google-Smtp-Source: AGHT+IF+dFa+oVN+px7oriCejWI8DZczRIcXQ/DWwnis3CtNEJ4xMwkMzgiRSThkIyWy6x7SDoJRZjZlfz6A3w== X-Received: from pjbqx5.prod.google.com ([2002:a17:90b:3e45:b0:2fa:a101:755]) (user=yepeilin job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4fc8:b0:2ee:c9b6:4c42 with SMTP id 98e67ed59e1d1-2febab78711mr21685276a91.16.1740980274486; Sun, 02 Mar 2025 21:37:54 -0800 (PST) Date: Mon, 3 Mar 2025 05:37:50 +0000 In-Reply-To: Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.48.1.711.g2feabab25a-goog Message-ID: <347830eb25b3bfea7687d4081852fb7d54e82307.1740978603.git.yepeilin@google.com> Subject: [PATCH bpf-next v4 06/10] arm64: insn: Add load-acquire and store-release instructions From: Peilin Ye To: bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Peilin Ye , bpf@ietf.org, Alexei Starovoitov , Xu Kuohai , Eduard Zingerman , David Vernet , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Jonathan Corbet , "Paul E. McKenney" , Puranjay Mohan , Ilya Leoshkevich , Heiko Carstens , Vasily Gorbik , Catalin Marinas , Will Deacon , Quentin Monnet , Mykola Lysenko , Shuah Khan , Ihor Solodrai , Yingchi Long , Josh Don , Barret Rhoden , Neel Natu , Benjamin Segall , linux-kernel@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add load-acquire ("load_acq", LDAR{,B,H}) and store-release ("store_rel", STLR{,B,H}) instructions. Breakdown of encoding: size L (Rs) o0 (Rt2) Rn Rt mask (0x3fdffc00): 00 111111 1 1 0 11111 1 11111 00000 00000 value, load_acq (0x08dffc00): 00 001000 1 1 0 11111 1 11111 00000 00000 value, store_rel (0x089ffc00): 00 001000 1 0 0 11111 1 11111 00000 00000 As suggested by Xu [1], include all Should-Be-One (SBO) bits ("Rs" and "Rt2" fields) in the "mask" and "value" numbers. It is worth noting that we are adding the "no offset" variant of STLR instead of the "pre-index" variant, which has a different encoding. Reference: Arm Architecture Reference Manual (ARM DDI 0487K.a, ID032224), * C6.2.161 LDAR * C6.2.353 STLR [1] https://lore.kernel.org/bpf/4e6641ce-3f1e-4251-8daf-4dd4b77d08c4@huaweicloud.com/ Acked-by: Xu Kuohai Signed-off-by: Peilin Ye --- arch/arm64/include/asm/insn.h | 8 ++++++++ arch/arm64/lib/insn.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/arch/arm64/include/asm/insn.h b/arch/arm64/include/asm/insn.h index 2d8316b3abaf..39577f1d079a 100644 --- a/arch/arm64/include/asm/insn.h +++ b/arch/arm64/include/asm/insn.h @@ -188,8 +188,10 @@ enum aarch64_insn_ldst_type { AARCH64_INSN_LDST_STORE_PAIR_PRE_INDEX, AARCH64_INSN_LDST_LOAD_PAIR_POST_INDEX, AARCH64_INSN_LDST_STORE_PAIR_POST_INDEX, + AARCH64_INSN_LDST_LOAD_ACQ, AARCH64_INSN_LDST_LOAD_EX, AARCH64_INSN_LDST_LOAD_ACQ_EX, + AARCH64_INSN_LDST_STORE_REL, AARCH64_INSN_LDST_STORE_EX, AARCH64_INSN_LDST_STORE_REL_EX, AARCH64_INSN_LDST_SIGNED_LOAD_IMM_OFFSET, @@ -351,6 +353,8 @@ __AARCH64_INSN_FUNCS(ldr_imm, 0x3FC00000, 0x39400000) __AARCH64_INSN_FUNCS(ldr_lit, 0xBF000000, 0x18000000) __AARCH64_INSN_FUNCS(ldrsw_lit, 0xFF000000, 0x98000000) __AARCH64_INSN_FUNCS(exclusive, 0x3F800000, 0x08000000) +__AARCH64_INSN_FUNCS(load_acq, 0x3FDFFC00, 0x08DFFC00) +__AARCH64_INSN_FUNCS(store_rel, 0x3FDFFC00, 0x089FFC00) __AARCH64_INSN_FUNCS(load_ex, 0x3FC00000, 0x08400000) __AARCH64_INSN_FUNCS(store_ex, 0x3FC00000, 0x08000000) __AARCH64_INSN_FUNCS(mops, 0x3B200C00, 0x19000400) @@ -602,6 +606,10 @@ u32 aarch64_insn_gen_load_store_pair(enum aarch64_insn_register reg1, int offset, enum aarch64_insn_variant variant, enum aarch64_insn_ldst_type type); +u32 aarch64_insn_gen_load_acq_store_rel(enum aarch64_insn_register reg, + enum aarch64_insn_register base, + enum aarch64_insn_size_type size, + enum aarch64_insn_ldst_type type); u32 aarch64_insn_gen_load_store_ex(enum aarch64_insn_register reg, enum aarch64_insn_register base, enum aarch64_insn_register state, diff --git a/arch/arm64/lib/insn.c b/arch/arm64/lib/insn.c index b008a9b46a7f..9bef696e2230 100644 --- a/arch/arm64/lib/insn.c +++ b/arch/arm64/lib/insn.c @@ -540,6 +540,35 @@ u32 aarch64_insn_gen_load_store_pair(enum aarch64_insn_register reg1, offset >> shift); } +u32 aarch64_insn_gen_load_acq_store_rel(enum aarch64_insn_register reg, + enum aarch64_insn_register base, + enum aarch64_insn_size_type size, + enum aarch64_insn_ldst_type type) +{ + u32 insn; + + switch (type) { + case AARCH64_INSN_LDST_LOAD_ACQ: + insn = aarch64_insn_get_load_acq_value(); + break; + case AARCH64_INSN_LDST_STORE_REL: + insn = aarch64_insn_get_store_rel_value(); + break; + default: + pr_err("%s: unknown load-acquire/store-release encoding %d\n", + __func__, type); + return AARCH64_BREAK_FAULT; + } + + insn = aarch64_insn_encode_ldst_size(size, insn); + + insn = aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RT, insn, + reg); + + return aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RN, insn, + base); +} + u32 aarch64_insn_gen_load_store_ex(enum aarch64_insn_register reg, enum aarch64_insn_register base, enum aarch64_insn_register state,