From patchwork Tue Mar 12 22:48:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hassan Naveed X-Patchwork-Id: 10850103 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B9CD17E6 for ; Tue, 12 Mar 2019 22:48:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76C082940E for ; Tue, 12 Mar 2019 22:48:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 66D49296F1; Tue, 12 Mar 2019 22:48:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5AB32940E for ; Tue, 12 Mar 2019 22:48:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726985AbfCLWsK (ORCPT ); Tue, 12 Mar 2019 18:48:10 -0400 Received: from mail-eopbgr710114.outbound.protection.outlook.com ([40.107.71.114]:7968 "EHLO NAM05-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726008AbfCLWsJ (ORCPT ); Tue, 12 Mar 2019 18:48:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wavesemi.onmicrosoft.com; s=selector1-wavecomp-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=px6azOeF5G5eDzNqDiu6bgy7p14pfhXsTre46cURaas=; b=dyLArUxsRGUqhW+FluWa+TRP/2GXXOYUotGBp3BRe/kFsXjlpc6CGpmUrr27SksTzts3fiYYZYXBU+O/188rqmxrlHFdAeujFpFASW1cPnyKjz+5isC5xPi6v9l5sPwlbeBxvkJrxo7rVk8x4lEu2jCmAafh+1d9FZ5icxeqlm8= Received: from CY4PR2201MB1349.namprd22.prod.outlook.com (10.171.209.23) by CY4PR2201MB1720.namprd22.prod.outlook.com (10.165.90.35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1686.18; Tue, 12 Mar 2019 22:48:06 +0000 Received: from CY4PR2201MB1349.namprd22.prod.outlook.com ([fe80::901a:e86:7fcf:8856]) by CY4PR2201MB1349.namprd22.prod.outlook.com ([fe80::901a:e86:7fcf:8856%3]) with mapi id 15.20.1686.021; Tue, 12 Mar 2019 22:48:06 +0000 From: Hassan Naveed To: Paul Burton CC: "kafai@fb.com" , "songliubraving@fb.com" , "yhs@fb.com" , "netdev@vger.kernel.org" , "bpf@vger.kernel.org" , "linux-mips@vger.kernel.org" , Hassan Naveed , Ralf Baechle , Paul Burton , James Hogan , Alexei Starovoitov , Daniel Borkmann , "open list:MIPS" , open list Subject: [PATCH v2 2/3] MIPS: eBPF: Provide eBPF support for MIPS64R6 Thread-Topic: [PATCH v2 2/3] MIPS: eBPF: Provide eBPF support for MIPS64R6 Thread-Index: AQHU2SWozNoTqPTzRU+UcrV5QYZ6oQ== Date: Tue, 12 Mar 2019 22:48:06 +0000 Message-ID: <20190312224706.6121-2-hnaveed@wavecomp.com> References: <20190312224706.6121-1-hnaveed@wavecomp.com> In-Reply-To: <20190312224706.6121-1-hnaveed@wavecomp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR06CA0039.namprd06.prod.outlook.com (2603:10b6:a03:14b::16) To CY4PR2201MB1349.namprd22.prod.outlook.com (2603:10b6:910:64::23) authentication-results: spf=none (sender IP is ) smtp.mailfrom=hnaveed@wavecomp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.21.0 x-originating-ip: [67.207.99.198] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9ed2f9bb-472b-443e-8833-08d6a73cca98 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:CY4PR2201MB1720; x-ms-traffictypediagnostic: CY4PR2201MB1720: x-microsoft-exchange-diagnostics: =?iso-8859-1?q?1=3BCY4PR2201MB1720=3B23=3A?= =?iso-8859-1?q?tbFXCCHDCxxE2vv3XlursSWBiwm8RT8nhPm4FRDRHhx/cpjgwkOienyUDW/Z?= =?iso-8859-1?q?VtVxZaB1MwIMPebl08U9aw6HMS/yPmgxrmi8cFpoGnI+uWHUAbEveLN6lpEk?= =?iso-8859-1?q?zZQJ6PqEaiZ1rJ6f0Vmc7xOZpOr4gaWDT2TfHrfEU5YkRF4zocbzI8ddExLo?= =?iso-8859-1?q?aVbZXeNZb9PEHyW0fRvEjl8R1S3rJ74Ata9ywFopZTALNazNY0qDRG0kRtnR?= =?iso-8859-1?q?tiJEEkh6FXjfGjsCaIk6jvnmuvRFJU82JixyiOrAmpy/qLHIFsGIkGjtsyuF?= =?iso-8859-1?q?PnX7eNM9S2/PAnUE4Dr/eV6GEnLQD9EvmiyOL54iklQZnhLQdxQ1Kl8QEJ6V?= =?iso-8859-1?q?6o7yrks1KOo2rgwOeuqSs2I/RDyOuyfDFjok06CfJ3cFNVJvjK9R5MHLQ06M?= =?iso-8859-1?q?pktBtiOqi9NmRfPxqrQVNnEfBe119XenLKnREyzQVlwF+KIbjsfuZsPYmNfz?= =?iso-8859-1?q?kkIrOthGxVugVKeJwTsRNCVxYyM5CJuB1C2wLGBZ5yzVtQfecQ70JSlN4c4E?= =?iso-8859-1?q?Gd7mt/d+1N6vUcc4w1MZdY0yiHY5xVSOqtWDC0g3Cm+xqSsWzzqGrvMoTv6v?= =?iso-8859-1?q?VHb2d48VqxbwQJgbcu5bp6woPF74/zCjCzdarCr7eo702hT0i22gNiD44CfS?= =?iso-8859-1?q?7JsRxO2csyaIN9Dpgm3RbwVWyQgrE+YjVJQSFumcELUGT/Ey+l6luHBa6v9o?= =?iso-8859-1?q?lNLugHz8QhLzKAWWOwl8xZIsTumLbVLitDrk43GqhZy1hlED13b2T3SpL51f?= =?iso-8859-1?q?lKQ3uTBAHgeMwmc1OdpueXcuGw+o/ZVklzqxUOxyRBGiwYHuxzXsadA3imE7?= =?iso-8859-1?q?H4YjS+I5INKPQLQlCw5BoG0/UBuX5eKu9FyrU/fSiILi/O4W78wtCrbaChfa?= =?iso-8859-1?q?cyMuEDpLfCe2MAiDFmQPRdmAmiokSI/PlNMwMwd1j+dveNf+qB/HbFmV9GEs?= =?iso-8859-1?q?KeAFK9/5DCi4nVRx9CZgvZ9H2BpX0cRe6cWqdP7DsuKBS0Y93c1Juw3lH+FC?= =?iso-8859-1?q?Z2MZDKuK0Y7jMbgC5FCBj8cSJlTusGvoDA0T4JyejL1cWgNFz0YJ001SrFKz?= =?iso-8859-1?q?rGyqD6PjzdF/Ljg2qiUvsdIhdK++s6XJuv9mWqDYOVs5k7BU7jcw0lMG3NOf?= =?iso-8859-1?q?IVNr/OHPVS2K3Sy73SfuQSqKvpS4VuqDqaP9x3djx1NcgIWnk/aDZd6zLc6u?= =?iso-8859-1?q?NSz6n0Fi/KYNXsOvtO6+jzDPVT+tGRxHlTOIkBlaMtqD4zzv/UKuivbGYswh?= =?iso-8859-1?q?NQEuSI4VlbqJHQBirisc?= x-microsoft-antispam-prvs: x-forefront-prvs: 09749A275C x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(376002)(346002)(136003)(39840400004)(366004)(396003)(189003)(199004)(66066001)(105586002)(99286004)(26005)(2906002)(53936002)(3846002)(86362001)(6116002)(36756003)(14454004)(71190400001)(71200400001)(50226002)(476003)(7736002)(305945005)(68736007)(478600001)(7416002)(6506007)(386003)(186003)(6512007)(106356001)(6862004)(1076003)(97736004)(6486002)(5660300002)(25786009)(446003)(4326008)(2616005)(102836004)(8676002)(11346002)(37006003)(76176011)(316002)(81166006)(81156014)(6636002)(486006)(8936002)(52116002)(256004)(54906003)(6436002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY4PR2201MB1720;H:CY4PR2201MB1349.namprd22.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: wavecomp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: +Wets5094GrFD1lhj1WmJPUboP8bjgoacgZTbU8i0HW3JTf3hdBmd4Nl+aGl3KvJKFl6kZ3Um6UErbn5OKfb1iBInX+uQ0cpUsF0AyK3xvetH4ZZp2cu1hgLCoG1702fMCoU13OjjhwqpYZKAkpu18zAhPINiZzw9n+bTa/tofbvnLrbqLQgC3LWzirezl51u5Tab2fei0jG1VBhfV7poL7f2CTtR6hFnVUqanzbB1cCW4otdXkXJzb9j5tBuDEsm2k6KkjwIkx8TLecTwif9E6XR/92NrUB8p7tfg8XvQaNXEWUd+AsmPpu4c0JEZOE3gfU2FeMADXioG5P1dJWXe2p+ms6rLkVfobHJl+gXk4vHxzYHXYMPR2QNCZQWECsBObHzCIXypovL9OWHuIABeHOC7btusDVbs6PkNiX1jI= MIME-Version: 1.0 X-OriginatorOrg: wavecomp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9ed2f9bb-472b-443e-8833-08d6a73cca98 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Mar 2019 22:48:06.1096 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 463607d3-1db3-40a0-8a29-970c56230104 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR2201MB1720 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently eBPF support is available on MIPS64R2 only. Use MIPS64R6 variants of instructions like multiply, divide, movn, movz so eBPF can run on the newer ISA. Also, we only need to check ISA revision before JIT'ing code, because we know the CPU is running a 64-bit kernel because eBPF JIT is only included in kernels with CONFIG_64BIT=y due to Kconfig dependencies. Signed-off-by: Hassan Naveed --- arch/mips/net/ebpf_jit.c | 78 +++++++++++++++++++++++++++++++++++----- 1 file changed, 69 insertions(+), 9 deletions(-) diff --git a/arch/mips/net/ebpf_jit.c b/arch/mips/net/ebpf_jit.c index 0effd3cba9a7..26eef9ad3896 100644 --- a/arch/mips/net/ebpf_jit.c +++ b/arch/mips/net/ebpf_jit.c @@ -22,6 +22,7 @@ #include #include #include +#include #include /* Registers used by JIT */ @@ -677,8 +678,12 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, if (insn->imm == 1) /* Mult by 1 is a nop */ break; gen_imm_to_reg(insn, MIPS_R_AT, ctx); - emit_instr(ctx, dmultu, MIPS_R_AT, dst); - emit_instr(ctx, mflo, dst); + if (MIPS_ISA_REV >= 6) { + emit_instr(ctx, dmulu, dst, dst, MIPS_R_AT); + } else { + emit_instr(ctx, dmultu, MIPS_R_AT, dst); + emit_instr(ctx, mflo, dst); + } break; case BPF_ALU64 | BPF_NEG | BPF_K: /* ALU64_IMM */ dst = ebpf_to_mips_reg(ctx, insn, dst_reg); @@ -700,8 +705,12 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, if (insn->imm == 1) /* Mult by 1 is a nop */ break; gen_imm_to_reg(insn, MIPS_R_AT, ctx); - emit_instr(ctx, multu, dst, MIPS_R_AT); - emit_instr(ctx, mflo, dst); + if (MIPS_ISA_REV >= 6) { + emit_instr(ctx, mulu, dst, dst, MIPS_R_AT); + } else { + emit_instr(ctx, multu, dst, MIPS_R_AT); + emit_instr(ctx, mflo, dst); + } break; case BPF_ALU | BPF_NEG | BPF_K: /* ALU_IMM */ dst = ebpf_to_mips_reg(ctx, insn, dst_reg); @@ -732,6 +741,13 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, break; } gen_imm_to_reg(insn, MIPS_R_AT, ctx); + if (MIPS_ISA_REV >= 6) { + if (bpf_op == BPF_DIV) + emit_instr(ctx, divu_r6, dst, dst, MIPS_R_AT); + else + emit_instr(ctx, modu, dst, dst, MIPS_R_AT); + break; + } emit_instr(ctx, divu, dst, MIPS_R_AT); if (bpf_op == BPF_DIV) emit_instr(ctx, mflo, dst); @@ -754,6 +770,13 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, break; } gen_imm_to_reg(insn, MIPS_R_AT, ctx); + if (MIPS_ISA_REV >= 6) { + if (bpf_op == BPF_DIV) + emit_instr(ctx, ddivu_r6, dst, dst, MIPS_R_AT); + else + emit_instr(ctx, modu, dst, dst, MIPS_R_AT); + break; + } emit_instr(ctx, ddivu, dst, MIPS_R_AT); if (bpf_op == BPF_DIV) emit_instr(ctx, mflo, dst); @@ -819,11 +842,23 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, emit_instr(ctx, and, dst, dst, src); break; case BPF_MUL: - emit_instr(ctx, dmultu, dst, src); - emit_instr(ctx, mflo, dst); + if (MIPS_ISA_REV >= 6) { + emit_instr(ctx, dmulu, dst, dst, src); + } else { + emit_instr(ctx, dmultu, dst, src); + emit_instr(ctx, mflo, dst); + } break; case BPF_DIV: case BPF_MOD: + if (MIPS_ISA_REV >= 6) { + if (bpf_op == BPF_DIV) + emit_instr(ctx, ddivu_r6, + dst, dst, src); + else + emit_instr(ctx, modu, dst, dst, src); + break; + } emit_instr(ctx, ddivu, dst, src); if (bpf_op == BPF_DIV) emit_instr(ctx, mflo, dst); @@ -903,6 +938,13 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, break; case BPF_DIV: case BPF_MOD: + if (MIPS_ISA_REV >= 6) { + if (bpf_op == BPF_DIV) + emit_instr(ctx, divu_r6, dst, dst, src); + else + emit_instr(ctx, modu, dst, dst, src); + break; + } emit_instr(ctx, divu, dst, src); if (bpf_op == BPF_DIV) emit_instr(ctx, mflo, dst); @@ -1006,8 +1048,15 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, emit_instr(ctx, dsubu, MIPS_R_T8, dst, src); emit_instr(ctx, sltu, MIPS_R_AT, dst, src); /* SP known to be non-zero, movz becomes boolean not */ - emit_instr(ctx, movz, MIPS_R_T9, MIPS_R_SP, MIPS_R_T8); - emit_instr(ctx, movn, MIPS_R_T9, MIPS_R_ZERO, MIPS_R_T8); + if (MIPS_ISA_REV >= 6) { + emit_instr(ctx, seleqz, MIPS_R_T9, + MIPS_R_SP, MIPS_R_T8); + } else { + emit_instr(ctx, movz, MIPS_R_T9, + MIPS_R_SP, MIPS_R_T8); + emit_instr(ctx, movn, MIPS_R_T9, + MIPS_R_ZERO, MIPS_R_T8); + } emit_instr(ctx, or, MIPS_R_AT, MIPS_R_T9, MIPS_R_AT); cmp_eq = bpf_op == BPF_JGT; dst = MIPS_R_AT; @@ -1366,6 +1415,17 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, if (src < 0) return src; if (BPF_MODE(insn->code) == BPF_XADD) { + /* + * If mem_off does not fit within the 9 bit ll/sc + * instruction immediate field, use a temp reg. + */ + if (MIPS_ISA_REV >= 6 && + (mem_off >= BIT(8) || mem_off < -BIT(8))) { + emit_instr(ctx, daddiu, MIPS_R_T6, + dst, mem_off); + mem_off = 0; + dst = MIPS_R_T6; + } switch (BPF_SIZE(insn->code)) { case BPF_W: if (get_reg_val_type(ctx, this_idx, insn->src_reg) == REG_32BIT) { @@ -1720,7 +1780,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) unsigned int image_size; u8 *image_ptr; - if (!prog->jit_requested || !cpu_has_mips64r2) + if (!prog->jit_requested || MIPS_ISA_REV < 2) return prog; tmp = bpf_jit_blind_constants(prog);