From patchwork Sat Oct 15 06:37:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Yang X-Patchwork-Id: 13007611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07174C43219 for ; Sat, 15 Oct 2022 07:00:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229577AbiJOHAh (ORCPT ); Sat, 15 Oct 2022 03:00:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229583AbiJOHAb (ORCPT ); Sat, 15 Oct 2022 03:00:31 -0400 Received: from esa10.fujitsucc.c3s2.iphmx.com (esa10.fujitsucc.c3s2.iphmx.com [68.232.159.247]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37D4832B82 for ; Sat, 15 Oct 2022 00:00:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj1; t=1665817229; x=1697353229; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=nvZLWYauKX1IRWwLxOrcarGC3WnfU7D+NGxtIx/y0Wo=; b=dPrOLiHQDB7b/oKb/qPWVCHQp/MBT69HNQLOROHDAGude0IV0SVYykw+ M/6UPFjC4/nfFOkb+4NgugAieE4xfUC3oBOGfDIdYM1f1pYHqLQnTGAa1 t91kvcNxRCPRXis4nv3S3xYolCnZpXAGlb59mt8pFQUhmuOhYqi0x+rMh MTftkeKRvmICn68oBq7XIXGrrdAZf0VuTfsoIQ5v/gLn/iyKzD+w8Pvrb A3f8vsS+TmjvX5m1EZeai7lu1UGpu2gnok51iJBib6q3h+J9n+b0dB28p PADv7uySxEnr840Qtcja4X6nvw3cRqi421Qw5UDbqbv3WQob8gP3MGF4P A==; X-IronPort-AV: E=McAfee;i="6500,9779,10500"; a="67689750" X-IronPort-AV: E=Sophos;i="5.95,186,1661785200"; d="scan'208";a="67689750" Received: from mail-tycjpn01lp2170.outbound.protection.outlook.com (HELO JPN01-TYC-obe.outbound.protection.outlook.com) ([104.47.23.170]) by ob1.fujitsucc.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Oct 2022 15:37:11 +0900 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jm92INuT8A+16bArBZFCMASAAVQu3lBjimAHS8AmoKCuOW6WH1n2GY5Mu1ecJj/uo6GKgn5+j0V+pfvoYLHCnrcHs7G38ahfjBveM8b4EMyk3JeuTckmdjaYjdApjsrKHwaRGYF3v1q+hLKo8vnrpEzxAQMYDWzEKjo3BmDm0Q7vD7mVRYsl1tVw3FF20fcX6xx3bRPPKqg16HIHp0TQgdBxgnzaT81VEGhemNWnl0YvYSFuwcGCy991s6L/x7GttFiRgopGHwaIMllOh0s2nuyWIV1MY4iV3kMOcPx+XqfM8gdlUu5bx0DMpcTwNB13tPjcDjwqIWmPKvT++HfigQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nvZLWYauKX1IRWwLxOrcarGC3WnfU7D+NGxtIx/y0Wo=; b=UP5mHcgZrhO0y+O2dqXSvf+Vgiai/tqLPhOVytLy/qEqDWXjuNml866iCFbIib91N53QK1XaXSevnPn84m/NNpoqQ1/VyIwHMo7Eyqq9HyI8puXPVYJiOLU6mSXnu9i2ZyiaOsigJ2futsWLITNoge0TlmiZFU1CisA2TDmqkeyK1NH1slgPbdnK7MNpAK7AGKM++XOZ/Q7y6i9RRotOi9N1uipBpQh1+/+zEJHLWEFaUOecs+u5u0nnyWwpmsPMLe9ST5RS+bTz0glz8QdWqiMPsBDeB3Z0Ghh2qW4QzHEd6Og3X+ihtNlX2RGpRLlfPqTadnBdK4K60WBwQ+yLZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fujitsu.com; dmarc=pass action=none header.from=fujitsu.com; dkim=pass header.d=fujitsu.com; arc=none Received: from OS3PR01MB9499.jpnprd01.prod.outlook.com (2603:1096:604:1c8::5) by OSZPR01MB8661.jpnprd01.prod.outlook.com (2603:1096:604:185::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.29; Sat, 15 Oct 2022 06:37:05 +0000 Received: from OS3PR01MB9499.jpnprd01.prod.outlook.com ([fe80::5989:c90d:abf4:e100]) by OS3PR01MB9499.jpnprd01.prod.outlook.com ([fe80::5989:c90d:abf4:e100%6]) with mapi id 15.20.5723.029; Sat, 15 Oct 2022 06:37:05 +0000 From: "yangx.jy@fujitsu.com" To: "linux-rdma@vger.kernel.org" , "jgg@nvidia.com" , "rpearsonhpe@gmail.com" CC: "leon@kernel.org" , "lizhijian@fujitsu.com" , "y-goto@fujitsu.com" , "zyjzyj2000@gmail.com" , "yangx.jy@fujitsu.com" Subject: [PATCH v6 2/8] RDMA: Extend RDMA kernel ABI to support atomic write Thread-Topic: [PATCH v6 2/8] RDMA: Extend RDMA kernel ABI to support atomic write Thread-Index: AQHY4GCK8z/n8XKDxECJBUE9Y21x9g== Date: Sat, 15 Oct 2022 06:37:04 +0000 Message-ID: <20221015063648.52285-3-yangx.jy@fujitsu.com> References: <20221015063648.52285-1-yangx.jy@fujitsu.com> In-Reply-To: <20221015063648.52285-1-yangx.jy@fujitsu.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.34.1 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=fujitsu.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: OS3PR01MB9499:EE_|OSZPR01MB8661:EE_ x-ms-office365-filtering-correlation-id: e7bf29b2-5fb4-453e-21ab-08daae77ad52 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: CzS75lsotChH8azRCEJQCp4X5kZ0VoZ0oZer8DbeHcQxyH2opxv0QyDCfrmjiRknnvQR93ozMY/VEAygMX4nxoofotAcb6UNI0R2SgjcCQwcfMoYfS0947uJ9gzAsreqf+MS5TVrR4zt6RJl+PJgy7YnEmpSpPKjdqaYNVWQV55UhdYHfLEUmGcWaC2MgFjFKEb+iHZbSmZ+AwuCW9nK9FybCm0V15/egyWW5m6S4c8VIP9sdQKFvVTqUkevee44yRl6js9oN9PQ5Z1y1eyej+VkbsCrXoNMIxOSHHaHnjeql/wz/u3s9GdQ92Q+GZu2ZMdrmPHX8XuJ3QYhK2rWt4kX5ll4niJhstzzzBlj1Klr1Vhht9lV9z+gD40Qjt3icANebEfXmRL7/qp59op/P42gEppqyhIcaEjkCoYoue7r6oEWQI5e7Ng+n51YA9Yhd11uHqS9EhEvTsUQR8lomdyZbuf4gbxX8aW1s0kEVQlH0xuFCLMSrMrF67etVa+3LNOX4gEfVPTmvezIUwYvfMgegIJV0EvN7olkrxG/dVfRrHIx496LtzCzCBNRwIDHX/lPdSHDRyeloI4ItayH1F9s1KaTbxBfZTsSVVskfhcsp/PSC8bFRG2cpYTfl3WBxsAw8IYQ4/X2N5BxnpWOCebQ2hNz2s+WPqt1Z385GUliEdL3Z5fQ/CZqRFZLlwMhGteog3pLAARyId3zql/TM/Jhr6GKzsVTUDaqrjmVeRyzQ37MV+cSaT1K3+b4w0WVz6IDUVCM5xe3NV3xQ24cqfA+pDtGXSMSX4C3TrzuKBA= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:OS3PR01MB9499.jpnprd01.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(136003)(346002)(376002)(366004)(396003)(451199015)(1590799012)(6486002)(85182001)(316002)(86362001)(38100700002)(36756003)(5660300002)(110136005)(26005)(6512007)(54906003)(478600001)(122000001)(83380400001)(8936002)(82960400001)(91956017)(1580799009)(66946007)(76116006)(2616005)(107886003)(66556008)(66476007)(38070700005)(41300700001)(71200400001)(6506007)(66446008)(64756008)(8676002)(2906002)(4326008)(1076003)(186003);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?eucgb2312_cn?b?TmZNeEp5cVpWcUxKOTN6azI0?= =?eucgb2312_cn?b?U3pBNXZLRk90T0gxYWNneG9zWHRzZExpRXg2Q1cyRjNzbEJidXhBRHZqTkxSa0lq?= =?eucgb2312_cn?b?OHlVaHNxUGd0L2libGVXVk5weUJxY2YyQUVGbFZKUDlGbHlIZ2ZLMit2QVFOUTN6?= =?eucgb2312_cn?b?cWRJanBmdHZEcjdhanpXTk5XS3UvRTJHOGVwZ2lBQXFPbXE2NDczWTVqaE5xUUh5?= =?eucgb2312_cn?b?Q0ZTc05DWFc3Ym14dWtab1kzeTE4cnhWSnZMWmYxdTd2VzluaHgvREtiZ0dnT3Mr?= =?eucgb2312_cn?b?ZSsxQlVvWllDdzR6WEtvUDhVL1BmalJxRVZMWERRak1WQW5LOFNoaXdpQzhHYjBO?= =?eucgb2312_cn?b?SDVlblVDME9Jc1RoUjVBekl5ZnA1dy9GU3ZCYWUyTitJc3M1eVl2Q29aY0gwNDRv?= =?eucgb2312_cn?b?VUVWTGQrczdPNTNyNFpNNFNhMjBLajhBYk9TaXVjelZnMFUzR2l2MmFLekJmL0g4?= =?eucgb2312_cn?b?NWZ1R1FrdlJkZGhjdjBuR1pKVFpWYU9wcnlwMzUzWnBtOEdLalVQN1VLS241ZjBp?= =?eucgb2312_cn?b?VXROYmVTTzB4MzI3TDlOOXQrd1RvRmpnYlJ3ejQ0SUVJbXpJQ2hxNGU4OVk0Sldy?= =?eucgb2312_cn?b?WU5lR2pXSVdtTmt3c2Y2R0gzeG5KVTM1UlRrTEpFR1lCSFFhMGdPNE1tTStHandi?= =?eucgb2312_cn?b?aGlvb2FNQ3JpWGFnS0lMa0oyR0VOR1RYUkwrTjUyL25zcGRqV0l5ZnlEbStBbzZW?= =?eucgb2312_cn?b?bXU1TDRFSUs2dm5aZzJpNUhWNU9GZXpEcHZTL0dJZDNJelZqb2g1aWgxd1hLRzdX?= =?eucgb2312_cn?b?UE4vS3hrRHl5eEVSdGdoUjMwMEtGSWc1enlqaXRSSzJBTXN5MGtnRHJycnF1RDZz?= =?eucgb2312_cn?b?QThFRm5kZWVZenZiZXI3V2ZFS05tZXE0ZTJpWTdFcFhEbXV2OFpCYTF6VkVNZ0hQ?= =?eucgb2312_cn?b?S3RYWGt1cHhoUitjcG1peHFRUk5nTVVhSUZZQ21aMTBRMGJDekthYi9PTHEwR3NN?= =?eucgb2312_cn?b?UHdvZ1MrNms3UExlSnNpNHpUMmZXKzBTNHFTc00vUGhTSGRwRmMrR050WUQrMEpE?= =?eucgb2312_cn?b?cDdnMWFieXk2WXNHK1dlZG5ISURXWno3MjVTN1l2OXJrc3MxcHdPcFVZL1NlQytt?= =?eucgb2312_cn?b?bmYzTk9kcGlXZERIeThmMjdENGJPdUVSaC9ocXBlcUlKREw3ZEJKcWE3NnNOb1Bx?= =?eucgb2312_cn?b?dFRubHAyRXlpcDF2eTB2MTQ4WGwwR0lXSGdWVWhRZm9Ha3RIZUZHdm9nMlNwbHhB?= =?eucgb2312_cn?b?QXMxSjBsTmJyaDg0SllIbjJlT3Z3OHpSalRiLzBBZXVlSVhzNjZuVFpuZXlLWVg3?= =?eucgb2312_cn?b?QitSdDlNT3l2WXpKNVlaVzZ0TFQzY2dhUnh3emVOalVEa09RUi8yS3cxMW9ta3RB?= =?eucgb2312_cn?b?ejJsclhCSTRsM0xEV1NER1JhMWs4VWVxbGpGNUZ6c2s3MVJIZXAwQUQzNjFkc1k2?= =?eucgb2312_cn?b?UHJpdVpscEVaSk1PbUx3dlFqd013a0RFTGRtWHRyem81MDBqV3g3Vlo4WURkYlZ6?= =?eucgb2312_cn?b?cERQa3pXdS9pajFjNEFhamlxSVJPYXBDWDBVZC8xeFY4Nkp5WmVLVzRjaU5oRW41?= =?eucgb2312_cn?b?ZmlYVGRma2tpNWRUNmo3dm1UbStDS0g0OW1JTk5DL0pZVzNZS05UQzBrZ1paZ2RS?= =?eucgb2312_cn?b?QmxRaEp3OUg0VHdDZXNRM2tZSEZMcE4vSlJzaUN4UXl6ZzdZN1pWcFdaMXhlQlVi?= =?eucgb2312_cn?b?UlZuT0t3SHZGbHEwWDFsdXpRZkxaMVAwWVBVeVBOdXJJMWtFL2RjZzZ0dmp6M0xl?= =?eucgb2312_cn?b?UVFsci9zQjUyMjdTczZTQkNJWVlGcmR2dXgrcTRrdjdTRFRaMnNIV1FvMnlsMjBS?= =?eucgb2312_cn?b?QTJhRHRaNm9UYmhvSm15ZEg2MkhuVmpERDZBMnMwY3hlblNPaDBDTXZyd2FCaWN5?= =?eucgb2312_cn?b?a0h1R1pkcGpWeHJjS0I2dkFxNVhvVDNTYmFBRUF6WGFTWStpVE9wN1duQTI1QVZZ?= =?eucgb2312_cn?b?dkNPaFB4bjZIdDZYNStPaFo2VkluUzlMSXVSemtZZmRRK0ptMnFyK1BTcjMzREg4?= =?eucgb2312_cn?b?Y3AweWh4ajlSalB0RWpJZ2tsRWpYUDNxdzA4SEdxRkhFR054ZGdMY0o4TlNlVE9H?= =?eucgb2312_cn?b?S2Z3djg5S2JEMnVtbDVzY3RtRUlJQkVMVHJCVmhIYnh1Sjk4ZUtUQm9KWkVSYXRy?= =?eucgb2312_cn?b?L055M0cvaFRMSkI0NmJQbEcxa1E9PQ==?= MIME-Version: 1.0 X-OriginatorOrg: fujitsu.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: OS3PR01MB9499.jpnprd01.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7bf29b2-5fb4-453e-21ab-08daae77ad52 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Oct 2022 06:37:04.9936 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: P4cPhW9oAarIAUn07ulQwW5Rar04LLlnLWCkS1CYGFqAjMWOdijzO/TCD9X7O1Ky/gwW9WzTAk6DE5N220t4JA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OSZPR01MB8661 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org 1) Define new atomic write request/completion in kernel. 2) Define new atomic write capability in kernel. 3) Define new atomic write opcode for RC service in packet. Signed-off-by: Xiao Yang --- include/rdma/ib_pack.h | 2 ++ include/rdma/ib_verbs.h | 3 +++ 2 files changed, 5 insertions(+) diff --git a/include/rdma/ib_pack.h b/include/rdma/ib_pack.h index a9162f25beaf..f932d164af63 100644 --- a/include/rdma/ib_pack.h +++ b/include/rdma/ib_pack.h @@ -84,6 +84,7 @@ enum { /* opcode 0x15 is reserved */ IB_OPCODE_SEND_LAST_WITH_INVALIDATE = 0x16, IB_OPCODE_SEND_ONLY_WITH_INVALIDATE = 0x17, + IB_OPCODE_ATOMIC_WRITE = 0x1D, /* real constants follow -- see comment about above IB_OPCODE() macro for more details */ @@ -112,6 +113,7 @@ enum { IB_OPCODE(RC, FETCH_ADD), IB_OPCODE(RC, SEND_LAST_WITH_INVALIDATE), IB_OPCODE(RC, SEND_ONLY_WITH_INVALIDATE), + IB_OPCODE(RC, ATOMIC_WRITE), /* UC */ IB_OPCODE(UC, SEND_FIRST), diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 975d6e9efbcb..19bed1fb8e55 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -270,6 +270,7 @@ enum ib_device_cap_flags { /* The device supports padding incoming writes to cacheline. */ IB_DEVICE_PCI_WRITE_END_PADDING = IB_UVERBS_DEVICE_PCI_WRITE_END_PADDING, + IB_DEVICE_ATOMIC_WRITE = IB_UVERBS_DEVICE_ATOMIC_WRITE, }; enum ib_kernel_cap_flags { @@ -985,6 +986,7 @@ enum ib_wc_opcode { IB_WC_REG_MR, IB_WC_MASKED_COMP_SWAP, IB_WC_MASKED_FETCH_ADD, + IB_WC_ATOMIC_WRITE = IB_UVERBS_WC_ATOMIC_WRITE, /* * Set value of IB_WC_RECV so consumers can test if a completion is a * receive by testing (opcode & IB_WC_RECV). @@ -1325,6 +1327,7 @@ enum ib_wr_opcode { IB_UVERBS_WR_MASKED_ATOMIC_CMP_AND_SWP, IB_WR_MASKED_ATOMIC_FETCH_AND_ADD = IB_UVERBS_WR_MASKED_ATOMIC_FETCH_AND_ADD, + IB_WR_ATOMIC_WRITE = IB_UVERBS_WR_ATOMIC_WRITE, /* These are kernel only and can not be issued by userspace */ IB_WR_REG_MR = 0x20,