From patchwork Sat Oct 15 06:37:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Yang X-Patchwork-Id: 13007603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A1CEC4332F for ; Sat, 15 Oct 2022 06:38:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229513AbiJOGiZ (ORCPT ); Sat, 15 Oct 2022 02:38:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229556AbiJOGiX (ORCPT ); Sat, 15 Oct 2022 02:38:23 -0400 Received: from esa5.fujitsucc.c3s2.iphmx.com (esa5.fujitsucc.c3s2.iphmx.com [68.232.159.76]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63D77638D5 for ; Fri, 14 Oct 2022 23:38:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj1; t=1665815902; x=1697351902; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=bHF9YOc8aqmkl9KahjSnBQtYtFzT3N3NqxHz4aq1LHE=; b=PyrgvYW9rJAgjXGxmKaM0OOvLTwJ5CEtBnGUevGF9JQtmDD/8gEJ7CHu yC5RNvI+IjJyUhl3lJDhHkALuF9diaSGXgFj8njECFB2k5Ien25uRChfy InIaFBtaxigQVggJNhLXSlGU0CAESZ7eEJiF874YKxntEYPCiSFmqdygd KObNckkmXCoKwHxb1OBjvyw3wjm5Mo/CeYJ/VgARl6FiPFVnKhzs+I998 5T+EE9bGUJHuxDN8MVvK514mqoDwIdLV6hCrotYdBdOqJPC0nSSUQxpzH rXcA9AzFArWTWqig1/nN424hrDbeX8Jfp6S88TeviFiK2kk2Q+v32o9k6 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10500"; a="67758726" X-IronPort-AV: E=Sophos;i="5.95,186,1661785200"; d="scan'208";a="67758726" Received: from mail-tycjpn01lp2172.outbound.protection.outlook.com (HELO JPN01-TYC-obe.outbound.protection.outlook.com) ([104.47.23.172]) by ob1.fujitsucc.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Oct 2022 15:37:15 +0900 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WNHF97Pxxgc1K7fUK2ix7JO00Aic5YmZwXxC68eC9PK/J2BZxKdLJk3OJ3PhO+3OYU1GisXSRNvJzm9Uq/zN14rpnU+PNoKpEVWn23tXo1vIR4XTLvMjtsnuVxkzNY9lxIfItIzcLZKf/4x6Op57sthyKavvHFJeMvZZnuCcecheXennfqEEzBK+kCNUhByOJOLwOC4QPMfvNZ3HMZ/Tl+pJ8ImDEnR4CXiPTlokBLyqcOCfp8Uvol7G2jrnfVLGU7+sThhpoc5thxLh9JLrtSRA72NgvZUD7H8dn5evPG5qH/nGnlklHh27xAy22n5LbS9aIrFhGPOjbSmlwn6yAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bHF9YOc8aqmkl9KahjSnBQtYtFzT3N3NqxHz4aq1LHE=; b=b/oBULsUdcA9Xx/ZGzPFtIL7a5OcI7vDoD+qq7yt3j25NHNQJVyrP+BjXRRrsdWKJ+WfGQJKwm+HCfu8UxlRRoLq8lgg4Ey0CTfRqlQbZX6AswN0lH6py4qCECxf/ZJaUQeb/K43YyiCzIgppkGVHevuYH8L9lk2GTzBE5AOGU/DDFQerplBqNclNGESxWLQtcwFspj81Zu2ckl4vW1Qqn74MDT+/0wT1VmWskVDJ73+4mFkfW5fymhiK5cjAyTKdLIHq9wBf9MlBhCjvsCrSZMxUlqHfLRIBI5F6k114sYgCyQ1KQt3EWoHDlMF2Z8x500/pkq85ZPNtbFW9lGdLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fujitsu.com; dmarc=pass action=none header.from=fujitsu.com; dkim=pass header.d=fujitsu.com; arc=none Received: from OS3PR01MB9499.jpnprd01.prod.outlook.com (2603:1096:604:1c8::5) by OSZPR01MB8661.jpnprd01.prod.outlook.com (2603:1096:604:185::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5723.29; Sat, 15 Oct 2022 06:37:08 +0000 Received: from OS3PR01MB9499.jpnprd01.prod.outlook.com ([fe80::5989:c90d:abf4:e100]) by OS3PR01MB9499.jpnprd01.prod.outlook.com ([fe80::5989:c90d:abf4:e100%6]) with mapi id 15.20.5723.029; Sat, 15 Oct 2022 06:37:08 +0000 From: "yangx.jy@fujitsu.com" To: "linux-rdma@vger.kernel.org" , "jgg@nvidia.com" , "rpearsonhpe@gmail.com" CC: "leon@kernel.org" , "lizhijian@fujitsu.com" , "y-goto@fujitsu.com" , "zyjzyj2000@gmail.com" , "yangx.jy@fujitsu.com" Subject: [PATCH v6 6/8] RDMA/rxe: Make responder support atomic write on RC service Thread-Topic: [PATCH v6 6/8] RDMA/rxe: Make responder support atomic write on RC service Thread-Index: AQHY4GCLjMFWuCMSS0q/oYGR4F4sjg== Date: Sat, 15 Oct 2022 06:37:07 +0000 Message-ID: <20221015063648.52285-7-yangx.jy@fujitsu.com> References: <20221015063648.52285-1-yangx.jy@fujitsu.com> In-Reply-To: <20221015063648.52285-1-yangx.jy@fujitsu.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.34.1 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=fujitsu.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: OS3PR01MB9499:EE_|OSZPR01MB8661:EE_ x-ms-office365-filtering-correlation-id: b38c8cb3-e52b-4473-bd49-08daae77aeb7 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: K7l8yqPGaCTqGA6aj+OyUuDNTbTy6rQPJ0jjGzbEoeTuXV8jft0/1NX3oXnGI61Ty0tP2meT3uSlqNAro60mvtveDm2v/fOauubzm0QqNiEtxFlzEJ/CxCG7zVXn42azvET+u5D1TxUh/etm/y92FZiXvUvXqtih5eJU3yeE20OEW5g/JofXaigqLItCXN54PwKkPAJK5PfH2DT8gtlsVVcewwKq3rn3grR7oUYLT3t4yiQPaMcjYwepHvHd2M/oNi9HJtN3WvLzFVzpK+RrbErvxktK33s9Lt9Be79uMo6B9RTlfkpsrRkWz5UuHofWEdEt10dCqq5X35LBa3iubjIqgEkeygg98ltcOML9m3z/UqqbJBqPem51xayBZuhpf+vtEdL8Y9c50sSIZEP+iUFbN5W6w6tMr2VZ+6pHyT8VoZhSJz/kYrazBL38I/Tw1SvgC+Ss5DRDvmIvhDJKuvGCgtReTFQEsaVk6UlaOL1nC8X26PmSgGFQQg536mfo6nQrbqLAgFtI4A65X3cs0dm4ufW92pRdL0Hpk0JBwqyPLauaAjHHcJmyR+KzehiowDTgFQ/ukinxC9pL5ZN6u0mdoEYggAxde2bO2d7z2fRvDYfCFzy2nONdMXpKaCo06dvosOzcycZbulKPc8a+NaUkW61GkTAZxLEU3FppeDzsJsg7u+IaCkT05ovx6gFCJzwELegwewfBxyayBTyg/jRSpHfT70enyt0puyIJBW35aF1jrOdvHo+cKYCgvI+uAa+MhO9Lzu7N9ru//2ijqs+8SlbiyeiuBJ40FUnSUeo= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:OS3PR01MB9499.jpnprd01.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(136003)(346002)(376002)(366004)(396003)(451199015)(1590799012)(6486002)(45080400002)(85182001)(316002)(86362001)(38100700002)(36756003)(5660300002)(110136005)(26005)(6512007)(54906003)(478600001)(122000001)(83380400001)(8936002)(82960400001)(91956017)(1580799009)(66946007)(76116006)(2616005)(107886003)(66556008)(66476007)(38070700005)(41300700001)(71200400001)(6506007)(66446008)(64756008)(8676002)(2906002)(4326008)(1076003)(186003);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?eucgb2312_cn?b?aDN2TExJTWhMMDhDMy9XZmFp?= =?eucgb2312_cn?b?WVhYUlk4TW1pWnFiamxoN2VtcjRPTG00VldnTlZZYUNJSzhWOXQ2c280eno5Wmts?= =?eucgb2312_cn?b?Q3Q4NndzRG9RM2l4R2tVVEY1cDJFMnRTS2t3ZFRUek53THNmbXdKaXcvUnhyd0JS?= =?eucgb2312_cn?b?aEM4SUMyVzAreWFDUnV0Z2RBZG5ER3dxbStQc3RocUt6U0MrWFNrS3pYSnUyU2NJ?= =?eucgb2312_cn?b?c0tCVXZpVFFLc21UZWMySkxPRUJ3NE9GbXFRSnVDMW5vcU1Mb0VPanZNM3NEa053?= =?eucgb2312_cn?b?bTZUZHJpVjFsQUFyMUFoTmxKY2liQmxpRkcrUGljZWRGOENVZFp0VVNlQTVDZkcw?= =?eucgb2312_cn?b?clpwYUhDdjFxUHV4SFFDUlZ5akRtY2pwT1EzanN6MmhuZkM0SmtHemN4UUlEdk5O?= =?eucgb2312_cn?b?cGtWQWpZM2I4dEJNYys2eTRmSWlmK1AvTVBsRU01VE1BWWNsQ3F2cFoxazkzRTZJ?= =?eucgb2312_cn?b?UExCNHhPeXZCRW9sWkdoMWdhaThSeEJac1VXbm9HMVN5WmZUV0pVSHIvS2NLKzhP?= =?eucgb2312_cn?b?aHltckQ1Y1ZROU03cXFPTGdEMVlReWtaa2hDbXpLTHRJV0NreC9UMTJFNXloZFdl?= =?eucgb2312_cn?b?WHpMb0FyUGRTTWlpYlc4c1RLb0VCVkVIUzRHOEF4a0dvNXlKR0hMMitZRG1BY244?= =?eucgb2312_cn?b?Z1BDa3NscjNSdExxUUU4S2JHSzhkRnhwUU1OMXRVNEhNYmNVaGNhRHNFZCtPZFlS?= =?eucgb2312_cn?b?ZlMxUzN2amRNRzlPcXArdFdCMHB6OE5MU0duSXVjd1pxRGljMG54QXZWbk8ya0Yy?= =?eucgb2312_cn?b?S0phVXdoN1pGMWVMOWJSczBld3kyM3RZRWtlUW1ySExOQXN2bWRUVkg2aE1QVVJD?= =?eucgb2312_cn?b?MnB3UURib1NnbzBRK3E0aVlXWHJIaUhLRjYyZFd6K2twSnJhOWdnSFNSb202TmIv?= =?eucgb2312_cn?b?bVo3Z3NMbW9lTXFGTzBzKzZqU0JmRTQxL0dlR2Rha2NhQzBub0V1L0daekRQdU5y?= =?eucgb2312_cn?b?dStldmpvZVlOT0dNdTBjMTY0S3AySy9ROW1aRDVZQmhpZDYwSTA5OGhxYjl5SHVi?= =?eucgb2312_cn?b?MWd1WmZBTFZTdE9lK0lVVWZaNnRkbGcwZ0FiK1hGcU4razVjbXpjL0tXYkR1T2Ir?= =?eucgb2312_cn?b?ZFlCd3NYcERWRlhNMjdWZ1JkemF4WVFKVzgyL1laUit3YUdTZ200SWRnOVlqWHg0?= =?eucgb2312_cn?b?T0Y3bE96UXdCajJyTVNJdzF6Q21IS0crakpxYzZvTTQyTXJGMDhXb1lPWVZ0aFhR?= =?eucgb2312_cn?b?QmIzeFFyNGNUY3Z3ZmQ0YlRBY1dFK3k2UUJ2blYrZ2orcXRqVENNSHpKakFWWjV0?= =?eucgb2312_cn?b?QkMrcjd0Z0JQcjNUTjBScnpDUUh2L2RhU2ZNeG5GZWhQME42bGVGSnRXL2ptWDVC?= =?eucgb2312_cn?b?bmdKaXMyS1YwM3IxODVBZnI4clVNRXY1cDZpRDZkRzdlQVZVaU5SSkhSRTY4Y1h2?= =?eucgb2312_cn?b?R2RlOVJFVmt6dzZIRzI5Wm1MVWZIVk9wcEQ5aGdPS1FWVnJNMjI2MXJVNHBaOFdM?= =?eucgb2312_cn?b?WFF0ZkN1UmdmRnVXVUdhRDlWRlZ6R0kxdW9vMDhOUERWTFJVSUVFaDNjSXNLM3lw?= =?eucgb2312_cn?b?Tit3cTgzZVY2Uzkwdkwzdzl6ZWNmWjJkZ3N1RUF2alpTRkkvSkgwTHVIUDhySHFv?= =?eucgb2312_cn?b?VjNtYVIrbSsxdlZCWkxRcGkwaTRsVVlJazYwTkVnM1JtOU9YYXl3RXlNYlFncU56?= =?eucgb2312_cn?b?aTVXSHpOODQxaklnVUZEWXBMRzFCY2svL1Z3eTZyYnBOWGFKUlNwRzl4UUNQbVhW?= =?eucgb2312_cn?b?T1ptb0lzN3hQTjlRdERxemtjUkU2ZzhOS3RSUThYb0xsdUJFcGNXNUlnNGtLdzRS?= =?eucgb2312_cn?b?cVBZUzhCZ3BVODVrc3F5ZEFkbDAxMk5Qb20vVU9kbG5UV1o0R1dXL3VrVGtZUTJz?= =?eucgb2312_cn?b?UjZKTld3NDVlZDdza01hSFpOUC80THlaeGJ6Y0VJaVZvZDBlSFNwTmNPVTQ4Y3hS?= =?eucgb2312_cn?b?SHpobXV0SHBGV2pOVFpmaFJiQXBvUnB6aGRLdzFQN3JGM2VMblVQQXdRa3h4b3RR?= =?eucgb2312_cn?b?TXlWTHk1MnU5dWNHcjhXQVg4WTlHN1lVRDZldDFYcmtYMXErcGlkZ1B0b3EyYk9N?= =?eucgb2312_cn?b?alR2aFl1Um5mdVZRZlV2ZXVGak5qOUd1dkFmZXVCZTdTQWVqNUR2SmZNNDRlVThM?= =?eucgb2312_cn?b?OERDcTc1L3RBbkxYemlwbWRkNlE9PQ==?= MIME-Version: 1.0 X-OriginatorOrg: fujitsu.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: OS3PR01MB9499.jpnprd01.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b38c8cb3-e52b-4473-bd49-08daae77aeb7 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Oct 2022 06:37:08.1496 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tIifPGBnY/LN8xWk8tH5q28cxKSy9Ut8LFQReGV0+I8WpdBWiJdp16DW6wDbjqWbiOCuQIP/GhdOUdG2m51yrw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OSZPR01MB8661 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make responder process an atomic write request and send a read response on RC service. Signed-off-by: Xiao Yang --- drivers/infiniband/sw/rxe/rxe_resp.c | 84 ++++++++++++++++++++++++++-- 1 file changed, 79 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index ed5a09e86417..7d313d9a8d88 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -22,6 +22,7 @@ enum resp_states { RESPST_EXECUTE, RESPST_READ_REPLY, RESPST_ATOMIC_REPLY, + RESPST_ATOMIC_WRITE_REPLY, RESPST_COMPLETE, RESPST_ACKNOWLEDGE, RESPST_CLEANUP, @@ -57,6 +58,7 @@ static char *resp_state_name[] = { [RESPST_EXECUTE] = "EXECUTE", [RESPST_READ_REPLY] = "READ_REPLY", [RESPST_ATOMIC_REPLY] = "ATOMIC_REPLY", + [RESPST_ATOMIC_WRITE_REPLY] = "ATOMIC_WRITE_REPLY", [RESPST_COMPLETE] = "COMPLETE", [RESPST_ACKNOWLEDGE] = "ACKNOWLEDGE", [RESPST_CLEANUP] = "CLEANUP", @@ -260,7 +262,7 @@ static enum resp_states check_op_valid(struct rxe_qp *qp, case IB_QPT_RC: if (((pkt->mask & RXE_READ_MASK) && !(qp->attr.qp_access_flags & IB_ACCESS_REMOTE_READ)) || - ((pkt->mask & RXE_WRITE_MASK) && + ((pkt->mask & (RXE_WRITE_MASK | RXE_ATOMIC_WRITE_MASK)) && !(qp->attr.qp_access_flags & IB_ACCESS_REMOTE_WRITE)) || ((pkt->mask & RXE_ATOMIC_MASK) && !(qp->attr.qp_access_flags & IB_ACCESS_REMOTE_ATOMIC))) { @@ -364,7 +366,7 @@ static enum resp_states check_resource(struct rxe_qp *qp, } } - if (pkt->mask & RXE_READ_OR_ATOMIC_MASK) { + if (pkt->mask & (RXE_READ_OR_ATOMIC_MASK | RXE_ATOMIC_WRITE_MASK)) { /* it is the requesters job to not send * too many read/atomic ops, we just * recycle the responder resource queue @@ -415,7 +417,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, enum resp_states state; int access; - if (pkt->mask & RXE_READ_OR_WRITE_MASK) { + if (pkt->mask & (RXE_READ_OR_WRITE_MASK | RXE_ATOMIC_WRITE_MASK)) { if (pkt->mask & RXE_RETH_MASK) { qp->resp.va = reth_va(pkt); qp->resp.offset = 0; @@ -483,7 +485,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, goto err; } - if (pkt->mask & RXE_WRITE_MASK) { + if (pkt->mask & (RXE_WRITE_MASK | RXE_ATOMIC_WRITE_MASK)) { if (resid > mtu) { if (pktlen != mtu || bth_pad(pkt)) { state = RESPST_ERR_LENGTH; @@ -583,6 +585,7 @@ static struct resp_res *rxe_prepare_res(struct rxe_qp *qp, res->state = rdatm_res_state_new; break; case RXE_ATOMIC_MASK: + case RXE_ATOMIC_WRITE_MASK: res->first_psn = pkt->psn; res->last_psn = pkt->psn; res->cur_psn = pkt->psn; @@ -652,6 +655,55 @@ static enum resp_states atomic_reply(struct rxe_qp *qp, return ret; } +static enum resp_states atomic_write_reply(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + u64 src, *dst; + struct resp_res *res = qp->resp.res; + struct rxe_mr *mr = qp->resp.mr; + int payload = payload_size(pkt); + + if (!res) { + res = rxe_prepare_res(qp, pkt, RXE_ATOMIC_WRITE_MASK); + qp->resp.res = res; + } + + if (!res->replay) { +#ifdef CONFIG_64BIT + if (mr->state != RXE_MR_STATE_VALID) + return RESPST_ERR_RKEY_VIOLATION; + + memcpy(&src, payload_addr(pkt), payload); + + dst = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, payload); + /* check vaddr is 8 bytes aligned. */ + if (!dst || (uintptr_t)dst & 7) + return RESPST_ERR_MISALIGNED_ATOMIC; + + /* Do atomic write after all prior operations have completed */ + smp_store_release(dst, src); + + /* decrease resp.resid to zero */ + qp->resp.resid -= sizeof(payload); + + qp->resp.msn++; + + /* next expected psn, read handles this separately */ + qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK; + qp->resp.ack_psn = qp->resp.psn; + + qp->resp.opcode = pkt->opcode; + qp->resp.status = IB_WC_SUCCESS; + + return RESPST_ACKNOWLEDGE; +#else + return RESPST_ERR_UNSUPPORTED_OPCODE; +#endif /* CONFIG_64BIT */ + } + + return RESPST_ACKNOWLEDGE; +} + static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp, struct rxe_pkt_info *ack, int opcode, @@ -888,6 +940,8 @@ static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt) return RESPST_READ_REPLY; } else if (pkt->mask & RXE_ATOMIC_MASK) { return RESPST_ATOMIC_REPLY; + } else if (pkt->mask & RXE_ATOMIC_WRITE_MASK) { + return RESPST_ATOMIC_WRITE_REPLY; } else { /* Unreachable */ WARN_ON_ONCE(1); @@ -1061,6 +1115,19 @@ static int send_atomic_ack(struct rxe_qp *qp, u8 syndrome, u32 psn) return ret; } +static int send_read_response_ack(struct rxe_qp *qp, u8 syndrome, u32 psn) +{ + int ret = send_common_ack(qp, syndrome, psn, + IB_OPCODE_RC_RDMA_READ_RESPONSE_ONLY, + "RDMA READ response of length zero ACK"); + + /* have to clear this since it is used to trigger + * long read replies + */ + qp->resp.res = NULL; + return ret; +} + static enum resp_states acknowledge(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { @@ -1071,6 +1138,8 @@ static enum resp_states acknowledge(struct rxe_qp *qp, send_ack(qp, qp->resp.aeth_syndrome, pkt->psn); else if (pkt->mask & RXE_ATOMIC_MASK) send_atomic_ack(qp, AETH_ACK_UNLIMITED, pkt->psn); + else if (pkt->mask & RXE_ATOMIC_WRITE_MASK) + send_read_response_ack(qp, AETH_ACK_UNLIMITED, pkt->psn); else if (bth_ack(pkt)) send_ack(qp, AETH_ACK_UNLIMITED, pkt->psn); @@ -1182,7 +1251,9 @@ static enum resp_states duplicate_request(struct rxe_qp *qp, res->replay = 1; res->cur_psn = pkt->psn; qp->resp.res = res; - rc = RESPST_ATOMIC_REPLY; + rc = pkt->mask & RXE_ATOMIC_MASK ? + RESPST_ATOMIC_REPLY : + RESPST_ATOMIC_WRITE_REPLY; goto out; } @@ -1320,6 +1391,9 @@ int rxe_responder(void *arg) case RESPST_ATOMIC_REPLY: state = atomic_reply(qp, pkt); break; + case RESPST_ATOMIC_WRITE_REPLY: + state = atomic_write_reply(qp, pkt); + break; case RESPST_ACKNOWLEDGE: state = acknowledge(qp, pkt); break;