From patchwork Tue Aug 2 19:15:11 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Ledford X-Patchwork-Id: 1029932 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter2.kernel.org (8.14.4/8.14.4) with ESMTP id p72JFnuO023527 for ; Tue, 2 Aug 2011 19:15:49 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754286Ab1HBTPr (ORCPT ); Tue, 2 Aug 2011 15:15:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38861 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754258Ab1HBTPr (ORCPT ); Tue, 2 Aug 2011 15:15:47 -0400 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id p72JFCij011739 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 2 Aug 2011 15:15:13 -0400 Received: from schwoop.rdu.redhat.com (unused [10.11.230.202] (may be forged)) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id p72JFBHG014444 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 2 Aug 2011 15:15:12 -0400 Message-ID: <4E384CBF.6080905@redhat.com> Date: Tue, 02 Aug 2011 15:15:11 -0400 From: Doug Ledford User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20110707 Thunderbird/5.0 MIME-Version: 1.0 To: Roland Dreier , "linux-rdma@vger.kernel.org" Subject: [Patch libibverbs-1.1.5] Fix broken compat assumption on ppc arch X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter2.kernel.org [140.211.167.43]); Tue, 02 Aug 2011 19:15:49 +0000 (UTC) In the compat-1.0 wrapper, the post_send wrapper assumes that the layout of the tail end of the ibv_send_wr struct is the same in version 1.0 and 1.1. Unfortunately, padding of the struct makes this untrue depending on the architecture. In my case, it was ppc that was failing to compile because we were triggering the gcc built in memcpy check. We were scribbling past the end of the structure of the new structure because the old structure has alignment padding that the new one doesn't. There are two spots that are padded in the version 1.0 ibv_send_wr struct, 4 bytes before sg_list, and 4 more bytes before wr. So this patch conditionalizes whether or not we use memcpy on the basis of data that will be reduced to constants at compile time, so even though there is an if statement, it gets reduced to just the proper method for the given arch at compile time. And the patch will attempt a simple memcpy if possible, or two memcpy's in the case that wr needed padding to be aligned (as is the case on ppc), and fall back to field based copy just in case this is compiled on some arch with really odd field alignment characteristics. Signed-off-by: Doug Ledford diff -up libibverbs-1.1.5/src/compat-1_0.c.memcpy libibverbs-1.1.5/src/compat-1_0.c --- libibverbs-1.1.5/src/compat-1_0.c.memcpy 2011-06-28 17:13:45.000000000 -0400 +++ libibverbs-1.1.5/src/compat-1_0.c 2011-08-02 12:07:20.114511405 -0400 @@ -350,8 +350,46 @@ static int post_send_wrapper_1_0(struct real_wr->wr_id = w->wr_id; real_wr->next = NULL; - memcpy(&real_wr->sg_list, &w->sg_list, - sizeof *w - offsetof(struct ibv_send_wr, sg_list)); +#define TEST_SIZE_2_POINT(f1, f2) \ +((offsetof(struct ibv_send_wr, f1) - offsetof(struct ibv_send_wr, f2)) \ +== offsetof(struct ibv_send_wr_1_0, f1) - offsetof(struct ibv_send_wr_1_0, f2)) +#define TEST_SIZE_TO_END(f1) \ +((sizeof(struct ibv_send_wr) - offsetof(struct ibv_send_wr, f1)) == \ + (sizeof(struct ibv_send_wr_1_0) - offsetof(struct ibv_send_wr_1_0, f1))) + + if (TEST_SIZE_TO_END (sg_list)) + memcpy(&real_wr->sg_list, &w->sg_list, sizeof *real_wr + - offsetof(struct ibv_send_wr, sg_list)); + else if (TEST_SIZE_2_POINT (imm_data, sg_list) && + TEST_SIZE_TO_END (wr)) { + /* we have alignment up to wr, but padding between + * imm_data and wr, and we know wr itself is the + * same size */ + memcpy(&real_wr->sg_list, &w->sg_list, + offsetof(struct ibv_send_wr, imm_data) - + offsetof(struct ibv_send_wr, sg_list) + + sizeof real_wr->imm_data); + memcpy(&real_wr->wr, &w->wr, sizeof real_wr->wr); + } else { + real_wr->sg_list = w->sg_list; + real_wr->num_sge = w->num_sge; + real_wr->opcode = w->opcode; + real_wr->send_flags = w->send_flags; + real_wr->imm_data = w->imm_data; + if (TEST_SIZE_TO_END (wr)) + memcpy(&real_wr->wr, &w->wr, + sizeof real_wr->wr); + else { + real_wr->wr.atomic.remote_addr = + w->wr.atomic.remote_addr; + real_wr->wr.atomic.compare_add = + w->wr.atomic.compare_add; + real_wr->wr.atomic.swap = + w->wr.atomic.swap; + real_wr->wr.atomic.rkey = + w->wr.atomic.rkey; + } + } if (is_ud) real_wr->wr.ud.ah = w->wr.ud.ah->real_ah;