From patchwork Thu May 20 11:34:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 12269759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCBC7C433ED for ; Thu, 20 May 2021 11:37:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B5FDD60FEE for ; Thu, 20 May 2021 11:37:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241899AbhETLia (ORCPT ); Thu, 20 May 2021 07:38:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38789 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241356AbhETLg0 (ORCPT ); Thu, 20 May 2021 07:36:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621510503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3OVBDZCb6MIQPs6dPbIQhHKwZvFJuf68f56hTGwMOrY=; b=GFES1iEGHyJvP02hCPLRfG5rFWckEk7gWxgPmhpF3oHyOllqdbf/lGwtnNdw95CydPWomw svlM41dDC8V0tYtXMciHrM8RaB8v3le/M3GZPLxRhgmFDm8QWDmfcznMRJy1zFXJXjTidr Mgo4+JgzUgVs2Ril6eq1Ufk+Ff1SXjI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-466-UKu6pWHXNzuw252w3cqO5g-1; Thu, 20 May 2021 07:34:58 -0400 X-MC-Unique: UKu6pWHXNzuw252w3cqO5g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 535E7100960F; Thu, 20 May 2021 11:34:57 +0000 (UTC) Received: from gondolin.fritz.box (ovpn-113-166.ams2.redhat.com [10.36.113.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id A3D5D10016F2; Thu, 20 May 2021 11:34:55 +0000 (UTC) From: Cornelia Huck To: Heiko Carstens , Vasily Gorbik , Christian Borntraeger Cc: linux-s390@vger.kernel.org, kvm@vger.kernel.org, Eric Farman , Cornelia Huck , Matthew Rosato Subject: [PULL 1/3] vfio-ccw: Check initialized flag in cp_init() Date: Thu, 20 May 2021 13:34:48 +0200 Message-Id: <20210520113450.267893-2-cohuck@redhat.com> In-Reply-To: <20210520113450.267893-1-cohuck@redhat.com> References: <20210520113450.267893-1-cohuck@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Eric Farman We have a really nice flag in the channel_program struct that indicates if it had been initialized by cp_init(), and use it as a guard in the other cp accessor routines, but not for a duplicate call into cp_init(). The possibility of this occurring is low, because that flow is protected by the private->io_mutex and FSM CP_PROCESSING state. But then why bother checking it in (for example) cp_prefetch() then? Let's just be consistent and check for that in cp_init() too. Fixes: 71189f263f8a3 ("vfio-ccw: make it safe to access channel programs") Signed-off-by: Eric Farman Reviewed-by: Cornelia Huck Acked-by: Matthew Rosato Message-Id: <20210511195631.3995081-2-farman@linux.ibm.com> Signed-off-by: Cornelia Huck --- drivers/s390/cio/vfio_ccw_cp.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index b9febc581b1f..8d1b2771c1aa 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -638,6 +638,10 @@ int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 1); int ret; + /* this is an error in the caller */ + if (cp->initialized) + return -EBUSY; + /* * We only support prefetching the channel program. We assume all channel * programs executed by supported guests likewise support prefetching. From patchwork Thu May 20 11:34:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 12269763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F2EEC43460 for ; Thu, 20 May 2021 11:37:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 45F1C61090 for ; Thu, 20 May 2021 11:37:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236507AbhETLin (ORCPT ); Thu, 20 May 2021 07:38:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:25386 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241448AbhETLgb (ORCPT ); Thu, 20 May 2021 07:36:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621510506; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4kDItSm8+8o3sO4OmrJFnk4qK9G/voUO8JK5SZMKiGU=; b=SNgR9SHfyM7sAPyO9bP6CzxyUilblZ18K2TkePcacgzi+nEuLhGqJVW+CI/q9WWuF7GsWc l71+dBY5cQgmlNhHEu1RICBCXG0OhlV4v/C/+oN4IcZlpIqYo5ue4rDyWgLlWtDw4tk+bS psvuRgsux5t4QgE9vGl42Jozr8LrsAo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-111-TK4VJ8iQNha_3HHoNYtr7g-1; Thu, 20 May 2021 07:35:03 -0400 X-MC-Unique: TK4VJ8iQNha_3HHoNYtr7g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 036AD1923787; Thu, 20 May 2021 11:35:02 +0000 (UTC) Received: from gondolin.fritz.box (ovpn-113-166.ams2.redhat.com [10.36.113.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F53910013C1; Thu, 20 May 2021 11:34:57 +0000 (UTC) From: Cornelia Huck To: Heiko Carstens , Vasily Gorbik , Christian Borntraeger Cc: linux-s390@vger.kernel.org, kvm@vger.kernel.org, Eric Farman , Cornelia Huck , Matthew Rosato Subject: [PULL 2/3] vfio-ccw: Reset FSM state to IDLE inside FSM Date: Thu, 20 May 2021 13:34:49 +0200 Message-Id: <20210520113450.267893-3-cohuck@redhat.com> In-Reply-To: <20210520113450.267893-1-cohuck@redhat.com> References: <20210520113450.267893-1-cohuck@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Eric Farman When an I/O request is made, the fsm_io_request() routine moves the FSM state from IDLE to CP_PROCESSING, and then fsm_io_helper() moves it to CP_PENDING if the START SUBCHANNEL received a cc0. Yet, the error case to go from CP_PROCESSING back to IDLE is done after the FSM call returns. Let's move this up into the FSM proper, to provide some better symmetry when unwinding in this case. Signed-off-by: Eric Farman Reviewed-by: Cornelia Huck Acked-by: Matthew Rosato Message-Id: <20210511195631.3995081-3-farman@linux.ibm.com> Signed-off-by: Cornelia Huck --- drivers/s390/cio/vfio_ccw_fsm.c | 1 + drivers/s390/cio/vfio_ccw_ops.c | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c index 23e61aa638e4..e435a9cd92da 100644 --- a/drivers/s390/cio/vfio_ccw_fsm.c +++ b/drivers/s390/cio/vfio_ccw_fsm.c @@ -318,6 +318,7 @@ static void fsm_io_request(struct vfio_ccw_private *private, } err_out: + private->state = VFIO_CCW_STATE_IDLE; trace_vfio_ccw_fsm_io_request(scsw->cmd.fctl, schid, io_region->ret_code, errstr); } diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c index 491a64c61fff..c57d2a7f0919 100644 --- a/drivers/s390/cio/vfio_ccw_ops.c +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -279,8 +279,6 @@ static ssize_t vfio_ccw_mdev_write_io_region(struct vfio_ccw_private *private, } vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_IO_REQ); - if (region->ret_code != 0) - private->state = VFIO_CCW_STATE_IDLE; ret = (region->ret_code != 0) ? region->ret_code : count; out_unlock: From patchwork Thu May 20 11:34:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 12269761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07BA3C43461 for ; Thu, 20 May 2021 11:37:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DEE1F60E0B for ; Thu, 20 May 2021 11:37:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241363AbhETLij (ORCPT ); Thu, 20 May 2021 07:38:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51074 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241559AbhETLgc (ORCPT ); Thu, 20 May 2021 07:36:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621510510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/Au9nWIMCoQk6wVhEc03Ii+LBKp3hBKUFlVdTr5dd6w=; b=Pf/WCO+14mnhKSYmgXvM3RW5kZJUShxM+qsvwwmLg1jZnRScg48Gw2+lYBIMd38PEtuj2r /zkScVpYmC92CdASWu4XY9ohS+MEiBaNUyan6t85h9S9sRonPL/rFPGFX5VBGCuux80tg2 lwTZJPrwNtySIlkXQeUwpu47N1sZnMI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-590-lE_kh2uONVSGYLKc38e_pA-1; Thu, 20 May 2021 07:35:06 -0400 X-MC-Unique: lE_kh2uONVSGYLKc38e_pA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F32D5107ACF2; Thu, 20 May 2021 11:35:03 +0000 (UTC) Received: from gondolin.fritz.box (ovpn-113-166.ams2.redhat.com [10.36.113.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4F44F10013C1; Thu, 20 May 2021 11:35:02 +0000 (UTC) From: Cornelia Huck To: Heiko Carstens , Vasily Gorbik , Christian Borntraeger Cc: linux-s390@vger.kernel.org, kvm@vger.kernel.org, Eric Farman , Matthew Rosato , Cornelia Huck Subject: [PULL 3/3] vfio-ccw: Serialize FSM IDLE state with I/O completion Date: Thu, 20 May 2021 13:34:50 +0200 Message-Id: <20210520113450.267893-4-cohuck@redhat.com> In-Reply-To: <20210520113450.267893-1-cohuck@redhat.com> References: <20210520113450.267893-1-cohuck@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Eric Farman Today, the stacked call to vfio_ccw_sch_io_todo() does three things: 1) Update a solicited IRB with CP information, and release the CP if the interrupt was the end of a START operation. 2) Copy the IRB data into the io_region, under the protection of the io_mutex 3) Reset the vfio-ccw FSM state to IDLE to acknowledge that vfio-ccw can accept more work. The trouble is that step 3 is (A) invoked for both solicited and unsolicited interrupts, and (B) sitting after the mutex for step 2. This second piece becomes a problem if it processes an interrupt for a CLEAR SUBCHANNEL while another thread initiates a START, thus allowing the CP and FSM states to get out of sync. That is: CPU 1 CPU 2 fsm_do_clear() fsm_irq() fsm_io_request() vfio_ccw_sch_io_todo() fsm_io_helper() Since the FSM state and CP should be kept in sync, let's make a note when the CP is released, and rely on that as an indication that the FSM should also be reset at the end of this routine and open up the device for more work. Signed-off-by: Eric Farman Acked-by: Matthew Rosato Reviewed-by: Cornelia Huck Message-Id: <20210511195631.3995081-4-farman@linux.ibm.com> Signed-off-by: Cornelia Huck --- drivers/s390/cio/vfio_ccw_drv.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c index 8c625b530035..9b61e9b131ad 100644 --- a/drivers/s390/cio/vfio_ccw_drv.c +++ b/drivers/s390/cio/vfio_ccw_drv.c @@ -86,6 +86,7 @@ static void vfio_ccw_sch_io_todo(struct work_struct *work) struct vfio_ccw_private *private; struct irb *irb; bool is_final; + bool cp_is_finished = false; private = container_of(work, struct vfio_ccw_private, io_work); irb = &private->irb; @@ -94,14 +95,21 @@ static void vfio_ccw_sch_io_todo(struct work_struct *work) (SCSW_ACTL_DEVACT | SCSW_ACTL_SCHACT)); if (scsw_is_solicited(&irb->scsw)) { cp_update_scsw(&private->cp, &irb->scsw); - if (is_final && private->state == VFIO_CCW_STATE_CP_PENDING) + if (is_final && private->state == VFIO_CCW_STATE_CP_PENDING) { cp_free(&private->cp); + cp_is_finished = true; + } } mutex_lock(&private->io_mutex); memcpy(private->io_region->irb_area, irb, sizeof(*irb)); mutex_unlock(&private->io_mutex); - if (private->mdev && is_final) + /* + * Reset to IDLE only if processing of a channel program + * has finished. Do not overwrite a possible processing + * state if the final interrupt was for HSCH or CSCH. + */ + if (private->mdev && cp_is_finished) private->state = VFIO_CCW_STATE_IDLE; if (private->io_trigger)