From patchwork Tue Mar 14 20:56:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13175005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 94582C05027 for ; Tue, 14 Mar 2023 20:57:19 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.509834.786432 (Exim 4.92) (envelope-from ) id 1pcBhU-0002VP-9p; Tue, 14 Mar 2023 20:56:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 509834.786432; Tue, 14 Mar 2023 20:56:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhU-0002VI-5f; Tue, 14 Mar 2023 20:56:52 +0000 Received: by outflank-mailman (input) for mailman id 509834; Tue, 14 Mar 2023 20:56:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhT-0001PG-6N for xen-devel@lists.xenproject.org; Tue, 14 Mar 2023 20:56:51 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id bd6fe7e5-c2aa-11ed-87f5-c1b5be75604c; Tue, 14 Mar 2023 21:56:50 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EKchDl017378; Tue, 14 Mar 2023 20:56:33 GMT Received: from eur05-vi1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3pb0520156-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 20:56:33 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by PAXPR03MB7967.eurprd03.prod.outlook.com (2603:10a6:102:21a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Tue, 14 Mar 2023 20:56:30 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e%4]) with mapi id 15.20.6178.026; Tue, 14 Mar 2023 20:56:30 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: bd6fe7e5-c2aa-11ed-87f5-c1b5be75604c ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lWePQK1s5qlonxJcwc17I5daPq2zcSJZH51nt3y3H71wY6gfOKLq7tv4+alx8YCU9eRVukXzgUfWlrteom+Z5z9rAoXjLU/PbrukZnE4wn9mlbavwNA5wOCe2uZQpWZ9yS8k6dTehGDe/mSGXmU85Xrf5V5hgou8Y99SedG1HXBbYnK0z40lwNfjeJo6QKguVUgtLAk6XxdiYNYHf8tz5xNcIvIy8qgHUl6FwQqmQ0ETO4LZmIXosY54SqYfSyk2QQg12UKMV8wQH++WrS4CydO60JynRW5LhBKCJWP6qoMbcf3xfT8D4xwpm7uNMxCu5OCEOKrdCIBMuPlrQoXAxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x7tBqA5OxDzYHo/yEP4dPIJSk0WzvlCUdOtG1wDrd3k=; b=ZDVRNx9HfV+K8gSoeggq42pNM5AnOZyktEKjGKd8DEL0wyB1wNjMPBZM79GQrmNnQ9TQxZ/9dtHNjLE//E8IYPPL54TJwBWo9Nhz4MgypPxMpEbRBKARjUUML2lFV51NPUZWdecn01+RFzMF1IoPW71irrJTPLzoUpBE8G+ioDkGfWDRUqUJKv5ANPRcfTYbfZacGRMmMbDZHN1Srgm2YeJLZOjpL33bKyAndufK3Lf5RwaJ7CtdiXtf+DlBTfVvSHdCvgJyFIptyTLoECrUUA9KtbYTlY4rNQXyLAcibqWb8zG9sNxdWR1UdFl/FEmLgZ2J5v51mDLhoaOMZEi9ug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x7tBqA5OxDzYHo/yEP4dPIJSk0WzvlCUdOtG1wDrd3k=; b=Mwtf9yAUNxP8g5GXxvRmTAHeAYjlgxEDYluXBY6jA2+lM2Wi5D5CJZVlOPjcDsitQjYffAn7qoTIWYlJ18Ihc6KoFkdQHi7ghsQxsc4oD6Rwue1poums9Cd7OVAvB/qpVfmBbEu8EuJoZDccMz9V2GdGqnkRCkn8yCeXkM24bBlUUn25Q4Jtg1EelurOMoHOvQEH9FuTv+3pz+Gxuok2zFA0rxJrU8tAA8gkIoeAj3Rri551/qkxSiM/s7MKIOALbxqtC3j4TGwaupg2dQQTJsVrY6uJd9lEQGo+6+UjeK9GEvDNgh9BvJew3AwU9qay3BR8CG02iCkP3weSpPnG6Q== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v3 1/6] xen: add reference counter support Thread-Topic: [PATCH v3 1/6] xen: add reference counter support Thread-Index: AQHZVrdyBx9pogLqRUq3RbhGdjZNSA== Date: Tue, 14 Mar 2023 20:56:29 +0000 Message-ID: <20230314205612.3703668-2-volodymyr_babchuk@epam.com> References: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> In-Reply-To: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.39.2 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|PAXPR03MB7967:EE_ x-ms-office365-filtering-correlation-id: 42c5c5d9-763a-4a04-b8f4-08db24ce9628 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7laLWOEeMp6OS8XO67yqPwy8RIZn2Ju6KYuewQZeqb1zVx1eCDv5+bJrP9nl1kccTS8xg1szfLiUT8w+U65p5wG4KTZFECbKL/C4tjaJZ85jVQfU1tgyyse385I4/9GINKN2G1aFch9nKt7A+DX44oPKhPF4WxRjonbBYvT3c6Al97BZ+z+X+3OJAipdtlFKCEpZG5sdTA2vHs0+BgeKUWCYyzFcP/FjdEpY4wNDVoz2srkDGeo/GykRJ3vrkPDD1DMzBFmG54G/3eKY+98QeEnZtfkC+7P4HGpR59plCuAUnjV5q7GwPMLFId81XjSzEXfPkW8A7FEj3nvqaSzPAaRHqcqhh0hdYNIWRUJg1ZSQrL6Xkr3e0DTHiY6983L+tYpu33nttZBXlILKpwTPh6Hw+9shGIwn3cR9Lh6teFbwHehTPtJ3AuLVDcyAMXTB2LO5eaTtxiNIPG9GenYPawZVi49kU/gvGY1ZeUaPBd1kEb0NvLNxDLMPbq4uHIFIBoU/WAtAqBjvgK+xuXKpcjd9Ez3RFvr9wgHjFtrikxO0GSBdDhx+qlg+Pahr9Wlx30In2MG8q0c54iXh54IhzLuydefbGqyHj6G0CgI4oW4lA/hiN+hbGlS57ePhKnA/33lf++p7jf4HD0TyOtHQQ22SbJ2iHxQoGfPPOk8aAY6jl15k8hbLgVvo3syTyDkHOTvrKmpve7Msx10gRP18HA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(451199018)(54906003)(316002)(86362001)(36756003)(38070700005)(38100700002)(478600001)(122000001)(55236004)(83380400001)(6512007)(6506007)(2616005)(1076003)(186003)(26005)(5660300002)(6486002)(91956017)(41300700001)(6916009)(71200400001)(76116006)(64756008)(8936002)(8676002)(2906002)(66556008)(66476007)(66946007)(66446008)(4326008);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?raZ1IChzFUgdvCWliX699Al?= =?iso-8859-1?q?FD+urI0p1BrQ9CX2Y954xB8tt3MCbSpmHGVH5nj21bx3oawXXxN/9lmCr+mx?= =?iso-8859-1?q?Yqdw+KQ/wM3AAeuEW7lctUp/sTnkRFaJPO0Sj59WBUa+DjoB3UI2RtB42IXo?= =?iso-8859-1?q?ukJSl3AtzDQ5CmobCiVw4lMwFCf4bzfl+MoN7EujaYUlbWU088wY/Stn49lc?= =?iso-8859-1?q?prKfYLk6ZkUHtw07oq4NfR7xrKXLb6lE8PECjiSuKTDmy+BX8ztJAVR/nOdj?= =?iso-8859-1?q?SUmJFD9heJKveC+YxJ8yrP7SyTZuPdsAoaeD+APq4Wb9Z/AIH+xE+Hp5wrOo?= =?iso-8859-1?q?gi9Lb7WMSxAXqICAVMWJGcPQEdNDUrSzyIcK6ALaTYou+0U4pUonc9VVoeg1?= =?iso-8859-1?q?DnedOuDA6IBTDcjMvJj0zK+ckYbo9ypLfkDoWyb1MK+Q+hN/1z8BbH33okAO?= =?iso-8859-1?q?hZ0AjI83+ttSjGrqy+HmU58k5DUjdO4DMEtBg+E0VgAb7nyO9bMh7+BlrAvR?= =?iso-8859-1?q?0NnDSKJhh1GyoUrPg0lmGxPSzeMJvv6Ty4YZ2z3PUZNUZIH4MUMsXKKgDGWN?= =?iso-8859-1?q?3oG1pMvCtU0bCDqpstj1+m2w1Viz4DBcrQUemcoCYJVlPR74G8/SexUW0N/J?= =?iso-8859-1?q?DzS6PpTUHpYuda+RhRrR2nYunzbixe5b8gerG8/jykymXo1WxIf9Va+g7D3Z?= =?iso-8859-1?q?bDU9Y5CsNYm7Q5hPspzeD3lOWMFHcpc/EzHIOmnJF499ujBZ57tKRRbK22Ji?= =?iso-8859-1?q?+Cp9ickeUBu7s/oyB65fGW/ceTXl48Ob5Jd/FF2v1BBeQLPxCexvjN4Xyevv?= =?iso-8859-1?q?4qp2F5Z8Jtv0CxUNyJwpWUDUfI6j77sQieztRNLFsdbiu4ZpyYDLEFcaOjgF?= =?iso-8859-1?q?VKTsS4/9Hzg4912sLIZ3EAyFBJ+Mox6lBxFjwzooC6WoBeCPhrmDkh346Mbp?= =?iso-8859-1?q?mn9ChTHNIp/oNTv3x53XEUkj2KJK1O4kNqYgiWjTc6mZjh+MCMo9Wp3i5XsU?= =?iso-8859-1?q?YhzOVspp0QEga7gZytM4VWvdjlNvJrUBOmzZwfZ7Nn3vJkvnhmR7J1/XZ2+K?= =?iso-8859-1?q?aZDQ0W/U3yQw+S+ijLZc6UWTGIzqhjplgekNidE8aiz/6SBpHLksRGh+OeY1?= =?iso-8859-1?q?eMi4RykH0FxnE/xQ7ZjI1hbAKxOJIG8EfbBt8zz74Zh2PHwd2tOIxLnl1/zR?= =?iso-8859-1?q?W6lBW2r0nf1Wx9wQmOW9fPWrL62OaMrVVTNb3e9SM6QsvutJUTw2qLa1Dhy3?= =?iso-8859-1?q?O23BxwyEfRdZENjWEtVHFIL9A8IVGZd9EHoSbZwmMas98T0TyPGvxLmxP3HQ?= =?iso-8859-1?q?BrzKimdFQ7aIPxX6Tur5vA1HC30vfHub9XMmbgb7pvBo5CewhEBGIvDL3hO/?= =?iso-8859-1?q?bX58xy8aD+FqJyBivADbfnqBq2jCA3yblssZzC1hIATgWho3QVN0lfIV7yCr?= =?iso-8859-1?q?Q+BzgkYQzVr4HeKElcdtlHD8xleLmysCGubz53ljNoZ/31CkBbqrbe/8gGeB?= =?iso-8859-1?q?h6fIETgpRVh2J8/Ih4EHwp2lsJuHcpKlvu07Qfc90rVvqZz/406njrUs3iAz?= =?iso-8859-1?q?L9zGjEFRzxsMdA1NJoYli+TAaEfXPF7N0jrQnkUHO/O3D6im0ivQEVgpHVMx?= =?iso-8859-1?q?y3EntpZJKTW1guGSvyWOtgs5V6EBEAaeRiUC9tA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 42c5c5d9-763a-4a04-b8f4-08db24ce9628 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2023 20:56:29.2765 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: KKEhdFqOaTIxpw2mLIPTMYtxooX0VFh7q44D1fg7urKY62wNvwxbxlMPTYkWkTViY/kNtj70qnwdEC87FHnVdE2YshXTA+KbiYQAobv1wBw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7967 X-Proofpoint-ORIG-GUID: 9LTZF6K_WfMc60YSOQ2b5VdYZIEbzilF X-Proofpoint-GUID: 9LTZF6K_WfMc60YSOQ2b5VdYZIEbzilF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_14,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=587 priorityscore=1501 suspectscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140168 We can use reference counter to ease up object lifetime management. This patch adds very basic support for reference counters. refcnt should be used in the following way: 1. Protected structure should have refcnt_t field 2. This field should be initialized with refcnt_init() during object construction. 3. If code holds a valid pointer to a structure/object it can increase refcount with refcnt_get(). No additional locking is required. 4. Code should call refcnt_put() before dropping pointer to a protected structure. `destructor` is a call back function that should destruct object and free all resources, including structure protected itself. Destructor will be called if reference counter reaches zero. 5. If code does not hold a valid pointer to a protected structure it should use other locking mechanism to obtain a pointer. For example, it should lock a list that hold protected objects. Signed-off-by: Volodymyr Babchuk --- v3: - moved in from another patch series - used structure to encapsulate refcnt_t - removed #ifndef NDEBUG in favor of just calling ASSERT - added assertion to refcnt_put - added saturation support: code catches overflow and underflow - added EMACS magic at end of the file - fixed formatting --- xen/include/xen/refcnt.h | 59 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 xen/include/xen/refcnt.h diff --git a/xen/include/xen/refcnt.h b/xen/include/xen/refcnt.h new file mode 100644 index 0000000000..1ac05d927c --- /dev/null +++ b/xen/include/xen/refcnt.h @@ -0,0 +1,59 @@ +#ifndef __XEN_REFCNT_H__ +#define __XEN_REFCNT_H__ + +#include +#include + +#define REFCNT_SATURATED (INT_MIN / 2) + +typedef struct { + atomic_t refcnt; +} refcnt_t; + +static inline void refcnt_init(refcnt_t *refcnt) +{ + atomic_set(&refcnt->refcnt, 1); +} + +static inline int refcnt_read(refcnt_t *refcnt) +{ + return atomic_read(&refcnt->refcnt); +} + +static inline void refcnt_get(refcnt_t *refcnt) +{ + int old = atomic_add_unless(&refcnt->refcnt, 1, 0); + + if ( unlikely(old < 0) || unlikely (old + 1 < 0) ) + { + atomic_set(&refcnt->refcnt, REFCNT_SATURATED); + printk(XENLOG_ERR"refcnt saturation: old = %d\n", old); + WARN(); + } +} + +static inline void refcnt_put(refcnt_t *refcnt, void (*destructor)(refcnt_t *refcnt)) +{ + int ret = atomic_dec_return(&refcnt->refcnt); + + if ( ret == 0 ) + destructor(refcnt); + + if ( unlikely(ret < 0)) + { + atomic_set(&refcnt->refcnt, REFCNT_SATURATED); + printk(XENLOG_ERR"refcnt already hit 0: val = %d\n", ret); + WARN(); + } +} + +#endif + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */ From patchwork Tue Mar 14 20:56:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13175007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 247B5C6FD1F for ; Tue, 14 Mar 2023 20:57:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.509845.786452 (Exim 4.92) (envelope-from ) id 1pcBhy-0004DG-UA; Tue, 14 Mar 2023 20:57:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 509845.786452; Tue, 14 Mar 2023 20:57:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhy-0004D7-Qh; Tue, 14 Mar 2023 20:57:22 +0000 Received: by outflank-mailman (input) for mailman id 509845; Tue, 14 Mar 2023 20:57:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhw-0001PG-LY for xen-devel@lists.xenproject.org; Tue, 14 Mar 2023 20:57:20 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ce7c6aef-c2aa-11ed-87f5-c1b5be75604c; Tue, 14 Mar 2023 21:57:19 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EKchDo017378; Tue, 14 Mar 2023 20:56:35 GMT Received: from eur05-vi1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3pb0520156-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 20:56:35 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by PAXPR03MB7967.eurprd03.prod.outlook.com (2603:10a6:102:21a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Tue, 14 Mar 2023 20:56:30 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e%4]) with mapi id 15.20.6178.026; Tue, 14 Mar 2023 20:56:30 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ce7c6aef-c2aa-11ed-87f5-c1b5be75604c ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jC4APIhGtIE2q2vnB7Xgyp88/Qoj80o1jW8ZVmEsR9vzsBK/K7E9TKVH0ws3gYP/YP+M9F8QOAmNkVDpTUaIEk3cckeJDUxhkALykktfOg+vAAmly5lTrhiLI6GjXa7FSZr/zEVrsOryvVRh0JyevefLJHLd0d83VEWpAvWLCf8mAU95pr01MZRFUYQlym+9wKdLgtBxsULI9maLAFf1pnd7kIBUzCqSXrkc04IYaCdWRlS3+sOvIsChltiDYq8cIK7X8zgwLxhDG0dtK3mK5DFcFPvfL5TJEZCYO19KIVSLtu3BCcV52Bpz5JaJoAO1TSbPJb83UyZ2gNnVg6ID1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x3giRBF2vC8XfohoRJp2TLUtqgs0zjuWEsEx8QPu5VA=; b=Tz/R6YZTMwc70J7ONg+kKpXLIbOTFUKWt/ae5r+pOO9D5VNXhFn6WhsVnCQdWjogdSv3XeT2XCcV5leL/fvTurxyRI7ZWz5Qz5AdHQfgeBeYm709FDveXJAGYyKjjKXZCKINQxroaWKAa54ad9Yp4QFJjYP/Dp8DQDZwKKepiLPlsFmHLrU0VKlg0unh+xHLHFxO8X6gLNzBUeRkX1xTehxHhMyyKac2nQekWI1bWlUGlnncA5euBRJUWu4/x52fznakBZsCJV31jGGoifpIy4jirsFf0Aa2ayzC6coDM6iqO4G7qy802N1Q9eMvOJnPS4qB5zs6OJEFTp0E3WaVxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x3giRBF2vC8XfohoRJp2TLUtqgs0zjuWEsEx8QPu5VA=; b=dAviOwHeIzoIc/A3gZX5H+T7Vx+QUEPdEpSBpecXfxfM7wxgv433I/1U4T0S8qHWKVjVQyFyxzaa41H8qw6iLgiP29L6YfKNjxq988Xx9WL+ynHoFKMTz4gZd/wJqu9mFOA/6+EOwpbrsjDS2Dl2Jzcd2QZCwE79dvg98xmg7ATGRhSSajPuyV0LLY/GghGTCMCjMTI/30eQlwFTtbHgPBD9NCEpS2gnABk5CCzMT/Uu//Iv56WMyNZ2gkgN+djQnzDrzvKKA6TYmIzXe7a4IIr88MGTrb/SpxndggQwA7bOoVe4ZrQ3d93Xf0fOCczOGjYRc/MMtvIl0ErE2eQwgg== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Volodymyr Babchuk , Jan Beulich , Andrew Cooper , =?iso-8859-1?q?Roger_Pau_Monn=E9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Paul Durrant , Kevin Tian Subject: [PATCH v3 2/6] xen: pci: introduce reference counting for pdev Thread-Topic: [PATCH v3 2/6] xen: pci: introduce reference counting for pdev Thread-Index: AQHZVrdzsdckomMx4kauxHkZQ597Ig== Date: Tue, 14 Mar 2023 20:56:29 +0000 Message-ID: <20230314205612.3703668-3-volodymyr_babchuk@epam.com> References: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> In-Reply-To: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.39.2 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|PAXPR03MB7967:EE_ x-ms-office365-filtering-correlation-id: a0e2b907-4a39-4d48-f078-08db24ce9661 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Omev8bapjhzUupelKf/ySvHHGmZkYH1Re53t7uAlTwmc/m/KG/eOq4gqhxL7eVvdhCXNvK4yKPg0KAXz/9q88PVRTVdf3TADh/awFMo/IFz4VUg9lSXIrHpw6IjJwbCdI9Q6BlC5bD3PW89hBQjDPlpw1y1GZWuYUgLyipLMxQABF4kLGtjKoE6V8JIDEY/Gzf0c9ZIfyJFM9l8v8C/vpH8Z6vR/uUSQYdx04474IeJYm0S11RrJJ+L2cUSkqIEWQ7gjWZ0v8yMUGivy+D+eNZCZMvL2ty/Bb8b3FPZBwU5NdF47vEBZdVmyjufBR0B3188dQQPU8wVROQgk3ooA9aEN08zgQghYsGVkkeznWfM1jb2wX2sm4wKWDZhQkRc5r7lCsQ2YIlgOPX8Ri5Xr34n3Jon6a+mzKH1GXS5dsz8z6UjpDnk4lIMj3gqGkry7LKaGsd4DlfeTTD8A+MKEZLDCenj8T7X/A/TR5ULMHnHuyV6To2Zz8X2Az+LMQBXu3XPNSFUPIcrj7Nze/8D0LF4+SeehmhYdiFPGClE62m4aygvkYhSoDYzJdRl10nTXfISll/YY51F3qOhgTmprsqQOJEPtEdKa3nyCrvMWqSQqk0F+Ig9XqvdfGFUr6nwheUso77l/0yoDDUCKigg2GXOD1BfOs72EMbKO97QTNhH6RmsHxR9iDd5hOWzsnTdTU4JLV9ew9KYQomEIbEM8q5Ei6PnUYN5y6FRcB8c5iZY= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(451199018)(54906003)(316002)(86362001)(36756003)(38070700005)(38100700002)(478600001)(122000001)(55236004)(83380400001)(6512007)(6506007)(2616005)(1076003)(186003)(26005)(7416002)(5660300002)(6486002)(91956017)(41300700001)(6916009)(71200400001)(76116006)(64756008)(30864003)(8936002)(8676002)(2906002)(66556008)(66476007)(66946007)(66446008)(4326008)(21314003)(579004)(559001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?gm1U33arFh8VzSF4/mkdueb?= =?iso-8859-1?q?H4EX1NY93Wces6XjSzBQWrtE3m0ZoZY+b8z7f+rx2rPiwNGA95AMl1gnD+iS?= =?iso-8859-1?q?vODvNldMLOuEQXxIIcHHh7bjcws0hG3hsEu5InVqT/R0bnXXvPqA6YMJshIT?= =?iso-8859-1?q?JcyZ+g02D/dQVOHPJvIj4muQ1XFMbucVwQf2qTCL+ayfaIFQUCSNuy5RuNHo?= =?iso-8859-1?q?/jb+lUVUKrdYAEaFAOvgemTwXHgA2bBQidbBLJ40Mj3l2QLuiF6Tn9xCyVUR?= =?iso-8859-1?q?GkmjR6rHfzNLiDVNMszQVyvBKakXUDzChl+fwX7vqFOsSv84AiEQKVKCVW55?= =?iso-8859-1?q?oPJpgVXSkr4IWMTWmkw5cNF/zO85INObWmw+FhQ2q0PxgWhZYtbozMYjWA3p?= =?iso-8859-1?q?cvpF2PjJhoDFvMCVTgqilf3oolEZiNau7hWDJEfeBx4bVvuuEJPPmW6TjJ9Y?= =?iso-8859-1?q?zTOf6opbafCDxtPj7ZcV8dAqtYsfMf0rZ8x8PGq5buf70JqxRPPtZzgS9GnH?= =?iso-8859-1?q?iaMkbcLeZss+q7KYcTj6frdM0el4IUIGczR8IB/mmWC29w3kg4WS2mLVcCkY?= =?iso-8859-1?q?3dpcAJKuAvE0XYE2AuVtu8o0bZgOlxgJQ3HMQEdefWGnLpDrcwNSNUEnc8/4?= =?iso-8859-1?q?i8uF3m4CeVbHmsKgN2GXxD/u9N+wizoh3GOh9VPFY2WL4ambZMMyUmjUWEAx?= =?iso-8859-1?q?Des/+flRBnqyyYaAGs5D5dxfHxoIhaC/IW6RCiEZkRPu5OKUZS66wgohwbx+?= =?iso-8859-1?q?l6q+y3n6Z8oMIOkwxlGjUyz9XvrXDApMPN7mrbr/SLSPaMxNAd0fqnzj4UoE?= =?iso-8859-1?q?eCpM72XzpsIdqejcOTzABL/dILcEuoNZqHUzsy5kjpO59zadPQ3C3P/xfsEq?= =?iso-8859-1?q?8lzXSn1WGPjJtizkQtP0ev0DcGWGEY7bioUZSZrtjflzEHarv2kcKU2unz/y?= =?iso-8859-1?q?Ta96hHuxNTaq9rqSEEM9w07UMMOqWxE3DdU5xXUpOyogvnwlf8DCbGDXH3Ps?= =?iso-8859-1?q?WLg592znEzZ/JEznvKygyNM+9kx1qGY9mSn8FmEDmdD8iDBcqjo37CJ9xy7m?= =?iso-8859-1?q?lRZv/cTriCg3z+3FCGKUGYX8lkC1atpA1GELjlaJQzfYtXptrgn1TN8Dtf93?= =?iso-8859-1?q?ou6iOb5zQfznNPqWDo213/ezhElbSXYrXTGSHDNpbm5eMelhc+L9XVsE87KX?= =?iso-8859-1?q?BizKYwYQ1FXOkyV6ttLRB3SgKXSiLb2yzL6Ld//0xgzq3u2wRxYes7YnObzG?= =?iso-8859-1?q?tdmfkTjOBSC6UxmMZ93IUpWqvioqR6djfZjmEA2ceTGbz6NrCi4mq2EUsGLG?= =?iso-8859-1?q?LmO7hM3vecXlrIGEk/vxr+CfrUTs5XTETg+BmQOGR818z7gKFr46T9/b95it?= =?iso-8859-1?q?C9CFEjX8SRlsiUZKgPGyzgyGPt39FYbmXe7AlyeDGuQFsb0d/BhdiDFemDNT?= =?iso-8859-1?q?xjJKpeLCBSOof2TPZrLqFH8cPGwmhbIlGKS6pS31vO9uMzEc1R/twBSe3i7h?= =?iso-8859-1?q?GY/d1/FBvmAyx35wfWzr0NOOxld4vx4uOPcUNaYoZLETD+GE4fDU/U2IrBRZ?= =?iso-8859-1?q?DkOapXfIVg5XMHhSiRneblI8eflETG8aecJCKLIeQ0oWKuHs925av5vPw3Yu?= =?iso-8859-1?q?3e40KJoDLaj0ScnWBy64qQBGR2UrHArsoAqH4sw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a0e2b907-4a39-4d48-f078-08db24ce9661 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2023 20:56:29.7296 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 9m1ahttUilynrm5cSdiA2L7W44xc6n04nTe/wwHpmlbzRibjcyyH3z6G68VNOcwac1bzabl/8EerHr9hZUdNDKaLLNF3joahJIeQ9UgPGBc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7967 X-Proofpoint-ORIG-GUID: ZTGSVGK1gJqAvJXQjNTJFi0fYEwGzaEF X-Proofpoint-GUID: ZTGSVGK1gJqAvJXQjNTJFi0fYEwGzaEF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_14,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 priorityscore=1501 suspectscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140168 Prior to this change, lifetime of pci_dev objects was protected by global pcidevs_lock(). Long-term plan is to remove this log, so we need some other mechanism to ensure that those objects will not disappear under feet of code that access them. Reference counting is a good choice as it provides easy to comprehend way to control object lifetime. This patch adds two new helper functions: pcidev_get() and pcidev_put(). pcidev_get() will increase reference counter, while pcidev_put() will decrease it, destroying object when counter reaches zero. pcidev_get() should be used only when you already have a valid pointer to the object or you are holding lock that protects one of the lists (domain, pseg or ats) that store pci_dev structs. pcidev_get() is rarely used directly, because there already are functions that will provide valid pointer to pci_dev struct: pci_get_pdev(), pci_get_real_pdev(). They will lock appropriate list, find needed object and increase its reference counter before returning to the caller. Naturally, pci_put() should be called after finishing working with a received object. This is the reason why this patch have so many pcidev_put()s and so little pcidev_get()s: existing calls to pci_get_*() functions now will increase reference counter automatically, we just need to decrease it back when we finished. This patch removes "const" qualifier from some pdev pointers because pcidev_put() technically alters the contents of pci_dev structure. Signed-off-by: Volodymyr Babchuk Suggested-by: Jan Beulich --- v3: - Moved in from another patch series - Fixed code formatting (tabs -> spaces) - Removed erroneous pcidev_put in vga.c - Added missing pcidev_put in couple of places - removed mention of pci_get_pdev_by_domain() --- xen/arch/x86/hvm/vmsi.c | 2 +- xen/arch/x86/irq.c | 4 + xen/arch/x86/msi.c | 44 +++++++- xen/arch/x86/pci.c | 3 + xen/arch/x86/physdev.c | 17 ++- xen/common/sysctl.c | 7 +- xen/drivers/passthrough/amd/iommu_init.c | 12 +- xen/drivers/passthrough/amd/iommu_map.c | 6 +- xen/drivers/passthrough/pci.c | 138 +++++++++++++++-------- xen/drivers/passthrough/vtd/quirks.c | 2 + xen/drivers/video/vga.c | 7 +- xen/drivers/vpci/vpci.c | 16 ++- xen/include/xen/pci.h | 18 +++ 13 files changed, 215 insertions(+), 61 deletions(-) diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c index 3cd4923060..8c3d673872 100644 --- a/xen/arch/x86/hvm/vmsi.c +++ b/xen/arch/x86/hvm/vmsi.c @@ -914,7 +914,7 @@ int vpci_msix_arch_print(const struct vpci_msix *msix) spin_unlock(&msix->pdev->vpci->lock); process_pending_softirqs(); - /* NB: we assume that pdev cannot go away for an alive domain. */ + if ( !pdev->vpci || !spin_trylock(&pdev->vpci->lock) ) return -EBUSY; if ( pdev->vpci->msix != msix ) diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index 20150b1c7f..87464d82c8 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2175,6 +2175,7 @@ int map_domain_pirq( msi->entry_nr = ret; ret = -ENFILE; } + pcidev_put(pdev); goto done; } @@ -2189,6 +2190,7 @@ int map_domain_pirq( msi_desc->irq = -1; msi_free_irq(msi_desc); ret = -EBUSY; + pcidev_put(pdev); goto done; } @@ -2273,10 +2275,12 @@ int map_domain_pirq( } msi_desc->irq = -1; msi_free_irq(msi_desc); + pcidev_put(pdev); goto done; } set_domain_irq_pirq(d, irq, info); + pcidev_put(pdev); spin_unlock_irqrestore(&desc->lock, flags); } else diff --git a/xen/arch/x86/msi.c b/xen/arch/x86/msi.c index d0bf63df1d..91926fce50 100644 --- a/xen/arch/x86/msi.c +++ b/xen/arch/x86/msi.c @@ -572,6 +572,10 @@ int msi_free_irq(struct msi_desc *entry) virt_to_fix((unsigned long)entry->mask_base)); list_del(&entry->list); + + /* Corresponds to pcidev_get() in msi[x]_capability_init() */ + pcidev_put(entry->dev); + xfree(entry); return 0; } @@ -644,6 +648,7 @@ static int msi_capability_init(struct pci_dev *dev, entry[i].msi.mpos = mpos; entry[i].msi.nvec = 0; entry[i].dev = dev; + pcidev_get(dev); } entry->msi.nvec = nvec; entry->irq = irq; @@ -703,22 +708,36 @@ static u64 read_pci_mem_bar(u16 seg, u8 bus, u8 slot, u8 func, u8 bir, int vf) !num_vf || !offset || (num_vf > 1 && !stride) || bir >= PCI_SRIOV_NUM_BARS || !pdev->vf_rlen[bir] ) + { + if ( pdev ) + pcidev_put(pdev); return 0; + } base = pos + PCI_SRIOV_BAR; vf -= PCI_BDF(bus, slot, func) + offset; if ( vf < 0 ) + { + pcidev_put(pdev); return 0; + } if ( stride ) { if ( vf % stride ) + { + pcidev_put(pdev); return 0; + } vf /= stride; } if ( vf >= num_vf ) + { + pcidev_put(pdev); return 0; + } BUILD_BUG_ON(ARRAY_SIZE(pdev->vf_rlen) != PCI_SRIOV_NUM_BARS); disp = vf * pdev->vf_rlen[bir]; limit = PCI_SRIOV_NUM_BARS; + pcidev_put(pdev); } else switch ( pci_conf_read8(PCI_SBDF(seg, bus, slot, func), PCI_HEADER_TYPE) & 0x7f ) @@ -925,6 +944,8 @@ static int msix_capability_init(struct pci_dev *dev, entry->dev = dev; entry->mask_base = base; + pcidev_get(dev); + list_add_tail(&entry->list, &dev->msi_list); *desc = entry; } @@ -999,6 +1020,7 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) { struct pci_dev *pdev; struct msi_desc *old_desc; + int ret; ASSERT(pcidevs_locked()); pdev = pci_get_pdev(NULL, msi->sbdf); @@ -1010,6 +1032,7 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) { printk(XENLOG_ERR "irq %d already mapped to MSI on %pp\n", msi->irq, &pdev->sbdf); + pcidev_put(pdev); return -EEXIST; } @@ -1020,7 +1043,10 @@ static int __pci_enable_msi(struct msi_info *msi, struct msi_desc **desc) __pci_disable_msix(old_desc); } - return msi_capability_init(pdev, msi->irq, desc, msi->entry_nr); + ret = msi_capability_init(pdev, msi->irq, desc, msi->entry_nr); + pcidev_put(pdev); + + return ret; } static void __pci_disable_msi(struct msi_desc *entry) @@ -1054,20 +1080,29 @@ static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) { struct pci_dev *pdev; struct msi_desc *old_desc; + int ret; ASSERT(pcidevs_locked()); pdev = pci_get_pdev(NULL, msi->sbdf); if ( !pdev || !pdev->msix ) + { + if ( pdev ) + pcidev_put(pdev); return -ENODEV; + } if ( msi->entry_nr >= pdev->msix->nr_entries ) + { + pcidev_put(pdev); return -EINVAL; + } old_desc = find_msi_entry(pdev, msi->irq, PCI_CAP_ID_MSIX); if ( old_desc ) { printk(XENLOG_ERR "irq %d already mapped to MSI-X on %pp\n", msi->irq, &pdev->sbdf); + pcidev_put(pdev); return -EEXIST; } @@ -1078,7 +1113,11 @@ static int __pci_enable_msix(struct msi_info *msi, struct msi_desc **desc) __pci_disable_msi(old_desc); } - return msix_capability_init(pdev, msi, desc); + ret = msix_capability_init(pdev, msi, desc); + + pcidev_put(pdev); + + return ret; } static void _pci_cleanup_msix(struct arch_msix *msix) @@ -1159,6 +1198,7 @@ int pci_prepare_msix(u16 seg, u8 bus, u8 devfn, bool off) } else rc = msix_capability_init(pdev, NULL, NULL); + pcidev_put(pdev); pcidevs_unlock(); return rc; diff --git a/xen/arch/x86/pci.c b/xen/arch/x86/pci.c index 97b792e578..c1fcdf08d6 100644 --- a/xen/arch/x86/pci.c +++ b/xen/arch/x86/pci.c @@ -92,7 +92,10 @@ int pci_conf_write_intercept(unsigned int seg, unsigned int bdf, pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bdf)); if ( pdev ) + { rc = pci_msi_conf_write_intercept(pdev, reg, size, data); + pcidev_put(pdev); + } pcidevs_unlock(); diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index 2f1d955a96..96214a3d40 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -533,7 +533,14 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(0, restore_msi.bus, restore_msi.devfn)); - ret = pdev ? pci_restore_msi_state(pdev) : -ENODEV; + if ( pdev ) + { + ret = pci_restore_msi_state(pdev); + pcidev_put(pdev); + } + else + ret = -ENODEV; + pcidevs_unlock(); break; } @@ -548,7 +555,13 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) pcidevs_lock(); pdev = pci_get_pdev(NULL, PCI_SBDF(dev.seg, dev.bus, dev.devfn)); - ret = pdev ? pci_restore_msi_state(pdev) : -ENODEV; + if ( pdev ) + { + ret = pci_restore_msi_state(pdev); + pcidev_put(pdev); + } + else + ret = -ENODEV; pcidevs_unlock(); break; } diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c index 02505ab044..9af07fa92a 100644 --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -438,7 +438,7 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) { physdev_pci_device_t dev; uint32_t node; - const struct pci_dev *pdev; + struct pci_dev *pdev; if ( copy_from_guest_offset(&dev, ti->devs, i, 1) ) { @@ -454,8 +454,11 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) node = XEN_INVALID_NODE_ID; else node = pdev->node; - pcidevs_unlock(); + if ( pdev ) + pcidev_put(pdev); + + pcidevs_unlock(); if ( copy_to_guest_offset(ti->nodes, i, &node, 1) ) { ret = -EFAULT; diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c index 9773ccfcb4..f90b1c1e58 100644 --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -646,6 +646,7 @@ static void cf_check parse_ppr_log_entry(struct amd_iommu *iommu, u32 entry[]) if ( pdev ) guest_iommu_add_ppr_log(pdev->domain, entry); + pcidev_put(pdev); } static void iommu_check_ppr_log(struct amd_iommu *iommu) @@ -749,6 +750,11 @@ static bool_t __init set_iommu_interrupt_handler(struct amd_iommu *iommu) } pcidevs_lock(); + /* + * XXX: it is unclear if this device can be removed. Right now + * there is no code that clears msi.dev, so no one will decrease + * refcount on it. + */ iommu->msi.dev = pci_get_pdev(NULL, PCI_SBDF(iommu->seg, iommu->bdf)); pcidevs_unlock(); if ( !iommu->msi.dev ) @@ -1274,7 +1280,7 @@ static int __init cf_check amd_iommu_setup_device_table( { if ( ivrs_mappings[bdf].valid ) { - const struct pci_dev *pdev = NULL; + struct pci_dev *pdev = NULL; /* add device table entry */ iommu_dte_add_device_entry(&dt[bdf], &ivrs_mappings[bdf]); @@ -1299,7 +1305,10 @@ static int __init cf_check amd_iommu_setup_device_table( pdev->msix ? pdev->msix->nr_entries : pdev->msi_maxvec); if ( !ivrs_mappings[bdf].intremap_table ) + { + pcidev_put(pdev); return -ENOMEM; + } if ( pdev->phantom_stride ) { @@ -1317,6 +1326,7 @@ static int __init cf_check amd_iommu_setup_device_table( ivrs_mappings[bdf].intremap_inuse; } } + pcidev_put(pdev); } amd_iommu_set_intremap_table( diff --git a/xen/drivers/passthrough/amd/iommu_map.c b/xen/drivers/passthrough/amd/iommu_map.c index 993bac6f88..9d621e3d36 100644 --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -724,14 +724,18 @@ int cf_check amd_iommu_get_reserved_device_memory( if ( !iommu ) { /* May need to trigger the workaround in find_iommu_for_device(). */ - const struct pci_dev *pdev; + struct pci_dev *pdev; pcidevs_lock(); pdev = pci_get_pdev(NULL, sbdf); pcidevs_unlock(); if ( pdev ) + { iommu = find_iommu_for_device(seg, bdf); + /* XXX: Should we hold pdev reference till end of the loop? */ + pcidev_put(pdev); + } if ( !iommu ) continue; } diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index b42acb8d7c..b32382aca0 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -328,6 +328,7 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) *((u8*) &pdev->bus) = bus; *((u8*) &pdev->devfn) = devfn; pdev->domain = NULL; + refcnt_init(&pdev->refcnt); arch_pci_init_pdev(pdev); @@ -422,33 +423,6 @@ static struct pci_dev *alloc_pdev(struct pci_seg *pseg, u8 bus, u8 devfn) return pdev; } -static void free_pdev(struct pci_seg *pseg, struct pci_dev *pdev) -{ - /* update bus2bridge */ - switch ( pdev->type ) - { - unsigned int sec_bus, sub_bus; - - case DEV_TYPE_PCIe2PCI_BRIDGE: - case DEV_TYPE_LEGACY_PCI_BRIDGE: - sec_bus = pci_conf_read8(pdev->sbdf, PCI_SECONDARY_BUS); - sub_bus = pci_conf_read8(pdev->sbdf, PCI_SUBORDINATE_BUS); - - spin_lock(&pseg->bus2bridge_lock); - for ( ; sec_bus <= sub_bus; sec_bus++ ) - pseg->bus2bridge[sec_bus] = pseg->bus2bridge[pdev->bus]; - spin_unlock(&pseg->bus2bridge_lock); - break; - - default: - break; - } - - list_del(&pdev->alldevs_list); - pdev_msi_deinit(pdev); - xfree(pdev); -} - static void __init _pci_hide_device(struct pci_dev *pdev) { if ( pdev->domain ) @@ -517,10 +491,14 @@ struct pci_dev *pci_get_real_pdev(pci_sbdf_t sbdf) { if ( !(sbdf.devfn & stride) ) continue; + sbdf.devfn &= ~stride; pdev = pci_get_pdev(NULL, sbdf); if ( pdev && stride != pdev->phantom_stride ) + { + pcidev_put(pdev); pdev = NULL; + } } return pdev; @@ -548,13 +526,18 @@ struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf) list_for_each_entry ( pdev, &pseg->alldevs_list, alldevs_list ) if ( pdev->sbdf.bdf == sbdf.bdf && (!d || pdev->domain == d) ) + { + pcidev_get(pdev); return pdev; + } } else list_for_each_entry ( pdev, &d->pdev_list, domain_list ) if ( pdev->sbdf.bdf == sbdf.bdf ) + { + pcidev_get(pdev); return pdev; - + } return NULL; } @@ -663,7 +646,10 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, PCI_SBDF(seg, info->physfn.bus, info->physfn.devfn)); if ( pdev ) + { pf_is_extfn = pdev->info.is_extfn; + pcidev_put(pdev); + } pcidevs_unlock(); if ( !pdev ) pci_add_device(seg, info->physfn.bus, info->physfn.devfn, @@ -818,7 +804,9 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn) if ( pdev->domain ) list_del(&pdev->domain_list); printk(XENLOG_DEBUG "PCI remove device %pp\n", &pdev->sbdf); - free_pdev(pseg, pdev); + list_del(&pdev->alldevs_list); + pdev_msi_deinit(pdev); + pcidev_put(pdev); break; } @@ -848,7 +836,7 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, { ret = iommu_quarantine_dev_init(pci_to_dev(pdev)); if ( ret ) - return ret; + goto out; target = dom_io; } @@ -878,6 +866,7 @@ static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus, pdev->fault.count = 0; out: + pcidev_put(pdev); if ( ret ) printk(XENLOG_G_ERR "%pd: deassign (%pp) failed (%d)\n", d, &PCI_SBDF(seg, bus, devfn), ret); @@ -1011,7 +1000,10 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn) pdev->fault.count >>= 1; pdev->fault.time = now; if ( ++pdev->fault.count < PT_FAULT_THRESHOLD ) + { + pcidev_put(pdev); pdev = NULL; + } } pcidevs_unlock(); @@ -1022,6 +1014,8 @@ void pci_check_disable_device(u16 seg, u8 bus, u8 devfn) * control it for us. */ cword = pci_conf_read16(pdev->sbdf, PCI_COMMAND); pci_conf_write16(pdev->sbdf, PCI_COMMAND, cword & ~PCI_COMMAND_MASTER); + + pcidev_put(pdev); } /* @@ -1138,6 +1132,7 @@ static int __hwdom_init cf_check _setup_hwdom_pci_devices( printk(XENLOG_WARNING "Dom%d owning %pp?\n", pdev->domain->domain_id, &pdev->sbdf); + pcidev_put(pdev); if ( iommu_verbose ) { pcidevs_unlock(); @@ -1385,33 +1380,28 @@ static int iommu_remove_device(struct pci_dev *pdev) return iommu_call(hd->platform_ops, remove_device, devfn, pci_to_dev(pdev)); } -static int device_assigned(u16 seg, u8 bus, u8 devfn) +static int device_assigned(struct pci_dev *pdev) { - struct pci_dev *pdev; int rc = 0; ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); - - if ( !pdev ) - rc = -ENODEV; /* * If the device exists and it is not owned by either the hardware * domain or dom_io then it must be assigned to a guest, or be * hidden (owned by dom_xen). */ - else if ( pdev->domain != hardware_domain && - pdev->domain != dom_io ) + if ( pdev->domain != hardware_domain && + pdev->domain != dom_io ) rc = -EBUSY; return rc; } /* Caller should hold the pcidevs_lock */ -static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) +static int assign_device(struct domain *d, struct pci_dev *pdev, u32 flag) { const struct domain_iommu *hd = dom_iommu(d); - struct pci_dev *pdev; + uint8_t devfn; int rc = 0; if ( !is_iommu_enabled(d) ) @@ -1422,10 +1412,11 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) /* device_assigned() should already have cleared the device for assignment */ ASSERT(pcidevs_locked()); - pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); ASSERT(pdev && (pdev->domain == hardware_domain || pdev->domain == dom_io)); + devfn = pdev->devfn; + /* Do not allow broken devices to be assigned to guests. */ rc = -EBADF; if ( pdev->broken && d != hardware_domain && d != dom_io ) @@ -1460,7 +1451,7 @@ static int assign_device(struct domain *d, u16 seg, u8 bus, u8 devfn, u32 flag) done: if ( rc ) printk(XENLOG_G_WARNING "%pd: assign (%pp) failed (%d)\n", - d, &PCI_SBDF(seg, bus, devfn), rc); + d, &PCI_SBDF(pdev->seg, pdev->bus, devfn), rc); /* The device is assigned to dom_io so mark it as quarantined */ else if ( d == dom_io ) pdev->quarantine = true; @@ -1595,6 +1586,9 @@ int iommu_do_pci_domctl( ASSERT(d); /* fall through */ case XEN_DOMCTL_test_assign_device: + { + struct pci_dev *pdev; + /* Don't support self-assignment of devices. */ if ( d == current->domain ) { @@ -1622,26 +1616,36 @@ int iommu_do_pci_domctl( seg = machine_sbdf >> 16; bus = PCI_BUS(machine_sbdf); devfn = PCI_DEVFN(machine_sbdf); - pcidevs_lock(); - ret = device_assigned(seg, bus, devfn); + pdev = pci_get_pdev(NULL, PCI_SBDF(seg, bus, devfn)); + if ( !pdev ) + { + printk(XENLOG_G_INFO "%pp non-existent\n", + &PCI_SBDF(seg, bus, devfn)); + ret = -EINVAL; + break; + } + + ret = device_assigned(pdev); if ( domctl->cmd == XEN_DOMCTL_test_assign_device ) { if ( ret ) { - printk(XENLOG_G_INFO "%pp already assigned, or non-existent\n", + printk(XENLOG_G_INFO "%pp already assigned\n", &PCI_SBDF(seg, bus, devfn)); ret = -EINVAL; } } else if ( !ret ) - ret = assign_device(d, seg, bus, devfn, flags); + ret = assign_device(d, pdev, flags); + + pcidev_put(pdev); pcidevs_unlock(); if ( ret == -ERESTART ) ret = hypercall_create_continuation(__HYPERVISOR_domctl, "h", u_domctl); break; - + } case XEN_DOMCTL_deassign_device: /* Don't support self-deassignment of devices. */ if ( d == current->domain ) @@ -1681,6 +1685,46 @@ int iommu_do_pci_domctl( return ret; } +static void release_pdev(refcnt_t *refcnt) +{ + struct pci_dev *pdev = container_of(refcnt, struct pci_dev, refcnt); + struct pci_seg *pseg = get_pseg(pdev->seg); + + printk(XENLOG_DEBUG "PCI release device %pp\n", &pdev->sbdf); + + /* update bus2bridge */ + switch ( pdev->type ) + { + unsigned int sec_bus, sub_bus; + + case DEV_TYPE_PCIe2PCI_BRIDGE: + case DEV_TYPE_LEGACY_PCI_BRIDGE: + sec_bus = pci_conf_read8(pdev->sbdf, PCI_SECONDARY_BUS); + sub_bus = pci_conf_read8(pdev->sbdf, PCI_SUBORDINATE_BUS); + + spin_lock(&pseg->bus2bridge_lock); + for ( ; sec_bus <= sub_bus; sec_bus++ ) + pseg->bus2bridge[sec_bus] = pseg->bus2bridge[pdev->bus]; + spin_unlock(&pseg->bus2bridge_lock); + break; + + default: + break; + } + + xfree(pdev); +} + +void pcidev_get(struct pci_dev *pdev) +{ + refcnt_get(&pdev->refcnt); +} + +void pcidev_put(struct pci_dev *pdev) +{ + refcnt_put(&pdev->refcnt, release_pdev); +} + /* * Local variables: * mode: C diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c index fcc8f73e8b..d240da0416 100644 --- a/xen/drivers/passthrough/vtd/quirks.c +++ b/xen/drivers/passthrough/vtd/quirks.c @@ -429,6 +429,8 @@ static int __must_check map_me_phantom_function(struct domain *domain, rc = domain_context_unmap_one(domain, drhd->iommu, 0, PCI_DEVFN(dev, 7)); + pcidev_put(pdev); + return rc; } diff --git a/xen/drivers/video/vga.c b/xen/drivers/video/vga.c index 0a03508bee..1049d4da6d 100644 --- a/xen/drivers/video/vga.c +++ b/xen/drivers/video/vga.c @@ -114,7 +114,7 @@ void __init video_endboot(void) for ( bus = 0; bus < 256; ++bus ) for ( devfn = 0; devfn < 256; ++devfn ) { - const struct pci_dev *pdev; + struct pci_dev *pdev; u8 b = bus, df = devfn, sb; pcidevs_lock(); @@ -126,7 +126,11 @@ void __init video_endboot(void) PCI_CLASS_DEVICE) != 0x0300 || !(pci_conf_read16(PCI_SBDF(0, bus, devfn), PCI_COMMAND) & (PCI_COMMAND_IO | PCI_COMMAND_MEMORY)) ) + { + if ( pdev ) + pcidev_put(pdev); continue; + } while ( b ) { @@ -157,6 +161,7 @@ void __init video_endboot(void) bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); pci_hide_device(0, bus, devfn); } + pcidev_put(pdev); } } diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 6d48d496bb..5232f9605b 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -317,8 +317,8 @@ static uint32_t merge_result(uint32_t data, uint32_t new, unsigned int size, uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) { - const struct domain *d = current->domain; - const struct pci_dev *pdev; + struct domain *d = current->domain; + struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; uint32_t data = ~(uint32_t)0; @@ -332,7 +332,11 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) /* Find the PCI dev matching the address. */ pdev = pci_get_pdev(d, sbdf); if ( !pdev || !pdev->vpci ) + { + if ( pdev ) + pcidev_put(pdev); return vpci_read_hw(sbdf, reg, size); + } spin_lock(&pdev->vpci->lock); @@ -378,6 +382,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) ASSERT(data_offset < size); } spin_unlock(&pdev->vpci->lock); + pcidev_put(pdev); if ( data_offset < size ) { @@ -420,8 +425,8 @@ static void vpci_write_helper(const struct pci_dev *pdev, void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, uint32_t data) { - const struct domain *d = current->domain; - const struct pci_dev *pdev; + struct domain *d = current->domain; + struct pci_dev *pdev; const struct vpci_register *r; unsigned int data_offset = 0; const unsigned long *ro_map = pci_get_ro_map(sbdf.seg); @@ -443,6 +448,8 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, pdev = pci_get_pdev(d, sbdf); if ( !pdev || !pdev->vpci ) { + if ( pdev ) + pcidev_put(pdev); vpci_write_hw(sbdf, reg, size, data); return; } @@ -483,6 +490,7 @@ void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, ASSERT(data_offset < size); } spin_unlock(&pdev->vpci->lock); + pcidev_put(pdev); if ( data_offset < size ) /* Tailing gap, write the remaining. */ diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index 5975ca2f30..6631643fb1 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -116,6 +117,9 @@ struct pci_dev { /* Device misbehaving, prevent assigning it to guests. */ bool broken; + /* Reference counter */ + refcnt_t refcnt; + enum pdev_type { DEV_TYPE_PCI_UNKNOWN, DEV_TYPE_PCIe_ENDPOINT, @@ -160,6 +164,14 @@ void pcidevs_lock(void); void pcidevs_unlock(void); bool_t __must_check pcidevs_locked(void); +/* + * Acquire and release reference to the given device. Holding + * reference ensures that device will not disappear under feet, but + * does not guarantee that code has exclusive access to the device. + */ +void pcidev_get(struct pci_dev *pdev); +void pcidev_put(struct pci_dev *pdev); + bool_t pci_known_segment(u16 seg); bool_t pci_device_detect(u16 seg, u8 bus, u8 dev, u8 func); int scan_pci_devices(void); @@ -177,8 +189,14 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn, int pci_remove_device(u16 seg, u8 bus, u8 devfn); int pci_ro_device(int seg, int bus, int devfn); int pci_hide_device(unsigned int seg, unsigned int bus, unsigned int devfn); + +/* + * Next two functions will find a requested device and acquire + * reference to it. Use pcidev_put() to release the reference. + */ struct pci_dev *pci_get_pdev(const struct domain *d, pci_sbdf_t sbdf); struct pci_dev *pci_get_real_pdev(pci_sbdf_t sbdf); + void pci_check_disable_device(u16 seg, u8 bus, u8 devfn); uint8_t pci_conf_read8(pci_sbdf_t sbdf, unsigned int reg); From patchwork Tue Mar 14 20:56:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13175004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CC911C74A44 for ; Tue, 14 Mar 2023 20:57:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.509831.786402 (Exim 4.92) (envelope-from ) id 1pcBhK-0001eV-Cm; Tue, 14 Mar 2023 20:56:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 509831.786402; Tue, 14 Mar 2023 20:56:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhK-0001eN-9z; Tue, 14 Mar 2023 20:56:42 +0000 Received: by outflank-mailman (input) for mailman id 509831; Tue, 14 Mar 2023 20:56:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhJ-0001PN-Cs for xen-devel@lists.xenproject.org; Tue, 14 Mar 2023 20:56:41 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b57a9eb0-c2aa-11ed-b464-930f4c7d94ae; Tue, 14 Mar 2023 21:56:37 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EKchDm017378; Tue, 14 Mar 2023 20:56:34 GMT Received: from eur05-vi1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3pb0520156-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 20:56:34 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by PAXPR03MB7967.eurprd03.prod.outlook.com (2603:10a6:102:21a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Tue, 14 Mar 2023 20:56:31 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e%4]) with mapi id 15.20.6178.026; Tue, 14 Mar 2023 20:56:31 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b57a9eb0-c2aa-11ed-b464-930f4c7d94ae ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PQDEFzFQEctIMciwl+OzILvSR+CtxdwKClPvqkzCdH+oDsbR9ODNhE8AncllQnO8TSsW/yZq+NotDcTmu2ybRbcGf2+PFWXDusjKKJOVzlT6LNkcytQ2Z+cVSqIUQSAMI6oUW6wfxrDROeeoKX7UHO1rytiTm1846btTwvuhwz1b1eQS9CkbUqNsa/otfhF40hbd+mru89BHYXDTG0ssSImnBMNrhT/Yew/ne3ka/Gwjd/3GICVvkxsan+rnmtwjZcJUT3Ulm/oQ0eLT/Hg4/jafJUd6SGrgGCS3RqHmqT+Du49cdLShBOtnQuq/sc0mIjwYUdcBwTMWgAWY0r+kNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bujiMIeI01tpfNGAxOt9A1TohJ0brfLaSY560ql/6y8=; b=ZcKZPIYzvjg3a+xuzBXN9aGpneT1iz6/fZq+mxyxa9YGVAItEWeYY1NLE0mMrgCOVxgrbx0TN82Vxxcj3IsH9oPtvcY+qlnDq/q1RdbvKVU9c4AnJEXu28uexFQPmRXI6AwYEhLNE9DTMjQ1aw3c5hAqWC38cF13WR4BJ4DGlEry/cuAcgHAD7UngKrQrjv1gdvABrmoaIs+mtzIa06juDL8zxqF8DhQjN2WRwXpSfdbwKmLImNXJjC2LdipW/SiUH6JuWJijiaEVDu3+y/q4KM42DyHZwAw14msDPjmjrL5popYL85mUbHpmhpazMUBCXd8tqcLsMhcKxbg9SErdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bujiMIeI01tpfNGAxOt9A1TohJ0brfLaSY560ql/6y8=; b=TRgsE2qqyXIug3Rvp9xC4Tsva8b3hoeQ/yoB+W4E3ti6aokwK8VgDRNvhjqgrdWQVrZnGqd3Sltn1cnzLJdaS6Wg2UY47E3VzHqcOKN1R0rpYNtgPzGZ/ZY48eumt44P/02l0A5osPHgV8KsKFXy99yMKmMrgAKVn2eIFx+VaCKlAhyA4PcGZfFCpUOUAoUpkG72EScSY256GzGWdj3F+JHkPw7P6btyawhmBoxhwi5UC93CoG711TG03oK2Rq0nKml0W5bTeIlFm66HrfCeKA67uf0Qz8z5oow4LxXYC1Db4NjjiNshUbUuMazID2KfYM3RzsI5tTVjiC//C60Fbw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Volodymyr Babchuk , =?iso-8859-1?q?Roger_Pau?= =?iso-8859-1?q?_Monn=E9?= Subject: [PATCH v3 3/6] vpci: crash domain if we wasn't able to (un) map vPCI regions Thread-Topic: [PATCH v3 3/6] vpci: crash domain if we wasn't able to (un) map vPCI regions Thread-Index: AQHZVrdzRVfDuhWpnEqbBuMRX+vTVw== Date: Tue, 14 Mar 2023 20:56:30 +0000 Message-ID: <20230314205612.3703668-4-volodymyr_babchuk@epam.com> References: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> In-Reply-To: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.39.2 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|PAXPR03MB7967:EE_ x-ms-office365-filtering-correlation-id: 2fc9dfd1-07b6-41fb-77c3-08db24ce968a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ubw5diB9t8NW2UgX8HlKvSfPhPzOV6VfAUHH6IlrTbvgSyEWDbBGJmEPOzijjBbJ2ISEr1+J1KlbY26l8Mo9OPpB4HBDdRVHyy3u6W+MBrYrENPj0rkKM2CXuuClV/T8FLlOBzT0oBsMqCEIKlBAcux6QrFAfAXtA3a2cIdX3oBLO9xKQ2eiXRgLav19aqHrcKYN4nyWqTWFjueOPQ5e2ooQhxk7jVzCkQtWA8ZMAOfTNsa5JO6aqxd23Zi3dEACxHE3z0vrj4nostgZEoLMiUAk0MIEAojvSC0lElHgP4Cep6BmlFnpTIsIDEybLIInVP0MN/y+5Ei6I7ESVKZjR1MpNp4HLLgfKPkkrDXOc6n8yrXIlRp+XDfVmVfTB2RNJdnO9DxxGgb7++9xqJq0VNACPjW2/pnsfvUuq8T/jwFN4qEotrrRl+HdzgXsM8rYOXanvJhTczGI7wTOpTjTUfp1aM8CQqRfj3gzhLePXPMQ6KmmtMbJYPuhcIWRBJzgU+29rxoAC6t1NU3tJaEEKaM/EXEr8oha4CCo7bCHarTsTdznN5bpTX5iIfqS9I47x0SFmJrt80vOlt/auRXsEc+0CqhYW8p9V3J4Wuk2XWt+ABb45nbSEcwrQRmU3ItQD60LvbXZEDr4VdkuGXB/HClpyOwGdbZ7Qlfd5hwTpfN76j/It9br7iepG9tRvOy4E9H4qIF+wk11kW3XX4kgQg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(451199018)(54906003)(316002)(86362001)(36756003)(38070700005)(38100700002)(478600001)(122000001)(55236004)(83380400001)(6512007)(6506007)(2616005)(1076003)(186003)(26005)(5660300002)(6486002)(91956017)(41300700001)(6916009)(71200400001)(76116006)(64756008)(8936002)(8676002)(2906002)(66556008)(66476007)(66946007)(66446008)(4326008);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?BescmCuB6tXcs8SN2vu/r2N?= =?iso-8859-1?q?6rpAoLJ25LJ6GVm+wdExCu82L2fj7XfOJUjpto7hknqujKfmoM/qDpJ+k+Q4?= =?iso-8859-1?q?FKt7uP2D93NzUeAfCR77zrZc+lUHEfxnmo36IDhPDzJ+Ig5ZnFWkIxlFVWK5?= =?iso-8859-1?q?R1bFMKYFPlJval64LdrRXZeUsIVX4UHJX10SMbV2Sf9owSyad3pdzW4fBi93?= =?iso-8859-1?q?U3aAsLJPm4h5lEULH3F6sE9TUuKVs6qvmLkQSAWu7pgoNuBCVwpajC2VmR7N?= =?iso-8859-1?q?1XPUVZ0Mt+HQAsrZDOWdDfYegRY/vx9TtrnJS3Y1znE3UlrR3WdtzM55j0dn?= =?iso-8859-1?q?PTQBor53yZY5E3CKqHB7qVu1QmUnht5ejB05lZZxIogSWtyXnqRuE47QgDFf?= =?iso-8859-1?q?WdvBCsU2Fn+5B3Lgxl8Bjz8arQChlF2HVFL/QNpD2GPoSwGzQ7dgVh06KmiE?= =?iso-8859-1?q?6aHy0afrtGc8+H/F1fyRS9ViOqxOaqtZ9DH+vRh2f2hupSpHyzamjCXmPv90?= =?iso-8859-1?q?4zzbvnGtddZPSKecenw2q0JOmM9sl1ct/8lmP/mwfRsCFIEOZEsWbjSIQJgH?= =?iso-8859-1?q?CuuAZusTFBRfXD7ZZdpejoA4ABFqPYo2M7C18pZ+QWN9txQFBKe268I53jrc?= =?iso-8859-1?q?XmJ9ulA0AVoA0yj9Yu6+5n5unr03XOaci3wrmx2xJYbLY9enxoOu6RecvyQ6?= =?iso-8859-1?q?LKwgdz10HRZkvjgLOcjgIjVBpFjwcJw6n1IbSRuPtTXhtznFd39uZO0SJsvE?= =?iso-8859-1?q?hIkiq+f/0doU4gOTWuoI/7lHMf/QlBSSBzyb72o0Ihz77y70ah7EqLBzZCEB?= =?iso-8859-1?q?Hlsuu6MN1Cc0iHVzAck3NqgGHPsgPhnmRfboMe5P/Y37nHP6PAtZ8vkmry8H?= =?iso-8859-1?q?nE7b0l2We9rnrWjjxHEM6JdTX+gMG9Cp8Eh4b6uX0x90bY+UxiGKYTTGaqJK?= =?iso-8859-1?q?TXlHUMr79nE0yjYUlBjPipi9qRII0VKJkdmjJ5P2XEY0vopGgR48zutU9vKX?= =?iso-8859-1?q?bPhQDJWJBgMPK/0AdcWiDJA/xVd0tr/Eu31aFgxao0qwsBS5z/oNOPemOQeQ?= =?iso-8859-1?q?huUZuMnYgJ6T79bZhyipVYDnbqQNABLv3LZ9mjMcUZyQmFO0kV7pxLWFtF/b?= =?iso-8859-1?q?9CvAp/VqvmIKKO4xs+vhTt42IpwZJK9p+yCkNHIWw3JsLUyrPuNTjtIvJxZb?= =?iso-8859-1?q?5/MrCzEbhmzMBt9nlXpRlfZ6l1E3hRZcnmCbPLASoPQkAwvKhel/WaWrMVdS?= =?iso-8859-1?q?CtkTgR2/amfTg1jEwJJLZrOc3Ovf8h+If3HE+2on7L67sB1/LH2H+aLSbWWf?= =?iso-8859-1?q?CxsI1oNGczp/C00cnadtxxd9KkYjFKzFBuLVilvymfncLKkF8SA9epjXNm2J?= =?iso-8859-1?q?8otkUsSXbY4J8G22jXCT2MwiLNBgIGFVx4suzwWLDGcp8ljAsgKvmhMRz+u6?= =?iso-8859-1?q?30QrglNFIU8moYYrxlZrAKHOejBfO5CpDes4KcfKhG1qAL2J+sTme8rrZqEY?= =?iso-8859-1?q?q5UUs3oWJMLH+OHPOM0ZbZN+byJRszlcsT9ZQMQzXWiWhSJrYHnARjZv7boM?= =?iso-8859-1?q?fVXQXmej2uXXJPGTkJYpAIHJ6n6Kqwv3rVC8jEZXVXTDDInIIYfGQfySaTnI?= =?iso-8859-1?q?V0NRV2wfIl+taZL3jaMxXwW/5rk+qG66qyBudzw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2fc9dfd1-07b6-41fb-77c3-08db24ce968a X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2023 20:56:30.0421 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4PvcZkAB80bioXeVm1mdBkohJAwQhAlnmY3u2nQfTvBufmTguLADkY+WLoI6GVLNu4YKUWh6IQAtpsi4e+x6OQrWvZYUqW3jAeNL1KYxS/Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7967 X-Proofpoint-ORIG-GUID: OJIrJ60dx9-k4b8P-5rmvGDskVIzpZIs X-Proofpoint-GUID: OJIrJ60dx9-k4b8P-5rmvGDskVIzpZIs X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_14,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=644 priorityscore=1501 suspectscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140168 In that unlikely case, when map_range() fails to do it's job, domain memory mapping will be left in inconsistent state. As there is no easy way to remove stale p2m mapping we need to crash domain, as FIXME suggests. Signed-off-by: Volodymyr Babchuk --- v3: - new patch --- xen/drivers/vpci/header.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index ec2e978a4e..8319fe4c1d 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -162,14 +162,11 @@ bool vpci_process_pending(struct vcpu *v) rangeset_destroy(v->vpci.mem); v->vpci.mem = NULL; if ( rc ) - /* - * FIXME: in case of failure remove the device from the domain. - * Note that there might still be leftover mappings. While this is - * safe for Dom0, for DomUs the domain will likely need to be - * killed in order to avoid leaking stale p2m mappings on - * failure. - */ + { vpci_remove_device(v->vpci.pdev); + if ( !is_hardware_domain(v->domain) ) + domain_crash(v->domain); + } } return false; From patchwork Tue Mar 14 20:56:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13175003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D4487C7618D for ; Tue, 14 Mar 2023 20:57:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.509830.786392 (Exim 4.92) (envelope-from ) id 1pcBhJ-0001PU-5E; Tue, 14 Mar 2023 20:56:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 509830.786392; Tue, 14 Mar 2023 20:56:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhJ-0001PM-2K; Tue, 14 Mar 2023 20:56:41 +0000 Received: by outflank-mailman (input) for mailman id 509830; Tue, 14 Mar 2023 20:56:39 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhH-0001PG-JM for xen-devel@lists.xenproject.org; Tue, 14 Mar 2023 20:56:39 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b562f4ea-c2aa-11ed-87f5-c1b5be75604c; Tue, 14 Mar 2023 21:56:37 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EKchDn017378; Tue, 14 Mar 2023 20:56:34 GMT Received: from eur05-vi1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3pb0520156-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 20:56:34 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by PAXPR03MB7967.eurprd03.prod.outlook.com (2603:10a6:102:21a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Tue, 14 Mar 2023 20:56:31 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e%4]) with mapi id 15.20.6178.026; Tue, 14 Mar 2023 20:56:31 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b562f4ea-c2aa-11ed-87f5-c1b5be75604c ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IUk7j6c2OI3hgPhgnXSR0ytuaFnhExLrKuxPX0pyr6GLtxm//1XFkITbunUKMrghmNd/x0sMYMwg5vG/U2Ab5uV1KepFA9syX4KXNPHN6XOidsVDtoJlPUESNEIn2c7e60AIRlFuqwoXil59+xXotWKjiHfAi/ZS9P9pPZvgSqUe13Nv97Tnj0LkCIXVORB/7M1ab+psB0rvr3tzd3eP+ZT8kPLzh4NGcqzRBXnpkL5ucewt7eCbIqJKeGdfkm5I8X7FsAzWu4bZQiWp53c2KpoEm5e2rwGLGETI4gAyjzVK41ierqWIHvHAjPuwiXIRmN/ibtcjtDkDz43NsFOnHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Qbz49nUoSSct90DRuc5faIaPrcHn0RvfZzcOrFLGObg=; b=CO6abEhEfnvaEBoIH2hOrG9qsGF/TW+utOQvrpg4iBaeVRVFPvtBW/s2OX9J9EBdmeNEgWj8NC2EChtJPomFW8BSEoyT9egN5L/Ia7TtdCh6oUKlA2+l9qtRrwaaHRnEjquti8UJ7vkXVuq4uKq7/iL7vWqfbnrmdwVC0B+JGePbo1pICp/zTkOK83hHNJCDxbqITbwhYqfI29sxMKHcT1dRnvS11+ywK9hwQ+Ce4+eL+bTlaak0UBNyPF8ly1xpulEol4CYGVNeHVyW8TBAEMc8o/UmJHUqyKr3cbkrq4zef/G5FEp0yLM+ZNaUYeqEvbPLOtoq3BdtePri7rb8xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qbz49nUoSSct90DRuc5faIaPrcHn0RvfZzcOrFLGObg=; b=HOz9K0KP5kKC+l5/OL8LaS6lvVvP1CQ8eYfbCNvwB3xkTZPzS/k4DEAPhppjrYdO8SNGhpOVBfPKTux44kcYtdr29KPN/NPDXyGth00WM5XUT/avZE2kvOSlRbfkgIeccgsl39nG4qe+P8U/T1MRUItMt2PQFSiIyYZeamWGXW+3D480x98CjNLyYo6UINmu2JHr8fS3e0A7WAbUJTFBARF0a5abOW9Jgw2iS3rgi2K5kAGyV4PtAlc2YChy+4sA5TaYUTD35tofL+GfDIBHuGSLJnpWApWMYtb1CN6Wq5CEiCtLvlJjB3Ob1EdQhA1hCEhgL65G5MRopFBNJfiT7Q== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , =?utf-8?q?Ro?= =?utf-8?q?ger_Pau_Monn=C3=A9?= Subject: [PATCH v3 4/6] vpci: restrict unhandled read/write operations for guests Thread-Topic: [PATCH v3 4/6] vpci: restrict unhandled read/write operations for guests Thread-Index: AQHZVrdz9O2stZ5aY0OrZqeMa7i3ng== Date: Tue, 14 Mar 2023 20:56:30 +0000 Message-ID: <20230314205612.3703668-5-volodymyr_babchuk@epam.com> References: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> In-Reply-To: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.39.2 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|PAXPR03MB7967:EE_ x-ms-office365-filtering-correlation-id: c2c75151-9880-435d-73ee-08db24ce96c8 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wa4gmeoamBJ7kZQJtyqzNdldYAYGXkuFnGB0JtMBdWWknGT18z9mZxt/TvA/wCc4+pOtaRExJnIkofE/uYoo+12Lp2R9nYGV0ZEVIYsLFCCVI7b15GpjOpExw1BX11sJSEatn2SudglpRb39ZOteCkhu2VCidDaMI/Yz/a9/FicfCOZP6WY5KiF4gBf3+chtS4gIuBjx8Fd8d0IgP/mVX0XVCOYuw9UbR76YCvz6I5eBPAL57zpUduQbNDg3CCvTIsGHRKK2UUYlRSa9WqMj15kB9VXJ3C1uzQjo/gMkRnq5CyAanMmRO8+CfTkcI3DwcyOpfgv57JIiWQOzgTRo7FzzJndd2JJM5okxrn0tA3ECsmo2XKAGYphhTvoO907IAkv9BUvqfanlC3Zuc2q/eukHhDbUDXgCgKcBNtjm7aeCf5udh/qkmcQsb1ZlM9r1rez9j0XxrxW+pgyeRUsrzR33Yh2wPQX0LAdOwwnFVaMCMURJerVOuMea71OFTnkqvChCduh5eFtZbOPmNEyeoN043fRW8glKQ+SFGnBv0mVw8YB69v9CkEBe5C/cyWizTL4KJNe1nOsQFBvDGTd5gsFjuhtbBKcPDWlj46ARbgaYZyWIBeoh3JlnZZN/cNqhv7qkev8HNvr8LHzgTqhL7p896G8alPlEG7car4eaHDFWnkQ9OcCnzAFv38tZzLQYmWy8Xlki51DpN7Ch8atzMQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(451199018)(54906003)(316002)(86362001)(36756003)(38070700005)(38100700002)(478600001)(122000001)(55236004)(83380400001)(6512007)(6506007)(2616005)(1076003)(186003)(26005)(5660300002)(6486002)(91956017)(41300700001)(6916009)(71200400001)(76116006)(64756008)(8936002)(8676002)(2906002)(66556008)(66476007)(66946007)(66446008)(4326008);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?q?4TDx9YPjNLFLpEBZ1S4otq+RgnDD?= =?utf-8?q?UmcdrWlYfKCCe7UKkZlpczsf0QVBtmuQoGH8q0dQRBnZLeJ1hqLKVc36pzQV4P2TT?= =?utf-8?q?Z8ddNP5zBy7lTcYTZsJaHrfkG3WwOvN7OcIDcncRZ06IqIzF9f8JgZC0h8vZOSrFn?= =?utf-8?q?6gF+d7YXQCuQsS2bO/leWD6oTR0vW4HIdoMtgXbKJhyPmljBCLh0hNzXu+RqUiDz+?= =?utf-8?q?obA5RsD4a9lSVoaOWty9XpzdE7SLfq5feJjKZrPF8rJkSdqT68NFC4Tw9EtAUnSyd?= =?utf-8?q?w4T9m0tUYRLDwuUPcof/uNuSyDKPSPvZfFCFBmXdJj37u45te1PPZ3q0SwzsOmNHd?= =?utf-8?q?lb8NSBHmPMW2FRyOTs5xkApWWIxhLHsuH5eiMinq29Mq2c3dpNQxSyK9JvSXCDupo?= =?utf-8?q?rHdhqTlWX4WJaLMnaYUFQT4DWLBqt7efrX5mBPl96szCtlitpoam/ECTGFcJi+gs1?= =?utf-8?q?vlFUdHvfia4/5PjSifCDkFY/3/pYoTI77dP88yxshEtxKc/lfk74ZlAtjT+SKNVeA?= =?utf-8?q?wTiTooUG4tKKP2AdlGU3nI1wvRl6YP1KEfi4r+dlTE/1PK2cdlLfxI7oF4wYaTXio?= =?utf-8?q?oRKG/ZtUepaclvS3r5MZTfP0qMyPJFegf19GpE9zYgDcOev3ho50ZM8noEJFkltmy?= =?utf-8?q?1oRbvVcUnxUd4ayzO6Q87/h+Dv3Aium8ZqZl7o5ysw+xXzn4FZM6jdfe3Amx0sb1s?= =?utf-8?q?xrOyMbWvo/JUrFPe6PdBQ6oJ9PUFE2KVKV56KsknaHKnE5KNbeXEWv+qTYy6fsBjI?= =?utf-8?q?et4ieObto/xI90NQH7sDvJVijR/Fuqj9sUC+EsdDDGSCChbua1g3hUAefk4zGUajr?= =?utf-8?q?vyDBk1Ih0oCDaJmsnl6r/M8ybg4txbNDzT0x9XkHtshs6Nse2+/dqcijTqzHaGOvf?= =?utf-8?q?0I91TQ0ciuOzfNVtVybf8cbJBw2tX8tklDMLnBcQ7mDT1d5Hj7g1KQ+UitAvK2D/I?= =?utf-8?q?ORAPKxqA4hsAkfEMB4detl5QHBwtNJaJCUi/Q0RSY+4lhIiv1NeqISbKUspwmYJoA?= =?utf-8?q?6iuAC9QAkCjtwa6vd3vNG92cPhpn5YlCM3H+bndXYbLXE35gz4MCerFzy8U9g7aIt?= =?utf-8?q?tSEyu/pkn96ZJ7d9wsbQMmKHNrvUXZVThdERGw6UbDcMxttAZQ+rQCmsHk911Q+QF?= =?utf-8?q?co96xvXQGkOW3C5gdFfvp1Vna9IkMA/P/WEbsL7VHa73W+dssHQ3PAEqOF64t0Lc0?= =?utf-8?q?sWKdgwaQpvhi76stq2QYs1cLQjCUnJk03SF1v2iGGcP6MYy4MHjT5Q7MwMv+rY7lh?= =?utf-8?q?dUDk6IDu+ZzBmCPcnCu2lizSoLhYkkoTsrmpnDP1eFgcUHqOU2WlmQpUgrc8rGKHz?= =?utf-8?q?G4tkK2Zkn5iL969NanhujK+/W0UhAwLKaQyfDXPnGsMU9dic6Ag69IA+Ebpsv0hCg?= =?utf-8?q?rBTZl5+n29UZ4B6CT3YRD9vYCTZocwd+rKK3AhEt9XU/imJ4NfnzjSiRxQIVIt6Go?= =?utf-8?q?GdYSv6VAFOm3iOT9dv3VssJtmnTNoZjsJ2JgENikykxb5f5BVdjy0AJZDb7t5/seS?= =?utf-8?q?OxIvug1bruwf8K77W+Hcy467E/GkOS3xuw=3D=3D?= Content-ID: MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2c75151-9880-435d-73ee-08db24ce96c8 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2023 20:56:30.3233 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BeVFOYNB++6ARqohR3O3rI9+aHy3QY0383f08DWTZgGCjG/QVv6mxEPV67ddYkN8Vnh+V67cigIFXCWt/61SbwN4ww8Na+sxMJbL4TgSNHo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7967 X-Proofpoint-ORIG-GUID: Aw_rLXq1YvBHF0Xoz0t-oX8wAnBrjMxz X-Proofpoint-GUID: Aw_rLXq1YvBHF0Xoz0t-oX8wAnBrjMxz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_14,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 priorityscore=1501 suspectscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140168 From: Oleksandr Andrushchenko A guest would be able to read and write those registers which are not emulated and have no respective vPCI handlers, so it will be possible for it to access the hardware directly. In order to prevent a guest from reads and writes from/to the unhandled registers make sure only hardware domain can access the hardware directly and restrict guests from doing so. Suggested-by: Roger Pau Monné Signed-off-by: Oleksandr Andrushchenko --- v3: - No changes Older comments from another series: Since v6: - do not use is_hwdom parameter for vpci_{read|write}_hw and use current->domain internally - update commit message New in v6 Moved into another series --- xen/drivers/vpci/vpci.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 5232f9605b..199ff55672 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -220,6 +220,10 @@ static uint32_t vpci_read_hw(pci_sbdf_t sbdf, unsigned int reg, { uint32_t data; + /* Guest domains are not allowed to read real hardware. */ + if ( !is_hardware_domain(current->domain) ) + return ~(uint32_t)0; + switch ( size ) { case 4: @@ -260,9 +264,13 @@ static uint32_t vpci_read_hw(pci_sbdf_t sbdf, unsigned int reg, return data; } -static void vpci_write_hw(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, - uint32_t data) +static void vpci_write_hw(pci_sbdf_t sbdf, unsigned int reg, + unsigned int size, uint32_t data) { + /* Guest domains are not allowed to write real hardware. */ + if ( !is_hardware_domain(current->domain) ) + return; + switch ( size ) { case 4: From patchwork Tue Mar 14 20:56:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13175001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A789CC6FD1F for ; Tue, 14 Mar 2023 20:57:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.509832.786412 (Exim 4.92) (envelope-from ) id 1pcBhL-0001tv-Pv; Tue, 14 Mar 2023 20:56:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 509832.786412; Tue, 14 Mar 2023 20:56:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhL-0001to-MX; Tue, 14 Mar 2023 20:56:43 +0000 Received: by outflank-mailman (input) for mailman id 509832; Tue, 14 Mar 2023 20:56:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhK-0001PN-1W for xen-devel@lists.xenproject.org; Tue, 14 Mar 2023 20:56:42 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b6002c49-c2aa-11ed-b464-930f4c7d94ae; Tue, 14 Mar 2023 21:56:37 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EKchDp017378; Tue, 14 Mar 2023 20:56:36 GMT Received: from eur05-vi1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3pb0520156-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 20:56:35 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by PAXPR03MB7967.eurprd03.prod.outlook.com (2603:10a6:102:21a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Tue, 14 Mar 2023 20:56:31 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e%4]) with mapi id 15.20.6178.026; Tue, 14 Mar 2023 20:56:31 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b6002c49-c2aa-11ed-b464-930f4c7d94ae ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A75yNUZY7TFYc8MqS2xbkx9820TE4p03QsYhwvDwJceTcShrdZv7AK+0gzdqetMeJXNGRTfTDiWcJqWx9wCklXHTEWMIVACcFVeRq5TM1Lcns9hl5y83t9eDB06AUghwi/ll4Um7AIZbgTeD/hhtDy8NezgFnLVYgsGsVA84XR89eCYXm6uRp1ooKF3RaIAOxMjq5NiJLRXlbb9ZLQbHlrqmCCLPQuBZ6dTk7qMCaOhyVP0sF+emBSzumhKuNQ7JgRXSFZRmC0wswS/HXjfdcT+Auh85ehXwRSdLX6XbyK5txtINdPv6JC+w6YxT0BTdcuyiI+ISCqZaRwlk8WTfkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DQPuJAI8PAXGvKTZe3ed59NbyldzeM5G1uslGFxDTuU=; b=O1qk2JsQoh7mjTZ+X7BQTKpKfycIQNoy/qfeAPHRBWWoUEBENrzfUYpLgud8zrzt8esGhlM1N/AZKw0mrje+RHHGuxkA02iSJ3gvZQUWigx7rhV+poWegVnN5zafMg00UFpZZEIac1CGKK25PhX9FVucBfAgXMAk3JNR4pGyRWzwgLLMs1DkMoqHsNpdN5R7RKyCWw31uXd0qu1/mV6EFk/0KzVSZpG4ewsKo/+P7YSzokr9vgPYZAjvVQttaXa1Ni+ZJS+qp8tAYXmbSu+ZNzJ3MHs62tXb13ZmtQ/plssIfx6t2zl3MW3HTOsoCvTYphrize6/VwGdwkv2l3RXkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DQPuJAI8PAXGvKTZe3ed59NbyldzeM5G1uslGFxDTuU=; b=UiTw8jzdxgvZxtmFry3mjQvEo5+fhDcUaJIM16EMoUp0YQyPgQFnCWRP5HkbdVD+FQbk6Zrg1Rq6Q2PPP/aD7TAjlM0ZFW3c4Za4A48wsa/sL9nzXsKAVz29ckN8lURquHgsjmJmLsd7/EW4FpXqWrneK++EJ0J4kdkJ1lWqOyb2xOZzSwP3H4kyTwI8WRmoAxrpclY2MGpWOPgePOqFogvrRQMdcdD3TpNa+DvgP9yF0GBYjdy51sydfBg+Z5PpL6LlJFlJ/pcQYdhYAEbe10WQas4Ju5iJywbTbt55nML72w0Ylaxe1FU8UETiN+icEvdaq1vra4fS9qC1gR33vQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Volodymyr Babchuk , =?iso-8859-1?q?Roger_Pau?= =?iso-8859-1?q?_Monn=E9?= , Jan Beulich Subject: [PATCH v3 5/6] vpci: use reference counter to protect vpci state Thread-Topic: [PATCH v3 5/6] vpci: use reference counter to protect vpci state Thread-Index: AQHZVrdzxSa6ccK6DkSWhcHx4QyzSw== Date: Tue, 14 Mar 2023 20:56:30 +0000 Message-ID: <20230314205612.3703668-6-volodymyr_babchuk@epam.com> References: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> In-Reply-To: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.39.2 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|PAXPR03MB7967:EE_ x-ms-office365-filtering-correlation-id: 26255621-4380-46a2-ab63-08db24ce9701 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qRSf0D0XD2GpdU2bv50hNlcA57M76+JincdXVf8vD+N5EpfrOHfX0K3HzcZsy3EgrZF2fLnQlTExI/3jy8h/BvT012FbrJDASZ2CLot/bdBeeXA3b3FgTEswr7VOrd9JZXTxAGqVk/f8zsgvSoKtv4CFPr5/fKQ8eRZw9i5unNCCt2GTBCL5IVA2DO/dIKOK+FaYN3qPyfL3EwEd8zRPPc5E3NUDqq9ZEqDaKtRvURy+QXEfx9ZOn0LVLbtCcPSicf21B36XCjWL80HUm8UKb7gxiqUBxfW9DlH1T/eR+M1c1tkff1EDBd6D/v/herLtx+sHh2yp15w9Be7k044Lbs225mMWTasXEYMfY+1RK0o9qazRbTvDJ2Cmg6g8EueppUZWwYOiiC5qWV0HMwUChAvztA0j8NTC59GVZ6T+Vao1UT96qLXFFzNC6vxx4UiHCnPkBsMmItuBSNgKBB453Po8rYUpmwE0WsA93WZMrI+i/8hlj/siamDsFgnrMEIm9wq4NshqrQBZ92QsAJtXtaVhxkTa7hvXlLSJWG6A7uzBBJZ3gTq9du+ClbgCvBI2q07XKz4USOdWThwsUVX/MypkqZq+QHGIz18Wuk29Y57OaoqpwBkCs0LvS97Ytz6cn6cMK/e8JV1NLqjW+EsEPMvyh+fLQLbAyH53CVXB5qePO1B1pS6M64BskJM1RirQ9m4GhD7IaQEUjvcrJHXF+A== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(451199018)(54906003)(316002)(86362001)(36756003)(38070700005)(38100700002)(478600001)(122000001)(55236004)(83380400001)(6512007)(6506007)(2616005)(1076003)(186003)(26005)(5660300002)(6486002)(91956017)(41300700001)(6916009)(71200400001)(76116006)(64756008)(8936002)(8676002)(2906002)(66556008)(66476007)(66946007)(66446008)(4326008);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?FvQ3q/HyvvcHjFSbDmpNzjl?= =?iso-8859-1?q?oqqdajZkphgF8raWBKm9/zb2hdGrJFDKR8qrQonixFYSry3FNJUpqg62OepX?= =?iso-8859-1?q?f5/PrI/DYTwVRTNASiAaQDuQiajd32iY7AVGz62YHW8j2ZxE2ZYWiFc4Omcy?= =?iso-8859-1?q?bm3kXH3MMGlUxBBcLf4syDWtPpcr+FtKcxXGarxIgiXkbx5bLcE3kLqTiXGz?= =?iso-8859-1?q?V6/Z/ticrWEzEOHfphhfr6QRU0PF4xBVwP4uRHZASKwTT9mmVXkxAyCLGhKw?= =?iso-8859-1?q?HSab9eUukJzS23s+2UUEFEC/mlXnr5bytgphYR7tRjyx+g5f+vX+XZm3ahfD?= =?iso-8859-1?q?7UalqTaYzWPBZP1GR6DigF1giS15I6ZmshFiYL4OQ82P2l0SCHlSYge7Nfi7?= =?iso-8859-1?q?jFJ9zuqYcx6Gkgu3bxhN8ly+1zgXxeCOKs2TYVvWCeni4zCmsaSmDX/C3qTO?= =?iso-8859-1?q?WPR+oB4QYfZAyfhzXD9oXXGGGxLahqLp4VTlj6noDHdckJLkwZESjdaUQ34L?= =?iso-8859-1?q?5aAcJKPtkkbaS9bOklMnr9v9K+3Xfe5QB2BD8UJcKhYU1v+VwdMlvQQlyLek?= =?iso-8859-1?q?9FsAHTeDxLe72ROF0Xf92qVJqG73G15O8NSs4mwofKwoRMS1dAnBft1qmWQd?= =?iso-8859-1?q?98Xtj6vxwhcxqMARbpDReqmkce6k/j8mQVhEBZxMhKGJcF4IScbGVLT0mW9M?= =?iso-8859-1?q?j9Gfnm3e+GRt1hBxZagpv0RttrQbCmtxucdlIVEwCmsz7x4UQiG5RGJBXwyP?= =?iso-8859-1?q?lyLp7jYbrUOyimWxIJ5XPheli+zGNjsqWGm3OrMTPDaApAaRI7v56DkYf7H8?= =?iso-8859-1?q?NPfAmIqEics0Ls+h3Xq0fzT1y+DxDkLZ/4T3cjjMswJbQcgXoXwFqgPp+4mc?= =?iso-8859-1?q?0WSnNUz8YK8IIJ85OmUD0TMtSPRq6S3cakBW3qvo19uv40IMpImVIqwtp1k0?= =?iso-8859-1?q?WTNkC8/v5Mava4sEpZixjKcq348EVLZt3liNE+rFFXMIgG+Chy5rxDCq1RCj?= =?iso-8859-1?q?zbbtYJ6hmIe91PL3y6f0Cxu4uHUmVYU7PzsZID/PXItg6a0xOkAXT1d31Oru?= =?iso-8859-1?q?fPZboDcj3GWWeVU/2somFWxNHH9uO+TPEKHsjjVahw/wrypX7TqgociTZmJ6?= =?iso-8859-1?q?mu4lO2W9U+AItHeGCqlyz5rB7k5QeUsp9zl5CXCLYECoRH5StHpL86JjkrXW?= =?iso-8859-1?q?J6XkEgVPn3W+nititDRoun3oqTEVgQc9khyNbadMN2edUni7fNWwzYZXkJTg?= =?iso-8859-1?q?IzP6ocnoNwhDvDDZanumgid9FR+T84nUpSaxm9DdDLoVxXxtD3QsqVQoF3Ps?= =?iso-8859-1?q?ZKqIuXNOqUINkwalrGg+pI1Xn9XOsZHoOfXqXncrpFgzLGp8dnG+Wc+UMQhb?= =?iso-8859-1?q?9Zl15nMGmBBt7Ep8cLYtHGQUcSwrFKgAxtJ1pEO1kG+VOFz+dyoT+9FDqHl9?= =?iso-8859-1?q?9qCR6bCPmgof+0r9ak8kenqZf6pHr7xNYWSULykz07yEVhzUDoHcphkSv46+?= =?iso-8859-1?q?OAp03Aw6cZiJOJRkKLeUEx4fKv96B66G1plGezLZaolpZJ0bGJAq/7Lwm26u?= =?iso-8859-1?q?Is8iGQy5EDJqGkk3zvrsrRfsLaXOgp7LzAfzDHDlcIV0HCkYFOSYiaxL42vG?= =?iso-8859-1?q?rIz9z1A6fJTFzZjYeFndzk2r9qJ8ibjlrTC8mgw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 26255621-4380-46a2-ab63-08db24ce9701 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2023 20:56:30.6514 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: U+yqjCsWYLdanZWrAoJ8UjOLqhwKwV2ARK/YwxdqbO5o37mM5Uz2h1Ik2Rs4LiAdhSZO70sI2nI+YANvZn+ZKu8qVpIopuzkaQcpfU32Fn4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7967 X-Proofpoint-ORIG-GUID: 2uH3HCuWR2ZzSu7IWt3Rd_WKDmon7GdU X-Proofpoint-GUID: 2uH3HCuWR2ZzSu7IWt3Rd_WKDmon7GdU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_14,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=754 priorityscore=1501 suspectscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140168 vPCI MMIO handlers are accessing pdevs without protecting this access with pcidevs_{lock|unlock}. This is not a problem as of now as these are only used by Dom0. But, towards vPCI is used also for guests, we need to properly protect pdev and pdev->vpci from being removed while still in use. For that use pdev reference counting. Signed-off-by: Volodymyr Babchuk Suggested-by: Jan Beulich --- v3: - Moved from another patch series --- xen/drivers/vpci/vpci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 199ff55672..005f38dc77 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -62,6 +62,7 @@ void vpci_remove_device(struct pci_dev *pdev) xfree(pdev->vpci->msi); xfree(pdev->vpci); pdev->vpci = NULL; + pcidev_put(pdev); } int vpci_add_handlers(struct pci_dev *pdev) @@ -72,6 +73,8 @@ int vpci_add_handlers(struct pci_dev *pdev) if ( !has_vpci(pdev->domain) ) return 0; + pcidev_get(pdev); + /* We should not get here twice for the same device. */ ASSERT(!pdev->vpci); From patchwork Tue Mar 14 20:56:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 13175002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5101AC05027 for ; Tue, 14 Mar 2023 20:57:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.509833.786422 (Exim 4.92) (envelope-from ) id 1pcBhN-00029v-0j; Tue, 14 Mar 2023 20:56:45 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 509833.786422; Tue, 14 Mar 2023 20:56:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhM-00029k-U3; Tue, 14 Mar 2023 20:56:44 +0000 Received: by outflank-mailman (input) for mailman id 509833; Tue, 14 Mar 2023 20:56:43 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pcBhL-0001PN-1j for xen-devel@lists.xenproject.org; Tue, 14 Mar 2023 20:56:43 +0000 Received: from mx0b-0039f301.pphosted.com (mx0b-0039f301.pphosted.com [148.163.137.242]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b66c9f33-c2aa-11ed-b464-930f4c7d94ae; Tue, 14 Mar 2023 21:56:38 +0100 (CET) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32EKchDq017378; Tue, 14 Mar 2023 20:56:36 GMT Received: from eur05-vi1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2175.outbound.protection.outlook.com [104.47.17.175]) by mx0b-0039f301.pphosted.com (PPS) with ESMTPS id 3pb0520156-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 14 Mar 2023 20:56:36 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com (2603:10a6:803:31::18) by PAXPR03MB7967.eurprd03.prod.outlook.com (2603:10a6:102:21a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.26; Tue, 14 Mar 2023 20:56:32 +0000 Received: from VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e]) by VI1PR03MB3710.eurprd03.prod.outlook.com ([fe80::967e:573a:15a9:176e%4]) with mapi id 15.20.6178.026; Tue, 14 Mar 2023 20:56:32 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b66c9f33-c2aa-11ed-b464-930f4c7d94ae ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mXcNR+EGVCiTxLHxLV3tWW7NBXi9iMnOsCAGFKzSDxvXZ2gHg2XlA/HvF0xVYiX3fAm8fmsysSLPIdVdBUtnSD1tMV4ZY7EhMRFLxmc7MEWIsdfpcWuTYymOw3fSSohcL/dnz8I1CiWXht/Eqxvb8r3zQ9w+mPNrwWhdcFdawUnsHZvtJzaMs6WJGOB1G8+PCc0bRrLVCqLJG29P6Wc4Rtpqvv5N/bUQhB0p+g2q9EhKYu5RgBYTit/rBXFEEx4aL1iiBFvPl3wkZwRusHUgjTA4PmtxaKHhrWkvpUSao/PFQUIzOwBic4FtkSlZEBuSGkvkXJj1orSiUhkwMvuQMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xaQsE3qqfsC6jF/uvxz8iK+gfBz0ZgRCtEGe75+aN/A=; b=XNLgugBkLKiBe/rSZil8+ySzIuZRqGQ2OqKj+SV/XXbrNK9j3YG2MZ9XHkTsYu7ZUda++RObufDo0xraBDbrodJMnBPVJq60AG84+FabmSGz1emz4CGJQBE5Q91be70Gqgw0t228vMcBKdIl/gXCU7FWkBRFAjY6lwqlcU9HgiOEp4PeRQbxxPsvrQQk2AVOvCirDFXHOHxbu8V4a+rWzbKkxH9AlMf+YcxZnaL1bcCtm4NNcATI08bMB/c3y2U071zfRYSjf6mP0QFWLdJQIVijtQWOL/VL6gXZncytG5y3gTLap8m9E6Gs0Ri61rWVw6JNW48tfKkS927WP2zgLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xaQsE3qqfsC6jF/uvxz8iK+gfBz0ZgRCtEGe75+aN/A=; b=RKM2GuUBR4KaCTmx0VkWpau4f3IicAaCnasPW8dctsFXNrtMlOBWECb78UhLxWUbRmG2oMEOiTGDy7V5rUqfT5KHuKuFC3WaB6lnY9/deQJwZUaDDZiovJK1dnp4MNUTqtGWEfQM6ZmzpMWUPuF1nVK78bHkH+FtPUKspHAj2xP8BeLy22MWKSqAMnt19J7NXmTJU6IA1Uv1L2xJh3vbaLWY+RbJnoXVPF6fCURJsKPzbDR6z0bbjv25N8VBUuDYEtP3RmHURBLDUByDx6BEdcLu+YwiFVBLMZz1TEhUrThEsqorz3S2caNYWzVJPhb7pw3FNkz6e96OCIAB/LnaXw== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Volodymyr Babchuk , Jan Beulich , Paul Durrant , =?iso-8859-1?q?Roger_Pau_Monn=E9?= Subject: [PATCH v3 6/6] xen: pci: print reference counter when dumping pci_devs Thread-Topic: [PATCH v3 6/6] xen: pci: print reference counter when dumping pci_devs Thread-Index: AQHZVrdzYEuAtB/tbEeKWg2MuafMXg== Date: Tue, 14 Mar 2023 20:56:30 +0000 Message-ID: <20230314205612.3703668-7-volodymyr_babchuk@epam.com> References: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> In-Reply-To: <20230314205612.3703668-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.39.2 x-ms-publictraffictype: Email x-ms-traffictypediagnostic: VI1PR03MB3710:EE_|PAXPR03MB7967:EE_ x-ms-office365-filtering-correlation-id: bdf92573-0bb1-4c9e-09cd-08db24ce9742 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: H1FlXHmivC1+lPC5K/TiE4jnUSlMzpsQQAPnjF3Q2qZO944+yUQObHIFwh4mY69e8NBl9YU1V2BiqSp+1oC9TTx1KfsTIk0UmilzhK4TmmUFq39NsgeUPe366bUlWYH240QCvfIQGML8QGz10PFrGN0mPQw4HnVq7ozns7wSRuOba8Va21om9L9fhgcaC7FMa5NSjotCQ2FB4i2v5dGEGeDGfhcIXJXwrXN08huAJtUx7aW087wcS61soithNfBNcOpp7KtwBC6V6asSe0nfF3+kkFFX/FCQ8VrGlirO5xBAbZBXKvH4AiDVon5FgCrmI6AzJwnPx2HPa96BZMVevBV6oawTCKMZse8C5BB2tTEEL2d/88vqTUXdqv+T9qYcFgA5MoisrUTuM9NDBvvumRTPpSiFMe6RwlObIppgxa9h1Lo58RZtPSDr+5LIMPtUkf14q7+OiSehm/ShhpWpm3lRqTnGMubm5nN9yRTEfmJloyWJaA6zr0O861CaoBP6RKMoPysl8y3IQd/b5nJnXe9xj+b1ZYWVrtJ/b7uflO3s+lRcf6XCZ0KHVqP0jB+Hz5/fB9TtZeGbFyVbS7Ucxm3IRZBBXHJ+e2PQpyvE2lXY9MgA5wWHGvcZ1q0FFyHmi+1NGim1CjdxqNb7qIf0Ux5nO2ySiU6nV0gck0cIAEib/coRhOB11ZkRKeHemPuicblAdI9f+4IA9WbNYqWLlw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR03MB3710.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(451199018)(54906003)(316002)(86362001)(36756003)(38070700005)(38100700002)(478600001)(122000001)(55236004)(83380400001)(6512007)(6506007)(2616005)(1076003)(186003)(26005)(5660300002)(6486002)(91956017)(41300700001)(6916009)(71200400001)(76116006)(64756008)(8936002)(8676002)(2906002)(66556008)(66476007)(66946007)(4744005)(66446008)(4326008);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?F7QZ+3S9n0ddrUPmQ8dlpS+?= =?iso-8859-1?q?6uBtXXhNtixliviL1WoZyJXQDTe6C6avVsPubOcKWPG0+9lDAQRWFIScJXtP?= =?iso-8859-1?q?2Roa1hiFUPjNy8gsyYRAFOW7No9jiRi7liuZLG2pnNvCGTE9R8dnAssi8Axk?= =?iso-8859-1?q?yugMZ/FLtVlOo6ixb7wX6H81e8BsSqa6NJbNGgrk+NZ3he7CKzd/3eTqD6Pt?= =?iso-8859-1?q?R5CL0IJr4MUQrABCFviPc6zVeZ3wXoFZln7TmJ8ttXZssg9jlZMtDUrx16vM?= =?iso-8859-1?q?9dt2T4fg5CSRuaGcYmLyJlWWjwoDEhdcAkb5NqoJmEfMhvRZtpaE1hoWTMUe?= =?iso-8859-1?q?86xvrvH42Br1VteNoIepVohoA9sUjHHBDYuYa3MVYDSd+A59F3fvRxQZ583/?= =?iso-8859-1?q?V9XWktsgm4KfUa7ArOnHecR1UFCgKBl51HZFfDcGcdq6Ds0VUc445RcU+7fG?= =?iso-8859-1?q?B+te4YprwEUcJ1kRrhxBh0E2/HykbW2M/mXF1RAVRwKSxlTdfgVjxhElnj5g?= =?iso-8859-1?q?fEZ3uh4VAz2uNRgDj1eizoW8GF8FmbbHHjPTuPbFxLi//FYLY1MrgzgMduIc?= =?iso-8859-1?q?cdpeLcVrSRlQsgBoo0jhuBUfiRH7xX50QebBAQkZBWYMUyIWLlHHDCsaD9u4?= =?iso-8859-1?q?8V+Yvp4F/Ic9yae6AM4jWHTTM0TjVeVINj+2hTUtG3gBL9kGdA7ui6qGHyKL?= =?iso-8859-1?q?7qcjhCriNBrW7f31ipgZPhZyxGajpNMa1DV/q4vxFTZ6REFSsiEpojo53r30?= =?iso-8859-1?q?OJzHpjm8gNHp4ingQkEsHQo9jGy87pG5+7GuNsL3NMdNzUW4wk1J5LL95bQF?= =?iso-8859-1?q?4QM4slMG9P4YwgVwLPEPd3tO0laCQtawawM1psGc71mKSGIn7xMOYwE+rjbg?= =?iso-8859-1?q?ru31DK2iDplojMwscFV73Y252MFIaezK8DJYUByyr9tyX6D1Iwjd+NwpIaTB?= =?iso-8859-1?q?SUnbkxP6cNr2TgvuqBqurO50dHd7+gH9K6DeF+C4una+sfuQHJJdhUp8Qt4T?= =?iso-8859-1?q?HRwHyHk9pGsu91t1GRhxYBwaTshqMsUSRx8fAUGaneYfuq894RA7tPNMT9nv?= =?iso-8859-1?q?yRqCSy5zTaqBtLxSYWlaeDMX6jn8VguUxogQfpI1mFJGmD2vhl+AK0ZB6OE+?= =?iso-8859-1?q?OSD7qiRHG82UyBs13aePR4tRtIZrzXjkQvtsrNU9KvLHkoleDa7ScG5YmsEA?= =?iso-8859-1?q?SO8oB+Jb7pvb1UURQEhvCzNfIOzCU4lRPm2M4L48Bow8XlHmIpvkx3cN6Uvp?= =?iso-8859-1?q?XwIGXTcZ+O4xJ9TFLAti/gtfb0uPBjKYs6P9ctoOkmtDOs/p1euf0FZQXGre?= =?iso-8859-1?q?fOHXwm7mJ1vRQskDpckCWj9Uyz2vf0MU2wpzSt/zoCai1Fnp60SdR1FgfLyR?= =?iso-8859-1?q?cGfl5JrWDkHkPfEKjfH0x3U0BiqMpnBCW+yEspP6kIR66+arv+zfWGUwUn5C?= =?iso-8859-1?q?b4Om1vr1DuhJ24fAPhiepu5nSXnFEMo7JYw/fvMnRo20yYzb2DJyLAnFpHuR?= =?iso-8859-1?q?TU2DWVpENz1smaM8qCi5g/cryoTD6pD/UV8jZpgHRd7XWp30nIWmZ64C/NAX?= =?iso-8859-1?q?6AAzT0F4UA8qtJcEKRbsK/qAu+/P8BRVoOqlF0Z6IC6Zhd/GMUnSsU9SrWy1?= =?iso-8859-1?q?CmgcSUrkFdVBRiTpPdmZiIpIIGBK/1Mvs3QEjjQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR03MB3710.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bdf92573-0bb1-4c9e-09cd-08db24ce9742 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Mar 2023 20:56:30.9483 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: AmuJHa2iLKwhJfDQ8eTCQv4E37gl8XIXhJnroxY3RgpJcPMGMevjXScIg7Trou3jW4FPZYPe7YMIBpy6NnliqxO/4bSQMgmkI9jKk22Z7ns= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR03MB7967 X-Proofpoint-ORIG-GUID: YnO7ExG08MaqGkmHHP97gzjEuEfTlmQh X-Proofpoint-GUID: YnO7ExG08MaqGkmHHP97gzjEuEfTlmQh X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-14_14,2023-03-14_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=823 priorityscore=1501 suspectscore=0 clxscore=1015 mlxscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 bulkscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2303140168 This can be handy during new reference counter approach evaluation. Signed-off-by: Volodymyr Babchuk --- v3: - Moved from another patch series --- xen/drivers/passthrough/pci.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index b32382aca0..1eb79e7d01 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1275,7 +1275,8 @@ static int cf_check _dump_pci_devices(struct pci_seg *pseg, void *arg) else #endif printk("%pd", pdev->domain); - printk(" - node %-3d", (pdev->node != NUMA_NO_NODE) ? pdev->node : -1); + printk(" - node %-3d refcnt %d", (pdev->node != NUMA_NO_NODE) ? pdev->node : -1, + refcnt_read(&pdev->refcnt)); pdev_dump_msi(pdev); printk("\n"); }