From patchwork Tue Feb 23 02:34:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Volodymyr Babchuk X-Patchwork-Id: 12099775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,PDS_BAD_THREAD_QP_64,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E8EAC4332E for ; Tue, 23 Feb 2021 02:35:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0E31464E41 for ; Tue, 23 Feb 2021 02:35:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E31464E41 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=epam.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.88519.166467 (Exim 4.92) (envelope-from ) id 1lENXf-0004IF-75; Tue, 23 Feb 2021 02:35:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 88519.166467; Tue, 23 Feb 2021 02:35:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lENXf-0004I4-1Y; Tue, 23 Feb 2021 02:35:15 +0000 Received: by outflank-mailman (input) for mailman id 88519; Tue, 23 Feb 2021 02:35:14 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lENXd-00046u-V9 for xen-devel@lists.xenproject.org; Tue, 23 Feb 2021 02:35:13 +0000 Received: from mx0b-0039f301.pphosted.com (unknown [148.163.137.242]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 721913e0-99e5-4f4c-9933-dbdb10ca54f0; Tue, 23 Feb 2021 02:35:04 +0000 (UTC) Received: from pps.filterd (m0174680.ppops.net [127.0.0.1]) by mx0b-0039f301.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 11N2QQxk004083; Tue, 23 Feb 2021 02:35:01 GMT Received: from eur01-db5-obe.outbound.protection.outlook.com (mail-db5eur01lp2052.outbound.protection.outlook.com [104.47.2.52]) by mx0b-0039f301.pphosted.com with ESMTP id 36vqte83qr-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 23 Feb 2021 02:35:01 +0000 Received: from AM0PR03MB3508.eurprd03.prod.outlook.com (2603:10a6:208:4f::23) by AM0PR0302MB3235.eurprd03.prod.outlook.com (2603:10a6:208:a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3868.32; Tue, 23 Feb 2021 02:34:57 +0000 Received: from AM0PR03MB3508.eurprd03.prod.outlook.com ([fe80::a9a4:6122:8de2:64cb]) by AM0PR03MB3508.eurprd03.prod.outlook.com ([fe80::a9a4:6122:8de2:64cb%6]) with mapi id 15.20.3846.042; Tue, 23 Feb 2021 02:34:57 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 721913e0-99e5-4f4c-9933-dbdb10ca54f0 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n8UsQ/zNRlGvj/CMCAtMQlV8xoeGe/qY82ASPb8bsv1EJdfCD5A166Brx+0DHQCkbAvOD1j0OJ7Rury/pO+pMG69Q/YnB0J4EimaaZTI77s1IgEpb5JMteMdHZxmAd3/fcXj8hcFvvZRcIGP8pKEw691wnOqPbCHPzi1y+rFYFCe7dafaFJOy1ewuh04VM4e5PmlkjjRuWqsmOCYtuXQ7XdgDKEKawldJ28zOD/M2QkCa4xC9uFKERuBr1193ecVyrBJ36iqOouoStzvs5ZUhysNObYjnBrSzNwAQx6pXyFMnQxtpn8PsAXILqS3kEDPb/L3LGD3JgM9JKJMgVl+ZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z7bkZJeAVBuSp9Gyc797wEvVdAIwSUhCtPUTvuKcVEE=; b=CYpIuVSxRvIsnDd2YCEd7pN6Q2wWhZkEbUaFx1JQjeRvPZMICDtYtdkrltjVN2DVPhhiVnZIi8XIZgD66iqoAQxRa0OIevf6+98UwhOkgJnMKEcR4M0DeP8JqnTaBJ5ZCCmRqtj7aWvozqUrzlFzy2/jHz7mLGdtotJE8iDlNmSL6lVsTVrotp7CO7IYSlocaAJGuXH+DDRX35mzl3tWxTPICziYo273iHLH94nocdeZE6XarCAN/VlClsVDF3kMAN/ej0AsGgp/xrtN4z1WIo3pC7vcR7KsBLcap9yYpc/UwOpHxpm4ZkcPflDVVejpyNHP4hlAZArV8Zs+Jr8xZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z7bkZJeAVBuSp9Gyc797wEvVdAIwSUhCtPUTvuKcVEE=; b=fugEaoUmhgNcWbKy3Jbn87JBECOIM+DBEd6OK4KY7VQk/TriVRWxJk6qYa7HtezQvBznJSNn+ZTLSAgIcJnaIkFaLpNavDiz0NF1XaIVdryoRNv9EQ5AY0OVPVLQB+eEQOR6BuZR6rf6aTnoNr7qVFa3c5HON/7uS1hZ2Ymc57BwP1WOwD7rRZur3oh/ld2uDMH6fYdLCuQv+7p+3am8SWVwVLfv1D/gcWzaN77wVi5MYadAMGYDzavi2Wn74YFXwUmiGvt52nM2qpcWuE7snspokLwC5qfXGGxIg06VXhiJJkmXXSLjw8pMS05qZtKGrs8nqu1JhtoSUKdjhjYPgQ== From: Volodymyr Babchuk To: "xen-devel@lists.xenproject.org" CC: Volodymyr Babchuk , Andrew Cooper , George Dunlap , Ian Jackson , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [RFC PATCH 05/10] preempt: add try_preempt() function Thread-Topic: [RFC PATCH 05/10] preempt: add try_preempt() function Thread-Index: AQHXCYx5ySuVthNLHUKZmgWg9dU+LA== Date: Tue, 23 Feb 2021 02:34:57 +0000 Message-ID: <20210223023428.757694-6-volodymyr_babchuk@epam.com> References: <20210223023428.757694-1-volodymyr_babchuk@epam.com> In-Reply-To: <20210223023428.757694-1-volodymyr_babchuk@epam.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.30.1 authentication-results: lists.xenproject.org; dkim=none (message not signed) header.d=none;lists.xenproject.org; dmarc=none action=none header.from=epam.com; x-originating-ip: [176.36.48.175] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: cd8759b3-5f5f-4e1d-6730-08d8d7a39c3a x-ms-traffictypediagnostic: AM0PR0302MB3235: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: MhrIJOptljGFO3p3aYiupiRFD8WDZIGxT2LOroitiU2FYLsNLy7sqcVLaps+dEwnuLF3/B0VeWLSmp9kMqOwUNMUwxNGBmFGb3bfU+ayslZjWk5NTxo/vcsyCSNuq4VuSrqDxh9hCoolxbKYuHdT4VhAisIJZLEKLHFN3bIx5LW6B3zD8trTDrXqaoT8LB0RmRjrgzrEYL3cD2HFGtyouUKWbcckqSHPiukVbyphSDHHzPoO5S5crIdGmq3HZ4+vHEGmFJ+yiC7tjm9b8vu+4hDdX47/oEqsshLksylKgiNtQQiqG8AVpodCEPnbbWgYfXAtpjpLG2k1ubpiV6fHK8RvywkJbj9i+jvPBqAgKDVISDp5V4yygJMAxuWGdwDdmoIfPHO3gd4y8eiZEvjGnDiI1EoLbDOr9rzOJZd7UBBtvTwhPILjAFygF4GvV2YVZLquqy56zcDiiAm/mVXdusTU1Pbeze/qvRrjh/XX5CojxnnGTc2GFcFDJ4Eg4jaZYmFdnDDZ/hNkh9X4zE/VKfxnig3dcOdwDVQSi2tCo2gTrJ1PkZY0Z3pRE9EXtYU2 x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR03MB3508.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(366004)(136003)(39860400002)(346002)(376002)(316002)(6512007)(8936002)(8676002)(86362001)(2906002)(4326008)(478600001)(6486002)(186003)(54906003)(66556008)(55236004)(83380400001)(66446008)(66946007)(66476007)(64756008)(71200400001)(1076003)(2616005)(26005)(76116006)(6916009)(36756003)(5660300002)(6506007)(2004002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?q?C1Exbo5RCcolDWAgr5FgqW9T4?= =?iso-8859-1?q?dHuKN7wXk1WlE6QoaYMt3/QR1CBY5yMyg+2D24BJ3NSotM7zkk2mZMWCAmhn?= =?iso-8859-1?q?bLt/wWPdLJ7Y9FXTRsfHk3wp5MW2fVwAIzDfTNT1dP09xdNj8K1yimvDhXuI?= =?iso-8859-1?q?/Wt/qFAzGjIFFhH8sK2sm/Mzy+tD8+o1wi/WFMoPwDe1kdTl+Fx+AUvOEJOc?= =?iso-8859-1?q?XelS1/IFBqKNhw5SsvyAsiDu5Hi0oCObvbQxjZVj6uSBFaRzVCHwHc2Hts/a?= =?iso-8859-1?q?tlvdpkrKFRNaMGzIxnXySxacjEl/2Xbtd0LW6pOBtD7D+k2QiD+AbMnbO8Hg?= =?iso-8859-1?q?zRGhXfReETtTyRX+8DBTGKHOTGGZUgk4gD4UHMjL82XaDidijtKV/kL82bFG?= =?iso-8859-1?q?iCOrG3Oa6vQYh0hgNzN3E9LZgCKdF3J2JqPfZsQbtDooM62A8HvfntxEEsF8?= =?iso-8859-1?q?i7KBgsvFbL0s3vPx9oTIGaWcIfYH2OlXjHOgZEXq4pf0MlVwOt3+PP5dykDL?= =?iso-8859-1?q?/YjAQ6hibiTraP6BUtaRzPycX4oh0khilItJFU0RjafR5CN+pufeBKBBtNC5?= =?iso-8859-1?q?is2ebEZPKF5cW+58GNmC2Ei4XyMh5fOlQtqZVwhIIegbiniPPM2tbqv/IxLy?= =?iso-8859-1?q?VTvvJu4ruzA2MFRIQkudW9ml837l+lipmZ2oq3vUuagg3uTKgJynF4tHjnEi?= =?iso-8859-1?q?UwIhfLfoL/ijxZ37CYHejZ6hqLrbxriYWVzNoy1f3YoEJxCuBFQ9bHXGc9lK?= =?iso-8859-1?q?d6pTct07FMbRkAI0ujDIqIVpW8zNFpdJzD+uIBNmkrpogCnhmCeQDaF2N18x?= =?iso-8859-1?q?Z6a1MeGvEJgX8eCSpwoyTznVHZnZ3T4AycK1yXN93jcShE3QNY22aHpshMt9?= =?iso-8859-1?q?xss1yBFbmArwxBXGampv6cCzKeq8EqMbVztDjdyGK8o3asYWAGNXcFbQFDTt?= =?iso-8859-1?q?Rf8JLqN5ImzLEq120yznvmt8PARzJmWbPvZ0EkVKrEB1D2ymyvNFy4t1BZaS?= =?iso-8859-1?q?0araHVAEuGIAY6lsNOHGILwlrHAXM5q4+8rfpAv/XyYCy4QOZxPfo/xeZuqj?= =?iso-8859-1?q?gZf2ZfPBcFAbf01NbtfQ8rcUwAMELEAHqwdSt4+GfUIlCjR2VIGt4I581ILE?= =?iso-8859-1?q?y0VOMh9H4EK5abNhUj9saOQ7DN7KUXsOov0MAVN5Ul1lO69jWSlTr9TkjaTw?= =?iso-8859-1?q?T+V+1h5yudbJB1ovkxGDLGQQWoFs4dt2/R3DKQsW84qrNyrYShvI+NzG6FRc?= =?iso-8859-1?q?fLp9CJ2RxX2oABKj8zZvp43q96J28zlUNGEIQtZKlbNTHclhzkQIm6Ydko9v?= =?iso-8859-1?q?pXeFc4rCkvIY07ZzE9BWGnog9LwTVSU3MLdDXy/1DqfysZpf1nMMKs8dnfu?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR03MB3508.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: cd8759b3-5f5f-4e1d-6730-08d8d7a39c3a X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Feb 2021 02:34:57.0931 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: qcl/SXjF/ndaYdz1kJRPCFAz+NiQgEIuok3cm7vSoPoHTV2w2VNmvDV06qtTrEUkCfmcB2esxfpMFdKF46OwRDcnBE1H6bpwUqNf/cj+3bo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0302MB3235 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 spamscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 priorityscore=1501 adultscore=0 bulkscore=0 clxscore=1015 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102230018 This function can be used to preempt code running in hypervisor mode. Generally, there are two reasons to preempt while in HYP mode: 1. IRQ arrived. This may woke vCPU with higher scheduling priority. 2. Exit from atomic context. While we were in atomic context, state of the system may changed and we need to reschedule. It is very inefficient to call scheduler each time we leave atomic context, so very simple optimists is used. There are cases when we *know* that there might be reasons for preemption. One example - is IRQ. In this case we call try_preempt(true). This will force rescheduling if we are outside atomic context or it will ensure that scheduler will be called right after leaving atomic context. This is done by calling try_preempt(false) when we are leaving atomic context. try_preempt(false) will check if there was call to try_preempt(true) in atomic context and call scheduler only in this case. Also macro preempt_enable_no_sched() is introduced. It is meant to be used by scheduler itself, because we don't want to initiate rescheduling inside scheduler code. Signed-off-by: Volodymyr Babchuk --- xen/common/preempt.c | 32 +++++++++++++++++++++++++++++++- xen/include/xen/preempt.h | 8 ++++++++ 2 files changed, 39 insertions(+), 1 deletion(-) diff --git a/xen/common/preempt.c b/xen/common/preempt.c index ad61c8419a..98699aaa1f 100644 --- a/xen/common/preempt.c +++ b/xen/common/preempt.c @@ -4,6 +4,7 @@ * Track atomic regions in the hypervisor which disallow sleeping. * * Copyright (c) 2010, Keir Fraser + * Copyright (c) 2021, EPAM Systems * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -21,13 +22,42 @@ #include #include +#include +#include #include DEFINE_PER_CPU(atomic_t, __preempt_count); +DEFINE_PER_CPU(unsigned int, need_reschedule); bool_t in_atomic(void) { - return atomic_read(&preempt_count()) || in_irq() || local_irq_is_enabled(); + return atomic_read(&preempt_count()) || in_irq(); +} + +void try_preempt(bool force) +{ + /* + * If caller wants us to call the scheduler, but we are in atomic + * context - update the flag. We will try preemption upon exit + * from atomic context. + */ + if ( force && in_atomic() ) + { + this_cpu(need_reschedule) = 1; + return; + } + + /* idle vCPU schedules via soft IRQs */ + if ( unlikely(system_state != SYS_STATE_active) || + in_atomic() || + is_idle_vcpu(current) ) + return; + + if ( force || this_cpu(need_reschedule) ) + { + this_cpu(need_reschedule) = 0; + wait(); + } } #ifndef NDEBUG diff --git a/xen/include/xen/preempt.h b/xen/include/xen/preempt.h index e217900d6e..df7352a75e 100644 --- a/xen/include/xen/preempt.h +++ b/xen/include/xen/preempt.h @@ -4,6 +4,7 @@ * Track atomic regions in the hypervisor which disallow sleeping. * * Copyright (c) 2010, Keir Fraser + * Copyright (c) 2021, EPAM Systems */ #ifndef __XEN_PREEMPT_H__ @@ -15,6 +16,8 @@ DECLARE_PER_CPU(atomic_t, __preempt_count); +void try_preempt(bool force); + #define preempt_count() (this_cpu(__preempt_count)) #define preempt_disable() do { \ @@ -23,6 +26,11 @@ DECLARE_PER_CPU(atomic_t, __preempt_count); #define preempt_enable() do { \ atomic_dec(&preempt_count()); \ + try_preempt(false); \ +} while (0) + +#define preempt_enable_no_sched() do { \ + atomic_dec(&preempt_count()); \ } while (0) bool_t in_atomic(void);