From patchwork Fri Apr 28 08:08:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 13226052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6FBC5C77B60 for ; Fri, 28 Apr 2023 08:09:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.527188.819510 (Exim 4.92) (envelope-from ) id 1psJ9k-0003e2-53; Fri, 28 Apr 2023 08:08:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 527188.819510; Fri, 28 Apr 2023 08:08:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9j-0003dB-WF; Fri, 28 Apr 2023 08:08:40 +0000 Received: by outflank-mailman (input) for mailman id 527188; Fri, 28 Apr 2023 08:08:38 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9i-0003M3-O4 for xen-devel@lists.xenproject.org; Fri, 28 Apr 2023 08:08:38 +0000 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [2a00:1450:4864:20::336]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id df5aa8d4-e59b-11ed-8611-37d641c3527e; Fri, 28 Apr 2023 10:08:35 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-3f182d745deso94547175e9.0 for ; Fri, 28 Apr 2023 01:08:35 -0700 (PDT) Received: from georged-x-u.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 13-20020a05600c230d00b003f31da39b62sm2569464wmo.18.2023.04.28.01.08.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 01:08:33 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: df5aa8d4-e59b-11ed-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1682669314; x=1685261314; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=oHctl/+Yw4lELN5Eo4ilEJN3AsQPRKgX697uKFaIV2k=; b=Rxi3Av75ra1OLpfSrLCkapSZR9wDbu+YuPgRG88Ofyz17/gMJjaL7PYRN6mjsrsrOF IZbwtz6LnI7x7AeMMhsaHUy4Kn8lmUy70VhNAfedtUOjJwXeDT+RqT7CKurPQb24hywd qqYpvVJ+6JP7kXZFqq5TYbNB05TWFXFXGkPNE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682669314; x=1685261314; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oHctl/+Yw4lELN5Eo4ilEJN3AsQPRKgX697uKFaIV2k=; b=G/ohaeCVdMw2tcyczYPzkWzwcLceRRjjz4x4EIQlz21UkIJ4TZql+3IdnYKTla4/SH cBWc5wEtHONyaXxVIJDc/7BSfjDlslXWFdlIHITLnGE6o27hDd3jfSIkflrnhWklSAIP YQBJ+mo6gpPAZIpTWjlWZv+bmq3DDY9aekyZynA96DN3VukQjBSF4Lg9BKPUsjt7hwpD mGFMqKd3/gVPqop5tfiEGbjeMxQ84LyLwx5OSAtfpAIdK2G0WVCcnj3Pa8yuoqcR/zqe RyoVvg72MC+7SswjKgpwdTfxQVji1dAcIUvK0PN8IPYGrw4CzmrBFqaG9aiksz+NGBUt F6pA== X-Gm-Message-State: AC+VfDxWJDEkeCPS6Yng15UHp3cn/bu7EY4eI5VEHwVh/BmKwF5jr4kv oSytNuopzy4gGKwiuWRby+emdq9yVXDmEVE6oOI= X-Google-Smtp-Source: ACHHUZ7gVLP4vqO2Cl4E//bGC+5sMhTO1AwKt98Epp5tTTyGBhb4S9xMh4Qm4crO2WL6jFoaF+w9CA== X-Received: by 2002:a1c:f717:0:b0:3f2:5999:4f3a with SMTP id v23-20020a1cf717000000b003f259994f3amr3344119wmh.31.1682669314091; Fri, 28 Apr 2023 01:08:34 -0700 (PDT) From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap , Anthony Perard , Wei Liu Subject: [PATCH 1/5] xenalyze: Handle start-of-day ->RUNNING transitions Date: Fri, 28 Apr 2023 09:08:28 +0100 Message-Id: <20230428080832.2461044-1-george.dunlap@cloud.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 A recent xentrace highlighted an unhandled corner case in the vcpu "start-of-day" logic, if the trace starts after the last running -> non-running transition, but before the first non-running -> running transition. Because start-of-day wasn't handled, vcpu_next_update() was expecting p->current to be NULL, and tripping out with the following error message when it wasn't: vcpu_next_update: FATAL: p->current not NULL! (d32768dv$p, runstate RUNSTATE_INIT) where 32768 is the DEFAULT_DOMAIN, and $p is the pcpu number. Instead of calling vcpu_start() piecemeal throughout sched_runstate_process(), call it at the top of the function if the vcpu in question is still in RUNSTATE_INIT, so that we can handle all the cases in one place. Sketch out at the top of the function all cases which we need to handle, and what to do in those cases. Some transitions tell us where v is running; some transitions tell us about what is (or is not) running on p; some transitions tell us neither. If a transition tells us where v is now running, update its state; otherwise leave it in INIT, in order to avoid having to deal with TSC skew on start-up. If a transition tells us what is or is not running on p, update p->current (either to v or NULL). Otherwise leave it alone. If neither, do nothing. Reifying those rules: - If we're continuing to run, set v to RUNNING, and use p->first_tsc as the runstate time. - If we're starting to run, set v to RUNNING, and use ri->tsc as the runstate time. - If v is being deschedled, leave v in the INIT state to avoid dealing with TSC skew; but set p->current to NULL so that whatever is scheduled next won't trigger the assert in vcpu_next_update(). - If a vcpu is waking up (switching from one non-runnable state to another non-runnable state), leave v in INIT, and p in whatever state it's in (which may be the default domain, or some other vcpu which has already run). While here, fix the comment above vcpu_start; it's called when the vcpu state is INIT, not when current is the default domain. Signed-off-by: George Dunlap --- CC: Anthony Perard CC: Wei Liu --- tools/xentrace/xenalyze.c | 159 ++++++++++++++++++++++++-------------- 1 file changed, 101 insertions(+), 58 deletions(-) diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c index 12dcca9646..ff9716cd12 100644 --- a/tools/xentrace/xenalyze.c +++ b/tools/xentrace/xenalyze.c @@ -6885,39 +6885,86 @@ void vcpu_next_update(struct pcpu_info *p, struct vcpu_data *next, tsc_t tsc) p->lost_record.seen_valid_schedule = 1; } -/* If current is the default domain, we're fixing up from something - * like start-of-day. Update what we can. */ -void vcpu_start(struct pcpu_info *p, struct vcpu_data *v) { - /* If vcpus are created, or first show up, in a "dead zone", this will - * fail. */ - if( !p->current || p->current->d->did != DEFAULT_DOMAIN) { - fprintf(stderr, "Strange, p->current not default domain!\n"); - error(ERR_FILE, NULL); - return; - } +/* + * If the vcpu in question is in state INIT, we're fixing up from something + * like start-of-day. Update what we can. + */ +void vcpu_start(struct pcpu_info *p, struct vcpu_data *v, + int old_runstate, int new_runstate, tsc_t ri_tsc) { + tsc_t tsc; + + /* + * + * Cases: + * running -> running: + * v -> running, using p->first_tsc + * {runnable, blocked} -> running: + * v -> running, using ri->tsc + * running -> {runnable, blocked}: + * Leave v INIT, but clear p->current in case another vcpu is scheduled + * blocked -> runnable: + * Leave INIT, and also leave p->current, since we still don't know who's scheduled here + */ + + /* + * NB that a vcpu won't come out of INIT until it starts running somewhere. + * If this event is pcpu that has already seen a scheduling event, p->current + * should be null; if this is the first scheduling event on this pcpu, + * p->current should be the default domain. + */ + if( old_runstate == RUNSTATE_RUNNING ) { + if ( !p->current || p->current->d->did != DEFAULT_DOMAIN) { + fprintf(stderr, "Strange, p->current not default domain!\n"); + error(ERR_FILE, NULL); + return; - if(!p->first_tsc) { - fprintf(stderr, "Strange, p%d first_tsc 0!\n", p->pid); - error(ERR_FILE, NULL); + } + + if(!p->first_tsc) { + fprintf(stderr, "Strange, p%d first_tsc 0!\n", p->pid); + error(ERR_FILE, NULL); + } + + if(p->first_tsc <= p->current->runstate.tsc) { + fprintf(stderr, "Strange, first_tsc %llx < default_domain runstate tsc %llx!\n", + p->first_tsc, + p->current->runstate.tsc); + error(ERR_FILE, NULL); + } + + /* Change default domain to 'queued' */ + runstate_update(p->current, RUNSTATE_QUEUED, p->first_tsc); + + /* + * Set current to NULL, so that if another vcpu (not in INIT) + * is scheduled here, we don't trip over the check in + * vcpu_next_update() + */ + p->current = NULL; } - if(p->first_tsc <= p->current->runstate.tsc) { - fprintf(stderr, "Strange, first_tsc %llx < default_domain runstate tsc %llx!\n", - p->first_tsc, - p->current->runstate.tsc); - error(ERR_FILE, NULL); + /* TSC skew at start-of-day is hard to deal with. Don't + * bring a vcpu out of INIT until it's seen to be actually + * running somewhere. */ + if ( new_runstate != RUNSTATE_RUNNING ) { + fprintf(warn, "First schedule for d%dv%d doesn't take us into a running state; leaving INIT\n", + v->d->did, v->vid); + + return; } - /* Change default domain to 'queued' */ - runstate_update(p->current, RUNSTATE_QUEUED, p->first_tsc); + tsc = ri_tsc; + if ( old_runstate == RUNSTATE_RUNNING ) { + /* FIXME: Copy over data from the default domain this interval */ + fprintf(warn, "Using first_tsc for d%dv%d (%lld cycles)\n", + v->d->did, v->vid, p->last_tsc - p->first_tsc); - /* FIXME: Copy over data from the default domain this interval */ - fprintf(warn, "Using first_tsc for d%dv%d (%lld cycles)\n", - v->d->did, v->vid, p->last_tsc - p->first_tsc); + tsc = p->first_tsc; + } /* Simulate the time since the first tsc */ - runstate_update(v, RUNSTATE_RUNNING, p->first_tsc); - p->time.tsc = p->first_tsc; + runstate_update(v, RUNSTATE_RUNNING, tsc); + p->time.tsc = tsc; p->current = v; pcpu_string_draw(p); v->p = p; @@ -7021,6 +7068,13 @@ void sched_runstate_process(struct pcpu_info *p) last_oldstate = v->runstate.last_oldstate; v->runstate.last_oldstate.wrong = RUNSTATE_INIT; + /* Handle all "start-of-day" issues in one place. This can be + * done before any of the other tracks or sanity checks. */ + if ( v->runstate.state == RUNSTATE_INIT ) { + vcpu_start(p, v, sevt.old_runstate, sevt.new_runstate, ri->tsc); + return; + } + /* Close vmexits when the putative reason for blocking / &c stops. * This way, we don't account cpu contention to some other overhead. */ if(sevt.new_runstate == RUNSTATE_RUNNABLE @@ -7190,32 +7244,27 @@ update: * or stopping actually running on a physical cpu. */ if ( type == CONTINUE ) { - if( v->runstate.state == RUNSTATE_INIT ) { - /* Start-of-day; account first tsc -> now to v */ - vcpu_start(p, v); - } else { - /* Continue running. First, do some sanity checks */ - if ( v->runstate.state == RUNSTATE_LOST ) { - fprintf(warn, "WARNING: continue with d%dv%d in RUNSTATE_LOST. Resetting current.\n", - v->d->did, v->vid); - if ( p->current ) - vcpu_prev_update(p, p->current, ri->tsc, RUNSTATE_LOST); - vcpu_next_update(p, v, ri->tsc); - } - else if( v->runstate.state != RUNSTATE_RUNNING ) { - /* This should never happen. */ - fprintf(warn, "FATAL: sevt.old_runstate running, but d%dv%d runstate %s!\n", - v->d->did, v->vid, runstate_name[v->runstate.state]); - error(ERR_FILE, NULL); - } else if ( v->p != p ) { - fprintf(warn, "FATAL: continue on p%d, but d%dv%d p%d!\n", - p->pid, v->d->did, v->vid, - v->p ? v->p->pid : -1); - error(ERR_FILE, NULL); - } - - runstate_update(v, RUNSTATE_RUNNING, ri->tsc); + /* Continue running. First, do some sanity checks */ + if ( v->runstate.state == RUNSTATE_LOST ) { + fprintf(warn, "WARNING: continue with d%dv%d in RUNSTATE_LOST. Resetting current.\n", + v->d->did, v->vid); + if ( p->current ) + vcpu_prev_update(p, p->current, ri->tsc, RUNSTATE_LOST); + vcpu_next_update(p, v, ri->tsc); + } + else if( v->runstate.state != RUNSTATE_RUNNING ) { + /* This should never happen. */ + fprintf(warn, "FATAL: sevt.old_runstate running, but d%dv%d runstate %s!\n", + v->d->did, v->vid, runstate_name[v->runstate.state]); + error(ERR_FILE, NULL); + } else if ( v->p != p ) { + fprintf(warn, "FATAL: continue on p%d, but d%dv%d p%d!\n", + p->pid, v->d->did, v->vid, + v->p ? v->p->pid : -1); + error(ERR_FILE, NULL); } + + runstate_update(v, RUNSTATE_RUNNING, ri->tsc); } else if ( sevt.old_runstate == RUNSTATE_RUNNING || v->runstate.state == RUNSTATE_RUNNING ) @@ -7232,10 +7281,7 @@ update: * # (should never happen) */ if( sevt.old_runstate == RUNSTATE_RUNNING ) { - if( v->runstate.state == RUNSTATE_INIT ) { - /* Start-of-day; account first tsc -> now to v */ - vcpu_start(p, v); - } else if( v->runstate.state != RUNSTATE_RUNNING + if( v->runstate.state != RUNSTATE_RUNNING && v->runstate.state != RUNSTATE_LOST ) { /* This should never happen. */ fprintf(warn, "FATAL: sevt.old_runstate running, but d%dv%d runstate %s!\n", @@ -7264,11 +7310,8 @@ update: vcpu_next_update(p, v, ri->tsc); } - else if ( v->runstate.state != RUNSTATE_INIT ) + else { - /* TSC skew at start-of-day is hard to deal with. Don't - * bring a vcpu out of INIT until it's seen to be actually - * running somewhere. */ runstate_update(v, sevt.new_runstate, ri->tsc); } From patchwork Fri Apr 28 08:08:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 13226050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B56BCC77B7E for ; Fri, 28 Apr 2023 08:09:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.527185.819485 (Exim 4.92) (envelope-from ) id 1psJ9h-00037G-CT; Fri, 28 Apr 2023 08:08:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 527185.819485; Fri, 28 Apr 2023 08:08:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9h-000379-9W; Fri, 28 Apr 2023 08:08:37 +0000 Received: by outflank-mailman (input) for mailman id 527185; Fri, 28 Apr 2023 08:08:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9g-00036x-G7 for xen-devel@lists.xenproject.org; Fri, 28 Apr 2023 08:08:36 +0000 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [2a00:1450:4864:20::335]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id df9af7b3-e59b-11ed-b224-6b7b168915f2; Fri, 28 Apr 2023 10:08:35 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-3f178da21afso64382075e9.1 for ; Fri, 28 Apr 2023 01:08:35 -0700 (PDT) Received: from georged-x-u.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 13-20020a05600c230d00b003f31da39b62sm2569464wmo.18.2023.04.28.01.08.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 01:08:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: df9af7b3-e59b-11ed-b224-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1682669314; x=1685261314; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ruSDN7fkH/OcFoZofz/uE6f8wHRubeKwBT3psKG1zkY=; b=Bbk8MHYhSTUiQuilXeTQUUn2+xE9TuMCk0qc3kF6rZaqQAM8riYJ0NuiVz5s8GR3UJ 1/DzStwJdpKKBPUMDubq4p/j6PdT90q4ac4ZeFbI7o3mdZYlV3igYQf/5PDMIQQPHVrO TLw+NXBZ9z6ysOr9wHa0ndsrIeb7ACZl2qjaw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682669314; x=1685261314; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ruSDN7fkH/OcFoZofz/uE6f8wHRubeKwBT3psKG1zkY=; b=krgU1RyyxB0FiGhRXpc06/0ghNhfqeHBG7eK2HHMmWo8iHcipuzJwHyuwXKW6ZliN3 WpLQQ6DZ54nh44QvuLW+nvQczWUq6hQoQIiTe47TPuA+jA0eKeBSsFGMTgYgxkgN5ig5 uK/D+6pfpdQ/aoST4VXTkQXhC4SRbEh8ceRcwtcZoE4808/x10GTyu4WY4HT+MQC/KuU iB0A8HNOKwTNuAd3uqJNHwob0JjL8z0FJ+ZmINrxikJStgNeW0dqPYx+hp4QYvWJ6gBg Xsio1SrE4fixiHZYcSrNTzO7GvW0bkJ5EHKz57hn1bJPnPNyNxiIwmOH9gciNqH05IcQ RHiw== X-Gm-Message-State: AC+VfDz1pWFn+t/7B0TRoy3TQSrG/NaMDjrViUeOmHu6F+6OW+mWrqCx BJ8v1e1la0lVzddjHJ0u3L0nYwtvmxpesL+0Y/U= X-Google-Smtp-Source: ACHHUZ662rNa/wbV8EG5ieb17gNhTuOnlw4IBT7Q1xgrvUjLcH2hUUYaFfSxqNOVQlypCm8sE4yC+w== X-Received: by 2002:a7b:cb94:0:b0:3f1:7ba6:d5ab with SMTP id m20-20020a7bcb94000000b003f17ba6d5abmr3392478wmi.36.1682669314569; Fri, 28 Apr 2023 01:08:34 -0700 (PDT) From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap , Wei Liu , Anthony PERARD Subject: [PATCH 2/5] xenalyze: Basic TRC_HVM_EMUL handling Date: Fri, 28 Apr 2023 09:08:29 +0100 Message-Id: <20230428080832.2461044-2-george.dunlap@cloud.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230428080832.2461044-1-george.dunlap@cloud.com> References: <20230428080832.2461044-1-george.dunlap@cloud.com> MIME-Version: 1.0 For now, mainly just do volume analysis and get rid of the warnings. Signed-off-by: George Dunlap --- CC: Wei Liu CC: Anthony PERARD --- tools/xentrace/xenalyze.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c index ff9716cd12..f7f8943079 100644 --- a/tools/xentrace/xenalyze.c +++ b/tools/xentrace/xenalyze.c @@ -987,6 +987,7 @@ enum { HVM_VOL_VMENTRY, HVM_VOL_VMEXIT, HVM_VOL_HANDLER, + HVM_VOL_EMUL, HVM_VOL_MAX }; @@ -1013,6 +1014,7 @@ const char *hvm_vol_name[HVM_VOL_MAX] = { [HVM_VOL_VMENTRY]="vmentry", [HVM_VOL_VMEXIT] ="vmexit", [HVM_VOL_HANDLER]="handler", + [HVM_VOL_EMUL]="emul", }; enum { @@ -5275,15 +5277,18 @@ void hvm_process(struct pcpu_info *p) if(vcpu_set_data_type(p->current, VCPU_DATA_HVM)) return; - if(ri->evt.sub == 2) - { + switch ( ri->evt.sub ) { + case 2: /* HVM_HANDLER */ UPDATE_VOLUME(p, hvm[HVM_VOL_HANDLER], ri->size); hvm_handler_process(ri, h); - } - else - { + break; + case 4: /* HVM_EMUL */ + UPDATE_VOLUME(p, hvm[HVM_VOL_EMUL], ri->size); + warn_once("WARNING: We don't yet analyze HVM_EMUL events.\n"); + /* FIXME: Collect analysis on this */ + break; + default: switch(ri->event) { - /* HVM */ case TRC_HVM_VMEXIT: case TRC_HVM_VMEXIT64: UPDATE_VOLUME(p, hvm[HVM_VOL_VMEXIT], ri->size); From patchwork Fri Apr 28 08:08:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 13226053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1AE0C7EE21 for ; Fri, 28 Apr 2023 08:09:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.527186.819495 (Exim 4.92) (envelope-from ) id 1psJ9i-0003MB-Iy; Fri, 28 Apr 2023 08:08:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 527186.819495; Fri, 28 Apr 2023 08:08:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9i-0003M4-Fx; Fri, 28 Apr 2023 08:08:38 +0000 Received: by outflank-mailman (input) for mailman id 527186; Fri, 28 Apr 2023 08:08:37 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9h-00036x-5v for xen-devel@lists.xenproject.org; Fri, 28 Apr 2023 08:08:37 +0000 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [2a00:1450:4864:20::32a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id dfe1bf24-e59b-11ed-b224-6b7b168915f2; Fri, 28 Apr 2023 10:08:35 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-3f178da21b5so65422235e9.3 for ; Fri, 28 Apr 2023 01:08:35 -0700 (PDT) Received: from georged-x-u.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 13-20020a05600c230d00b003f31da39b62sm2569464wmo.18.2023.04.28.01.08.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 01:08:34 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: dfe1bf24-e59b-11ed-b224-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1682669315; x=1685261315; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lcniXQbDbWeLRllJP53Ow0hRDGiOunRnJjI6sJOsHmU=; b=Tt9NLBoplSuQ4mo/EcdFjUCOO1/5GW1htY6ErRTJzM2I/E6v1i8WjeGghS5NQg54Bj DBIghLd/qZhORWId+mbFQiJ/xZGlY8KDRHPEcXyNAt7k7iPgktlPH44L9Ojzec1olVvt Sh1DQnmN0xpc+CsIbW6/Augr8nNKpdanhqrkA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682669315; x=1685261315; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lcniXQbDbWeLRllJP53Ow0hRDGiOunRnJjI6sJOsHmU=; b=LU+j8ITf3kWVksi/Wv+VhnwTDAoWz5ePB12NBklEPXYFn0Va1CctWowizA5j/TFacH QRjiECnV1SRrBFwAybMAkli2Q04FfIclmKm0VJDV2nCvZ050M1hpLSqtrbd57icJTQH1 frHY5eN/JJKUV6BQasK+Rgh4xWDgXnm9LTsVk0R8XoWnGZoB4USuy4j3PBqva8RLj6vX pDB5dNjHp74ZkDPyeS3ThC+1H7x1gK1uWwxbTJVB07eRuGLDYCmdQyQFgsttoRdLvhax jGwy3/+UP+Mvw5UVYykHn+GGlqqXQavj1VmowSd2qQPFc5De1mcL6M8mp1VPRgFyuej1 l1gw== X-Gm-Message-State: AC+VfDzJRcCy9ckXMBBj+39t+4fDzbuhfOzFRJ1ajHxVvBa2Zpc8Og4Y SXdJZd/+Pj4sWrVkNS5b5LrEq7VtF1RSeHZ7IAU= X-Google-Smtp-Source: ACHHUZ6S2ykCEmxqd6YKHBi4EZgGUrRFvLs9W6AhzdzaFW9dxCuWknEEQ3+DRECWsIh8rXyXIB6rYw== X-Received: by 2002:a7b:ce07:0:b0:3f1:e5f2:5e86 with SMTP id m7-20020a7bce07000000b003f1e5f25e86mr3185741wmc.23.1682669315034; Fri, 28 Apr 2023 01:08:35 -0700 (PDT) From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap Subject: [PATCH 3/5] credit: Limit load balancing to once per millisecond Date: Fri, 28 Apr 2023 09:08:30 +0100 Message-Id: <20230428080832.2461044-3-george.dunlap@cloud.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230428080832.2461044-1-george.dunlap@cloud.com> References: <20230428080832.2461044-1-george.dunlap@cloud.com> MIME-Version: 1.0 The credit scheduler tries as hard as it can to ensure that it always runs scheduling units with positive credit (PRI_TS_UNDER) before running those with negative credit (PRI_TS_OVER). If the next runnable scheduling unit is of priority OVER, it will always run the load balancer, which will scour the system looking for another scheduling unit of the UNDER priority. Unfortunately, as the number of cores on a system has grown, the cost of the work-stealing algorithm has dramatically increased; a recent trace on a system with 128 cores showed this taking over 50 microseconds. Add a parameter, load_balance_ratelimit, to limit the frequency of load balance operations on a given pcpu. Default this to 1 millisecond. Invert the load balancing conditional to make it more clear, and line up more closely with the comment above it. Overall it might be cleaner to have the last_load_balance checking happen inside csched_load_balance(), but that would require either passing both now and spc into the function, or looking them up again; both of which seemed to be worse than simply checking and setting the values before calling it. Signed-off-by: George Dunlap --- docs/misc/xen-command-line.pandoc | 6 +++++ xen/common/sched/credit.c | 40 ++++++++++++++++++++++++++----- xen/include/public/sysctl.h | 6 +++++ 3 files changed, 46 insertions(+), 6 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index e0b89b7d33..ae51a8cfa2 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1840,6 +1840,12 @@ By default, Xen will use the INVPCID instruction for TLB management if it is available. This option can be used to cause Xen to fall back to older mechanisms, which are generally slower. +### load-balance-ratelimit +> `= ` + +The minimum interval between load balancing events on a given pcpu. +At the moment only credit honors this parameter. + ### noirqbalance (x86) > `= ` diff --git a/xen/common/sched/credit.c b/xen/common/sched/credit.c index f2cd3d9da3..b8bdfd5f6a 100644 --- a/xen/common/sched/credit.c +++ b/xen/common/sched/credit.c @@ -50,6 +50,8 @@ #define CSCHED_TICKS_PER_TSLICE 3 /* Default timeslice: 30ms */ #define CSCHED_DEFAULT_TSLICE_MS 30 +/* Default load balancing ratelimit: 1ms */ +#define CSCHED_DEFAULT_LOAD_BALANCE_RATELIMIT_US 1000 #define CSCHED_CREDITS_PER_MSEC 10 /* Never set a timer shorter than this value. */ #define CSCHED_MIN_TIMER XEN_SYSCTL_SCHED_RATELIMIT_MIN @@ -153,6 +155,7 @@ struct csched_pcpu { unsigned int idle_bias; unsigned int nr_runnable; + s_time_t last_load_balance; unsigned int tick; struct timer ticker; @@ -218,7 +221,7 @@ struct csched_private { /* Period of master and tick in milliseconds */ unsigned int tick_period_us, ticks_per_tslice; - s_time_t ratelimit, tslice, unit_migr_delay; + s_time_t ratelimit, tslice, unit_migr_delay, load_balance_ratelimit; struct list_head active_sdom; uint32_t weight; @@ -612,6 +615,8 @@ init_pdata(struct csched_private *prv, struct csched_pcpu *spc, int cpu) BUG_ON(!is_idle_unit(curr_on_cpu(cpu))); cpumask_set_cpu(cpu, prv->idlers); spc->nr_runnable = 0; + + spc->last_load_balance = NOW(); } static void cf_check @@ -1267,7 +1272,8 @@ csched_sys_cntl(const struct scheduler *ops, && (params->ratelimit_us > XEN_SYSCTL_SCHED_RATELIMIT_MAX || params->ratelimit_us < XEN_SYSCTL_SCHED_RATELIMIT_MIN)) || MICROSECS(params->ratelimit_us) > MILLISECS(params->tslice_ms) - || params->vcpu_migr_delay_us > XEN_SYSCTL_CSCHED_MGR_DLY_MAX_US ) + || params->vcpu_migr_delay_us > XEN_SYSCTL_CSCHED_MGR_DLY_MAX_US + || params->load_balance_ratelimit_us > XEN_SYSCTL_CSCHED_LB_RATE_MAX_US) goto out; spin_lock_irqsave(&prv->lock, flags); @@ -1278,6 +1284,7 @@ csched_sys_cntl(const struct scheduler *ops, printk(XENLOG_INFO "Disabling context switch rate limiting\n"); prv->ratelimit = MICROSECS(params->ratelimit_us); prv->unit_migr_delay = MICROSECS(params->vcpu_migr_delay_us); + prv->load_balance_ratelimit = MICROSECS(params->load_balance_ratelimit_us); spin_unlock_irqrestore(&prv->lock, flags); /* FALLTHRU */ @@ -1285,6 +1292,7 @@ csched_sys_cntl(const struct scheduler *ops, params->tslice_ms = prv->tslice / MILLISECS(1); params->ratelimit_us = prv->ratelimit / MICROSECS(1); params->vcpu_migr_delay_us = prv->unit_migr_delay / MICROSECS(1); + params->load_balance_ratelimit_us = prv->load_balance_ratelimit / MICROSECS(1); rc = 0; break; } @@ -1676,9 +1684,17 @@ csched_runq_steal(int peer_cpu, int cpu, int pri, int balance_step) return NULL; } +/* + * Minimum delay, in microseconds, between load balance operations. + * This prevents spending too much time doing load balancing, particularly + * when the system has a high number of YIELDs due to spinlock priority inversion. + */ +static unsigned int __read_mostly load_balance_ratelimit_us = CSCHED_DEFAULT_LOAD_BALANCE_RATELIMIT_US; +integer_param("load-balance-ratelimit", load_balance_ratelimit_us); + static struct csched_unit * csched_load_balance(struct csched_private *prv, int cpu, - struct csched_unit *snext, bool *stolen) + struct csched_unit *snext, bool *stolen) { const struct cpupool *c = get_sched_res(cpu)->cpupool; struct csched_unit *speer; @@ -1963,10 +1979,12 @@ static void cf_check csched_schedule( * urgent work... If not, csched_load_balance() will return snext, but * already removed from the runq. */ - if ( snext->pri > CSCHED_PRI_TS_OVER ) - __runq_remove(snext); - else + if ( snext->pri <= CSCHED_PRI_TS_OVER + && now - spc->last_load_balance > prv->load_balance_ratelimit) { + spc->last_load_balance = now; snext = csched_load_balance(prv, sched_cpu, snext, &migrated); + } else + __runq_remove(snext); } while ( !unit_runnable_state(snext->unit) ); @@ -2181,6 +2199,14 @@ csched_global_init(void) XEN_SYSCTL_CSCHED_MGR_DLY_MAX_US, vcpu_migration_delay_us); } + if ( load_balance_ratelimit_us > XEN_SYSCTL_CSCHED_LB_RATE_MAX_US ) + { + load_balance_ratelimit_us = CSCHED_DEFAULT_LOAD_BALANCE_RATELIMIT_US; + printk("WARNING: load-balance-ratelimit outside of valid range [0,%d]us.\n" + "Resetting to default: %u\n", + XEN_SYSCTL_CSCHED_LB_RATE_MAX_US, load_balance_ratelimit_us); + } + return 0; } @@ -2223,6 +2249,8 @@ csched_init(struct scheduler *ops) prv->unit_migr_delay = MICROSECS(vcpu_migration_delay_us); + prv->load_balance_ratelimit = MICROSECS(load_balance_ratelimit_us); + return 0; } diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index 2b24d6bfd0..192458d635 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -637,6 +637,12 @@ struct xen_sysctl_credit_schedule { */ #define XEN_SYSCTL_CSCHED_MGR_DLY_MAX_US (100 * 1000) uint32_t vcpu_migr_delay_us; + /* + * Minimum delay, in microseconds, between load balance + * operations; max 1 second. + */ +#define XEN_SYSCTL_CSCHED_LB_RATE_MAX_US (1000000) + uint32_t load_balance_ratelimit_us; }; struct xen_sysctl_credit2_schedule { From patchwork Fri Apr 28 08:08:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 13226051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6D3A7C77B60 for ; Fri, 28 Apr 2023 08:09:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.527187.819505 (Exim 4.92) (envelope-from ) id 1psJ9j-0003bB-QH; Fri, 28 Apr 2023 08:08:39 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 527187.819505; Fri, 28 Apr 2023 08:08:39 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9j-0003b4-N8; Fri, 28 Apr 2023 08:08:39 +0000 Received: by outflank-mailman (input) for mailman id 527187; Fri, 28 Apr 2023 08:08:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9i-00036x-6D for xen-devel@lists.xenproject.org; Fri, 28 Apr 2023 08:08:38 +0000 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [2a00:1450:4864:20::32e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id e0175b10-e59b-11ed-b224-6b7b168915f2; Fri, 28 Apr 2023 10:08:36 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-3f09b4a156eso64347975e9.3 for ; Fri, 28 Apr 2023 01:08:36 -0700 (PDT) Received: from georged-x-u.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 13-20020a05600c230d00b003f31da39b62sm2569464wmo.18.2023.04.28.01.08.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 01:08:35 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e0175b10-e59b-11ed-b224-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1682669315; x=1685261315; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lLu7TscClVzVFwnxnkUi2eFFYfLdqDnUzBYuO4yEO4M=; b=MS6ycKFAY4yrzKVadA9O0uoVCowNIXOMxRBSbz1Xada8oMpsZ9G5KTsHr1wfuqoZBk VeJ3LEyktkgctiUxiKfqCBQhHyqBkQRYChGUwCfe+priBB2XqeKFv3TyNDAnBHwr6kjR 8fPcrPPxt/z4WgFuY2aMsdfgV0BujO+FwHqNw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682669315; x=1685261315; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lLu7TscClVzVFwnxnkUi2eFFYfLdqDnUzBYuO4yEO4M=; b=OvL51uncN0b8WBa5bHf87Lz0mySOgUB/2uNv/MgO4iE8HZemC7Tcyd/p5AsP/wDhfi iGxMqmaoC6DVwSAZNk/HVlgugR5Cb3anpDKljR3PQpj92TI/9R47Vr5LMF62aUZwDLTX OPp6JN0y3uO7d+GqmMTQfYAY2OMV841VwnF5mg8co7/OBK3VJeZuDurS7JNueVRiZ3mU Uh2lFf2k/C22C0Ge492dc5zPbtaFsT9ThvpiAYnbOcdiBWHyweFSSIk1skCqa5bojDgF JVx/HIbk4AYOfFgr3uc1TpM17vuT6T23i1zGGY7ZfVv6QwkQSWR1Iq/3aqbQqTtUjJaZ yA6g== X-Gm-Message-State: AC+VfDwpDAAvUucWzdCXCxBalF0guqThYvJqQNBKaCuiNfj8JriSWW8L zBvF/e2tkMXTyDGL21OwKkjGi7ogEKI4eC3cAR8= X-Google-Smtp-Source: ACHHUZ5awrW5ndtj6h3K+RiEO1oWn0kIUfGYxK1DkO+FZrvVuXWn53EffJsqWvILbOxVSWqykPhuhA== X-Received: by 2002:a7b:cb94:0:b0:3f1:7ba6:d5ab with SMTP id m20-20020a7bcb94000000b003f17ba6d5abmr3392523wmi.36.1682669315504; Fri, 28 Apr 2023 01:08:35 -0700 (PDT) From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap Subject: [PATCH 4/5] credit: Don't steal vcpus which have yielded Date: Fri, 28 Apr 2023 09:08:31 +0100 Message-Id: <20230428080832.2461044-4-george.dunlap@cloud.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230428080832.2461044-1-george.dunlap@cloud.com> References: <20230428080832.2461044-1-george.dunlap@cloud.com> MIME-Version: 1.0 On large systems with many vcpus yielding due to spinlock priority inversion, it's not uncommon for a vcpu to yield its timeslice, only to be immediately stolen by another pcpu looking for higher-priority work. To prevent this: * Keep the YIELD flag until a vcpu is removed from a runqueue * When looking for work to steal, skip vcpus which have yielded Signed-off-by: George Dunlap --- xen/common/sched/credit.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/xen/common/sched/credit.c b/xen/common/sched/credit.c index b8bdfd5f6a..70a1a57ba6 100644 --- a/xen/common/sched/credit.c +++ b/xen/common/sched/credit.c @@ -319,6 +319,11 @@ __runq_remove(struct csched_unit *svc) { BUG_ON( !__unit_on_runq(svc) ); list_del_init(&svc->runq_elem); + + /* + * Clear YIELD flag when scheduling back in + */ + clear_bit(CSCHED_FLAG_UNIT_YIELD, &svc->flags); } static inline void @@ -1638,6 +1643,13 @@ csched_runq_steal(int peer_cpu, int cpu, int pri, int balance_step) if ( speer->pri <= pri ) break; + /* + * Don't steal a UNIT which has yielded; it's waiting for a + * reason + */ + if (test_bit(CSCHED_FLAG_UNIT_YIELD, &speer->flags)) + continue; + /* Is this UNIT runnable on our PCPU? */ unit = speer->unit; BUG_ON( is_idle_unit(unit) ); @@ -1955,11 +1967,6 @@ static void cf_check csched_schedule( dec_nr_runnable(sched_cpu); } - /* - * Clear YIELD flag before scheduling out - */ - clear_bit(CSCHED_FLAG_UNIT_YIELD, &scurr->flags); - do { snext = __runq_elem(runq->next); @@ -1974,10 +1981,11 @@ static void cf_check csched_schedule( /* * SMP Load balance: * - * If the next highest priority local runnable UNIT has already eaten - * through its credits, look on other PCPUs to see if we have more - * urgent work... If not, csched_load_balance() will return snext, but - * already removed from the runq. + * If the next highest priority local runnable UNIT has + * already eaten through its credits (and we're below the + * balancing ratelimit), look on other PCPUs to see if we have + * more urgent work... If we don't, csched_load_balance() will + * return snext, but already removed from the runq. */ if ( snext->pri <= CSCHED_PRI_TS_OVER && now - spc->last_load_balance > prv->load_balance_ratelimit) { From patchwork Fri Apr 28 08:08:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 13226049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D044CC77B60 for ; Fri, 28 Apr 2023 08:09:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.527189.819516 (Exim 4.92) (envelope-from ) id 1psJ9k-0003pR-NQ; Fri, 28 Apr 2023 08:08:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 527189.819516; Fri, 28 Apr 2023 08:08:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9k-0003nA-Jh; Fri, 28 Apr 2023 08:08:40 +0000 Received: by outflank-mailman (input) for mailman id 527189; Fri, 28 Apr 2023 08:08:39 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1psJ9j-0003M3-D7 for xen-devel@lists.xenproject.org; Fri, 28 Apr 2023 08:08:39 +0000 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [2a00:1450:4864:20::32a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id e0a4c247-e59b-11ed-8611-37d641c3527e; Fri, 28 Apr 2023 10:08:37 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-3f1e2555b5aso44959705e9.0 for ; Fri, 28 Apr 2023 01:08:37 -0700 (PDT) Received: from georged-x-u.eng.citrite.net ([185.25.67.249]) by smtp.gmail.com with ESMTPSA id 13-20020a05600c230d00b003f31da39b62sm2569464wmo.18.2023.04.28.01.08.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 01:08:35 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e0a4c247-e59b-11ed-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1682669316; x=1685261316; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ull2YxWBFQl0HliuvqOLTfd5qwQa1W30wkLZFscPDpw=; b=c+VOSAyMBxEBJRiQ8XbRHkxqCgy0oM1qlBzbUlrqhy5sOrgE/OmyiX6kEah4qp9Mv1 0PH0dOE65KsGrnQ4cqQe7QqsgNJuJrDbY5ZtDA3X7mWDTuBb3ADkldtl8Pr82RMs1+QK xDvNRBvKGqm1IWFmyCTvrlH4OwlKt15nv/Woc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682669316; x=1685261316; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ull2YxWBFQl0HliuvqOLTfd5qwQa1W30wkLZFscPDpw=; b=GX67mYm0XihBfzix7fo1d/rZ2zdad+QYsmvs8dhqqV+N0MAJlQg6pRnvE0yCDH4Z78 Rzc+Kk1wgyFN4ayEv1YXowbOQnxAUzulI3f1osMNfLTZhnfV7ZvlPuE4kdJT1F6SqmkE s5Hzh5JeLWN/vXg3B6R2ZDdWhsGYJbqfyj6fRuh9CIdMFohkQ8URsxtmZJANjuqu++tA P0g+UHZbXlMM1LmMUj4WuxBJzw6v+Q4kOsdQfzi11KgtTmlvtvjOE9+WvI3US9Kk5LzG sSVIzDK7jcR+vrPkfkRQZZU9a3SH2UxvyF0MFVeJntsegJxZk7icVnYWKoX/cVCiUgJ2 loTA== X-Gm-Message-State: AC+VfDxHf7YNzgJW7RfiSiQKQarJiwRGOc6xRNZ2lGqtC3qkftfEqg/f 1WfmXCFPRp0mRun8B5rwAbVFttFtllTc4MzfNq8= X-Google-Smtp-Source: ACHHUZ4sKhcZrmWQOmeyk+jAWUc6aoWKQoq2pZkkD16DksbVl/Kln3xeeBjNzDXSKv8Zi17uTg+tkw== X-Received: by 2002:a05:600c:220d:b0:3f1:9526:22be with SMTP id z13-20020a05600c220d00b003f1952622bemr3282326wml.23.1682669316259; Fri, 28 Apr 2023 01:08:36 -0700 (PDT) From: George Dunlap To: xen-devel@lists.xenproject.org Cc: George Dunlap , Wei Liu , Andrew Cooper , Jan Beulich , Roger Pau Monne , Stefano Stabellini , Julien Grall Subject: [PATCH 5/5] SUPPORT.md: Make all security support explicit Date: Fri, 28 Apr 2023 09:08:32 +0100 Message-Id: <20230428080832.2461044-5-george.dunlap@cloud.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230428080832.2461044-1-george.dunlap@cloud.com> References: <20230428080832.2461044-1-george.dunlap@cloud.com> MIME-Version: 1.0 The initial goal of SUPPORT.md was to help both users, and the Xen Project Security Team, determine what functionality was security supported; i.e., what kinds of security bugs would trigger an XSA. Our proposal is that as of 4.18, all functionality not explicitly listed as security supported will be considered not security supported. Add some text to that effect. The patch as written cannot be applied, since specifying "xl.cfg core functionality" is a TODO; but it should do to start a discussion. Signed-off-by: Georg Dunlap --- CC: Wei Liu CC: Andrew Cooper CC: Jan Beulich CC: Roger Pau Monne CC: Stefano Stabellini CC: Julien Grall --- SUPPORT.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/SUPPORT.md b/SUPPORT.md index aa1940e55f..fcbcb44c44 100644 --- a/SUPPORT.md +++ b/SUPPORT.md @@ -17,6 +17,36 @@ for the definitions of the support status levels etc. Release Notes : RN +# General security support + +An XSA will always be issued for security-related bugs which are +present in a "plain vanilla" configuration. A "plain vanilla" +configuration is defined as follows: + +* The Xen hypervisor is built from a tagged release of Xen, or a + commit which was on the tip of one of the supported stable branches. + +* The Xen hypervisor was built with the default config for the platform + +* No Xen command-line parameters were specified + +* No parameters for Xen-related drivers in the Linux kernel were specified + +* No modifications were made to the default xl.conf + +* xl.cfg files use only core functionality + +* Alternate toolstacks only activate functionality activated by the + core functionality of xl.cfg files. + +Any system outside this configuration will only be considered security +supported if the functionality is explicitly listed as supported in +this document. + +If a security-related bug exits only in a configuration listed as not +security supported, the security team will generally not issue an XSA; +the bug will simply be handled in public. + # Feature Support ## Kconfig