From patchwork Thu Jan 19 21:34:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dionna Amalie Glaze X-Patchwork-Id: 13108850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CAD3C6379F for ; Thu, 19 Jan 2023 21:53:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230332AbjASVxs (ORCPT ); Thu, 19 Jan 2023 16:53:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230391AbjASVxI (ORCPT ); Thu, 19 Jan 2023 16:53:08 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C536AEDB3 for ; Thu, 19 Jan 2023 13:34:32 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id f132-20020a636a8a000000b00473d0b600ebso1587404pgc.14 for ; Thu, 19 Jan 2023 13:34:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LXeEK1Kbvuk2JPbM8yD6qcpUOvIqc9ntM80OCtFV2Pc=; b=tEnmvnxM4IUmcMY9AKBYh9I/Rb/pEUtEZLgiNEf9uIgLM6z7B0M3Ah9ot6TwzUuIr2 IB88l1wTD/fGBUWpxzoxMrnUq0Z+euhdRafpokJhocLPNGvD8ZS5i0pJsQzUh9Nm767/ 2JcHdDTv3QxeKaBorqwNJrpHXz4Xq6HIbVTLhTn5UbUtr3vFz2vWAQHv/45NPj9kPVJv x/bFoLggsTCaG1VuKdEQVw8DBhUVAghU2vxJqj0Nb2ZcARsToZMoeUY8trThDvkHpkHp 0VS2+QpFK6YTVWz338kYVSUUte8vAo2wBreh98p1mmsmRZ4z6RPuZOBjhenTXU6xr6IU h2GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LXeEK1Kbvuk2JPbM8yD6qcpUOvIqc9ntM80OCtFV2Pc=; b=XSCoiXEZS2n3lNVQS9ZeYSOXu4PP0dshA95rerx1wZcjri8K5lYhKuQ+PbaLKx7IeH I77aQfsRh6viQPlNglB9Cr5xmV4APTJUSjB9VsN8rptG8xwOgESipe86ZLrOlGNHwGWO 05xrcmIdW6/giiZFvO2zgsHV7/9MwZg+oexIhIVBeWdBtQrI7mn6TFqoOml+qdpV2ZNm imSgZrrTGKdKh8MYmkgn2PVmon8GfG3FVmz9G3hzn7J8rp/NcwEvEzehR9O+9Q1RszkI k8l3pE9VmlZl7gPiDzsU8FJg12PrGfNwrlsua3qRJZRMwUpiZBhB60PyCbMUzl4Kn7Qi bIEQ== X-Gm-Message-State: AFqh2kqwFFV3tr+QOj+J9q39J1Qpha7QOCp2xWhCcgl4BagBpx8Hqv7l oy73JkyMk23b3Zjb9emJ8fNP+3jpsNm/Ei7dQ48vsRHL3URNG6KcvC1FnLsWFtX0zVrV5mY+4HL JQs3Y0p6ZL37IUnBjmiGmoeXvTC7yVdZATtbVLsD+6JGpsI7/lNWC8dk7IwkxaSjPhbSe6x8= X-Google-Smtp-Source: AMrXdXtKk5XbMAdLQ8flQVxvy7BwTRsUbrJFrpNJeEnd82CJpiPjGzKPwBaraCWcPBUq3FvyH037KosXyJOkWQePsg== X-Received: from dionnaglaze.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2ee6]) (user=dionnaglaze job=sendgmr) by 2002:aa7:9727:0:b0:58c:1d0d:f1a6 with SMTP id k7-20020aa79727000000b0058c1d0df1a6mr1268749pfg.2.1674164070881; Thu, 19 Jan 2023 13:34:30 -0800 (PST) Date: Thu, 19 Jan 2023 21:34:24 +0000 In-Reply-To: <20230119213426.379312-1-dionnaglaze@google.com> Mime-Version: 1.0 References: <20230119213426.379312-1-dionnaglaze@google.com> X-Mailer: git-send-email 2.39.0.246.g2a6d74b583-goog Message-ID: <20230119213426.379312-2-dionnaglaze@google.com> Subject: [PATCH v3 1/2] kvm: sev: Add SEV-SNP guest request throttling From: Dionna Glaze To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Dionna Glaze , Thomas Lendacky , Paolo Bonzini , Joerg Roedel , Peter Gonda , Borislav Petkov Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The AMD-SP is a precious resource that doesn't have a scheduler other than a mutex lock queue. To avoid customers from causing a DoS, a module_param-set rate limit is added with a default of 2 requests per 2 seconds. These defaults were chosen empirically with a the assumption that current server-grade SEV-SNP machines will rarely exceed 128 VMs under usual circumstance. The 2 burst per 2 seconds means on average 1 request every second. We allow 2 requests back to back to allow for the guest to query the certificate length in an extended guest request without a pause. The 1 second average is our target for quality of service since empirical tests show that 64 VMs can concurrently request an attestation report with a maximum latency of 1 second. We don't anticipate more concurrency than that for a seldom used request for a majority well- behaved set of VMs. The majority point is decided as >64 VMs given the assumed 128 VM count for "extreme load". The throttling code is 2 << 32 given that invalid length is 1 and 2 is the next available code. This was suggested by Tom Lendacky, and will be included in a new revision of the GHCB specification. Cc: Thomas Lendacky Cc: Paolo Bonzini Cc: Joerg Roedel Cc: Peter Gonda Cc: Borislav Petkov Signed-off-by: Dionna Glaze --- arch/x86/include/asm/sev-common.h | 1 + arch/x86/kvm/svm/sev.c | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.h | 3 +++ include/uapi/linux/in.h | 1 + 4 files changed, 34 insertions(+) diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h index 1b111cde8c82..e3a6b039480d 100644 --- a/arch/x86/include/asm/sev-common.h +++ b/arch/x86/include/asm/sev-common.h @@ -158,6 +158,7 @@ struct snp_psc_desc { /* Guest message request error code */ #define SNP_GUEST_REQ_INVALID_LEN BIT_ULL(32) +#define SNP_GUEST_REQ_THROTTLED (((u64)2) << 32) #define GHCB_MSR_TERM_REQ 0x100 #define GHCB_MSR_TERM_REASON_SET_POS 12 diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index d0e58cffd1ed..cd9372ce6fc2 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -58,6 +58,14 @@ module_param_named(sev_es, sev_es_enabled, bool, 0444); /* enable/disable SEV-SNP support */ static bool sev_snp_enabled = true; module_param_named(sev_snp, sev_snp_enabled, bool, 0444); + +/* Throttle guest requests to a burst # per this many seconds */ +unsigned int guest_request_throttle_s = 2; +module_param(guest_request_throttle_s, int, 0444); + +/* Throttle guest requests to this many per the above many seconds */ +unsigned int guest_request_throttle_burst = 2; +module_param(guest_request_throttle_burst, int, 0444); #else #define sev_enabled false #define sev_es_enabled false @@ -333,6 +341,9 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp) goto e_free; mutex_init(&sev->guest_req_lock); + ratelimit_state_init(&sev->snp_guest_msg_rs, + guest_request_throttle_s * HZ, + guest_request_throttle_burst); ret = sev_snp_init(&argp->error, false); } else { ret = sev_platform_init(&argp->error); @@ -3595,6 +3606,14 @@ static void snp_cleanup_guest_buf(struct sev_data_snp_guest_request *data, unsig *rc = SEV_RET_INVALID_ADDRESS; } +static bool snp_throttle_guest_request(struct kvm_sev_info *sev) { + if (__ratelimit(&sev->snp_guest_msg_rs)) + return false; + + pr_info_ratelimited("svm: too many guest message requests\n"); + return true; +} + static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa) { struct sev_data_snp_guest_request data = {0}; @@ -3611,6 +3630,11 @@ static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t sev = &to_kvm_svm(kvm)->sev_info; + if (snp_throttle_guest_request(sev)) { + rc = SNP_GUEST_REQ_THROTTLED; + goto e_fail; + } + mutex_lock(&sev->guest_req_lock); rc = snp_setup_guest_buf(svm, &data, req_gpa, resp_gpa); @@ -3648,6 +3672,11 @@ static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gp sev = &to_kvm_svm(kvm)->sev_info; + if (snp_throttle_guest_request(sev)) { + rc = SNP_GUEST_REQ_THROTTLED; + goto e_fail; + } + data_gpa = vcpu->arch.regs[VCPU_REGS_RAX]; data_npages = vcpu->arch.regs[VCPU_REGS_RBX]; diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 8d1ba66860a4..7048f817efb0 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -105,6 +106,8 @@ struct kvm_sev_info { unsigned int snp_certs_len; /* Size of instance override for certs */ struct mutex guest_req_lock; + struct ratelimit_state snp_guest_msg_rs; /* Limit guest requests */ + u64 sev_features; /* Features set at VMSA creation */ }; diff --git a/include/uapi/linux/in.h b/include/uapi/linux/in.h index f243ce665f74..07a4cb149305 100644 --- a/include/uapi/linux/in.h +++ b/include/uapi/linux/in.h @@ -20,6 +20,7 @@ #define _UAPI_LINUX_IN_H #include +#include #include #include From patchwork Thu Jan 19 21:34:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dionna Amalie Glaze X-Patchwork-Id: 13108851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E684C004D4 for ; Thu, 19 Jan 2023 21:53:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231138AbjASVxv (ORCPT ); Thu, 19 Jan 2023 16:53:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230463AbjASVxN (ORCPT ); Thu, 19 Jan 2023 16:53:13 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6ACAAB1EC9 for ; Thu, 19 Jan 2023 13:34:34 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id x8-20020a17090a8a8800b0022941842cc0so3954575pjn.3 for ; Thu, 19 Jan 2023 13:34:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XP5abnKwoWHYzfvPZhByiAlFcIOsPkv0onTEpyjGkOA=; b=i4TDJZV9AsYZggXWT49Rh8u+Cqa1ja4Ok/tTT96quI+KK+vj/fKuizWOfYp3FBC28G RgcXCk0MTknMVs0eGoHSVbG5x5Jh2QrCBOtiIAM/pZitOj/QH/+EbKRD1hX/LGfTSXEm XdsGBNAQ1ve/IjyYAwCzZSBEpL1hNp7b8UEhbvKQjIAJiBG/QuhbRtiR7kReLh1wR9un 31CWO+3vnlDRNAe5wq9dDA9jC58lWSjR586Au3LI6Iuwb43aSzMweU/Z0WY31q2ZeFME jKluDnzlI9UyLtChnv4pZFVa+xdS3hjuPDq84zkdKmAGMXE59wBBT0khj6vWbkskA18m i1sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XP5abnKwoWHYzfvPZhByiAlFcIOsPkv0onTEpyjGkOA=; b=Xs6xtNHfTBk/1KNG894pYoU0gzVq9g1280gIWsv1gHImXv+Zh8+3ErgbFAVwKcWxf1 x+TirQV3dmNtB0nsO9kH7lvAEvcnwsTAPU6HL24Hk4DBCbFyi7t62X1S1Z69c+NitnZW kMhEi0xU88xu+jz3llDtRFEekyaBObyGULBdJWzjFIVpaZTyIBaU3RGsZ+6WSmTIXbff mq4hmXsVtNg6vJiy+i5xqescI0c3CsRvr1CKWzXusZiC497Lb7Nn/g0FSL4KVdYtF+a3 Ee0wxB0AXYvxpXoo2BJ+xFAD5Nii+tz35gQXVCWp6yv4JwTFjOYaXBCxB7KEpiUhRtxY Hfyg== X-Gm-Message-State: AFqh2kpjee/g8ND5hHHh5m9d7p2PE9Jo5QZiMfSn1RZiUqjxMXh34GRf h2Hf/BGW7ryHn1To6W1LxHREp8lZDgCub6iSfkVEHiDR5mp12aG1N4sqaltIGvvJgOEtsdkMeqU UikEUnIGZDky4qML6COxhlG5S6a8EMq0ho8ZF06YpkIJPteL6lmPqrOt/5Fh/5U9dACzmXE0= X-Google-Smtp-Source: AMrXdXvNYervhBkZcdhUEFpRVywQkR7lK/Et6wxL6MXKVleX90gS/xRSqlXGxsj5OxYrd15e9POisxZ7yWd9E0mRUg== X-Received: from dionnaglaze.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2ee6]) (user=dionnaglaze job=sendgmr) by 2002:a05:6a00:4088:b0:586:7e0c:372d with SMTP id bw8-20020a056a00408800b005867e0c372dmr1215084pfb.14.1674164072489; Thu, 19 Jan 2023 13:34:32 -0800 (PST) Date: Thu, 19 Jan 2023 21:34:25 +0000 In-Reply-To: <20230119213426.379312-1-dionnaglaze@google.com> Mime-Version: 1.0 References: <20230119213426.379312-1-dionnaglaze@google.com> X-Mailer: git-send-email 2.39.0.246.g2a6d74b583-goog Message-ID: <20230119213426.379312-3-dionnaglaze@google.com> Subject: [PATCH v3 2/2] kvm: sev: If ccp is busy, report throttled to guest From: Dionna Glaze To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Dionna Glaze , Thomas Lendacky , Paolo Bonzini , Joerg Roedel , Peter Gonda , Borislav Petkov Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The ccp driver can be overloaded even with 1 HZ throttling. The return value of -EBUSY means that there is no firmware error to report back to user space, so the guest VM would see this as exitinfo2 = 0. The false success can trick the guest to update its the message sequence number when it shouldn't have. Instead, when ccp returns -EBUSY, that is reported to userspace as the throttling return value. Cc: Thomas Lendacky Cc: Paolo Bonzini Cc: Joerg Roedel Cc: Peter Gonda Cc: Borislav Petkov Signed-off-by: Dionna Glaze --- arch/x86/kvm/svm/sev.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index cd9372ce6fc2..7da1cc300d7b 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3642,7 +3642,14 @@ static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t goto unlock; rc = sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &err); - if (rc) + + /* + * The ccp driver can return -EBUSY if the PSP is overloaded, so signal + * the request has been throttled. + */ + if (rc == -EBUSY) + rc = SNP_GUEST_REQ_THROTTLED; + else if (rc) /* use the firmware error code */ rc = err; @@ -3713,7 +3720,14 @@ static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gp if (sev->snp_certs_len) data_npages = sev->snp_certs_len >> PAGE_SHIFT; - if (rc) { + /* + * The ccp driver can return -EBUSY if the PSP is overloaded, so signal + * the request has been throttled. + */ + if (rc == -EBUSY) { + rc = SNP_GUEST_REQ_THROTTLED; + goto cleanup; + } else if (rc) { /* * If buffer length is small then return the expected * length in rbx.