[v15,21/23] x86/virt/tdx: Handle TDX interaction with ACPI S3 and deeper states

Commit Message

Huang, Kai Nov. 9, 2023, 11:55 a.m. UTC

TDX cannot survive from S3 and deeper states.  The hardware resets and
disables TDX completely when platform goes to S3 and deeper.  Both TDX
guests and the TDX module get destroyed permanently.

The kernel uses S3 to support suspend-to-ram, and S4 or deeper states to
support hibernation.  The kernel also maintains TDX states to track
whether it has been initialized and its metadata resource, etc.  After
resuming from S3 or hibernation, these TDX states won't be correct
anymore.

Theoretically, the kernel can do more complicated things like resetting
TDX internal states and TDX module metadata before going to S3 or
deeper, and re-initialize TDX module after resuming, etc, but there is
no way to save/restore TDX guests for now.

Until TDX supports full save and restore of TDX guests, there is no big
value to handle TDX module in suspend and hibernation alone.  To make
things simple, just choose to make TDX mutually exclusive with S3 and
hibernation.

Note the TDX module is initialized at runtime.  To avoid having to deal
with the fuss of determining TDX state at runtime, just choose TDX vs S3
and hibernation at kernel early boot.  It's a bad user experience if the
choice of TDX and S3/hibernation is done at runtime anyway, i.e., the
user can experience being able to do S3/hibernation but later becoming
unable to due to TDX being enabled.

Disable TDX in kernel early boot when hibernation support is available.
Currently there's no mechanism exposed by the hibernation code to allow
other kernel code to disable hibernation once for all.

Disable ACPI S3 when TDX is enabled by the BIOS.  For now the user needs
to disable TDX in the BIOS to use ACPI S3.  A new kernel command line
can be added in the future if there's a need to let user disable TDX
host via kernel command line.

Alternatively, the kernel could disable TDX when ACPI S3 is supported
and request the user to disable S3 to use TDX.  But there's no existing
kernel command line to do that, and BIOS doesn't always have an option
to disable S3.

Signed-off-by: Kai Huang <[e-mail]>
Reviewed-by: Kirill A. Shutemov <[e-mail]>
---

v14 -> v15:
 - Simplify the error message when hibernation_available() returns true
   by removing "Use 'nohibernate' kernel command line part".  Instead,
   explain how to resolve in the Documentation patch. (Rafael)
 - Simplify the comment around hibernation_available(). (Rafael)
 - Also guide acpi_suspend_lowlevel with CONFIG_SUSPEND.

v13 -> v14:
 - New patch

---
 arch/x86/virt/vmx/tdx/tdx.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

Comments

Dave Hansen Nov. 30, 2023, 5:20 p.m. UTC | #1

On 11/9/23 03:55, Kai Huang wrote:
>  #include <asm/page.h>
> @@ -1402,6 +1404,15 @@ static int __init tdx_init(void)
>  		return -ENODEV;
>  	}
>  
> +	/*
> +	 * At this point, hibernation_available() indicates whether or
> +	 * not hibernation support has been permanently disabled.
> +	 */
> +	if (hibernation_available()) {
> +		pr_err("initialization failed: Hibernation support is enabled\n");
> +		return -ENODEV;
> +	}
> +
>  	err = register_memory_notifier(&tdx_memory_nb);
>  	if (err) {
>  		pr_err("initialization failed: register_memory_notifier() failed (%d)\n",
> @@ -1417,6 +1428,11 @@ static int __init tdx_init(void)
>  		return -ENODEV;
>  	}
>  
> +#if defined(CONFIG_ACPI) && defined(CONFIG_SUSPEND)
> +	pr_info("Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.\n");
> +	acpi_suspend_lowlevel = NULL;
> +#endif

Rafael, are you OK with how this patch ended up?  An ack would be much
appreciated if so.

Patch

diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index 53a87034ad59..cc21a0f25bee 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -26,6 +26,8 @@ 
 #include <linux/sort.h>
 #include <linux/log2.h>
 #include <linux/reboot.h>
+#include <linux/acpi.h>
+#include <linux/suspend.h>
 #include <asm/msr-index.h>
 #include <asm/msr.h>
 #include <asm/page.h>
@@ -1402,6 +1404,15 @@  static int __init tdx_init(void)
 		return -ENODEV;
 	}
 
+	/*
+	 * At this point, hibernation_available() indicates whether or
+	 * not hibernation support has been permanently disabled.
+	 */
+	if (hibernation_available()) {
+		pr_err("initialization failed: Hibernation support is enabled\n");
+		return -ENODEV;
+	}
+
 	err = register_memory_notifier(&tdx_memory_nb);
 	if (err) {
 		pr_err("initialization failed: register_memory_notifier() failed (%d)\n",
@@ -1417,6 +1428,11 @@  static int __init tdx_init(void)
 		return -ENODEV;
 	}
 
+#if defined(CONFIG_ACPI) && defined(CONFIG_SUSPEND)
+	pr_info("Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.\n");
+	acpi_suspend_lowlevel = NULL;
+#endif
+
 	/*
 	 * Just use the first TDX KeyID as the 'global KeyID' and
 	 * leave the rest for TDX guests.