From patchwork Mon May 22 14:31:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lino Sanfilippo X-Patchwork-Id: 13250696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EF8FC77B75 for ; Mon, 22 May 2023 14:32:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233657AbjEVOc2 (ORCPT ); Mon, 22 May 2023 10:32:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56462 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233574AbjEVOc2 (ORCPT ); Mon, 22 May 2023 10:32:28 -0400 Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3132BA9; Mon, 22 May 2023 07:32:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1684765901; i=linosanfilippo@gmx.de; bh=lwQ+t7RCqn9sjjERhaQcuu0ceFAVx7n0LyVEjPw+TWs=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date; b=TDxYqCU9CtVWVrlfU1RaliEa09utKq9fw1wfmUq7CqDOeK5xPtbL9lAaqGO7GWqzM z3/o+MylPzTBZskzESGi2YoWElc7sljBW3ieIPVoE0lem41Erauj+m8OJFki8aXlGQ 6XTPfVG4LHofz1Bx4XunH/cvqMOLATg6cBQfvjZQhEdR7b9GJvJiBpyqqCX0KwG8iw svAnco1yeeGU5ZlgmKiKrZAxwNqDLtYVuJbhIPXyZvZaBRL80uXboOH7JSl28cbnbh qgfDfnxzee8UkSoaVlwHX53qQkxFw87r8Cowab78xUqOffHmKanxP2AxVbKyUbkjmC z8nmVh2nlCB5Q== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from Venus.speedport.ip ([84.162.2.106]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N4hvR-1qADD027vD-011gMN; Mon, 22 May 2023 16:31:41 +0200 From: Lino Sanfilippo To: peterhuewe@gmx.de, jarkko@kernel.org, jgg@ziepe.ca Cc: jsnitsel@redhat.com, hdegoede@redhat.com, oe-lkp@lists.linux.dev, lkp@intel.com, peter.ujfalusi@linux.intel.com, peterz@infradead.org, linux@mniewoehner.de, linux-integrity@vger.kernel.org, linux-kernel@vger.kernel.org, l.sanfilippo@kunbus.com, LinoSanfilippo@gmx.de, lukas@wunner.de, p.rosenberger@kunbus.com Subject: [PATCH 1/2] tpm, tpm_tis: Handle interrupt storm Date: Mon, 22 May 2023 16:31:04 +0200 Message-Id: <20230522143105.8617-1-LinoSanfilippo@gmx.de> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 X-Provags-ID: V03:K1:yIe+WGyH1ASzJ7ENp03LTV4IXOeonm6dRKoNYtjMLPwcfYAi2iq bsV98q//ftrbGfjns3dZRRrI8a0UPsJzqcfUV3ea8Pg/tF5VZGL1bX7WEsGetOCdoX/WwFA GQvGPY7rnu6kc3KPfzmEEfx0MQXnyZweOUAuaSmk8Bnsc6ZrWh4uo7+yfvYQbFJxrcs/EIF f6xWuxrA2CptkCSbSY0/w== UI-OutboundReport: notjunk:1;M01:P0:btJ3+woI3As=;es3Q1meS12RE+lnRJ4+NzVouEpm /8b8hj2thexAdcK5YFJ/GTvX3FAw+yLNuCKhp7Vrpuc52EHwzUHYEY6tu/6oKPWDfzApqveMl eHUSqnH5lU6Gg6HdOuNcURfE9QxCFqWdc6V9rrXSlOZSH/isqt2ieKoOTdU2MBFhm7Ejl66vX RcmXz2mgx56Byh9wHBTRBhgvziYs4AVdGWOA5XznZh0YCFCHaSXJWfEefpRZwHSDfL4ipyYIB GYBEyKz/SblbSy6mMN2VlC+1EAQBoKX4fkb1A4yoOyxpaIDpRqwEvSx/vABUvTX13SMMuu5TR PNxhDLV569Ms5UGlgta5qe79fHaDfplaqZufVBmcaeVaeYlULtGCAlGIp7DGaZroy9sEt2j48 E6Khx86YLAjBmcjkASEnA26+bYAruJkCHkNyknY6CC6XN6O2irhv33t5p5wOuLANJGfCxWMSQ SfdVKY2hAdtXoPnf6MX6i1/h5fWEXgHIXQn4HiKhGjIwjeMbeMbeKQNelv+H/FycwV0eCQIMa VgwcJzgKiJ0t9Hi26GXBz+8f8x3wYCgRVcRUmanaPZrmGCeIWaVJZgCF1JQEMJyik9bNyw8jj cD/WBZ6J0YS1SEa7+uBeIicABcJdWU9AYcOkWOesG0VOVCSJLD0S2/l88rZFMSFCHTxeYdvwD Y19hgBdHJZi1iaLyWB9fVQhNSSr2tjGCksO6EBkeNhekDaIR8ilet8LesGUl9yeNRbLJBt8O3 Di0uWzMxEBEECIKMT+CIrJeztQMHknzQMYM3zxx5d9b3dqMkjD1Ie/5pwZ92vXI14pSpmp/7W XaNecdyWSR9QKvqRgjMb6o4thr0ateiXCcAFd3tkZ/JE7ChO959fVXP3TUUJm6ZEZ/XQ+3ju5 DIJlCJVnXEAmyo2eYg2htxMTvFXP8wbE62XI88N0sqHmQhY2QCFobKxhHYm63wIbgefUJle9C kALmVPKqb2KoWjEhFO2CM8KUmv0= Precedence: bulk List-ID: X-Mailing-List: linux-integrity@vger.kernel.org From: Lino Sanfilippo Commit e644b2f498d2 ("tpm, tpm_tis: Enable interrupt test") enabled interrupts instead of polling on all capable TPMs. Unfortunately, on some products the interrupt line is either never asserted or never deasserted. The former causes interrupt timeouts and is detected by tpm_tis_core_init(). The latter results in interrupt storms. Recent reports concern the Lenovo ThinkStation P360 Tiny, Lenovo ThinkPad L490 and Inspur NF5180M6: https://lore.kernel.org/linux-integrity/20230511005403.24689-1-jsnitsel@redhat.com/ https://lore.kernel.org/linux-integrity/d80b180a569a9f068d3a2614f062cfa3a78af5a6.camel@kernel.org/ The current approach to avoid those storms is to disable interrupts by adding a DMI quirk for the concerned device. However this is a maintenance burden in the long run, so use a generic approach: Detect an interrupt storm by counting the number of unhandled interrupts within a 10 ms time interval. In case that more than 1000 were unhandled deactivate interrupts, deregister the handler and fall back to polling. This equals the implementation that handles interrupt storms in note_interrupt() by means of timestamps and counters in struct irq_desc. However the function to access this structure is private so the logic has to be reimplemented in the TPM TIS core. Since handler deregistration would deadlock from within the interrupt routine trigger a worker thread that executes the unregistration. Suggested-by: Lukas Wunner Signed-off-by: Lino Sanfilippo Reviewed-by: Jerry Snitselaar --- drivers/char/tpm/tpm_tis_core.c | 71 +++++++++++++++++++++++++++++++-- drivers/char/tpm/tpm_tis_core.h | 6 +++ 2 files changed, 74 insertions(+), 3 deletions(-) base-commit: 44c026a73be8038f03dbdeef028b642880cf1511 diff --git a/drivers/char/tpm/tpm_tis_core.c b/drivers/char/tpm/tpm_tis_core.c index 558144fa707a..458ebf8c2f16 100644 --- a/drivers/char/tpm/tpm_tis_core.c +++ b/drivers/char/tpm/tpm_tis_core.c @@ -752,6 +752,55 @@ static bool tpm_tis_req_canceled(struct tpm_chip *chip, u8 status) return status == TPM_STS_COMMAND_READY; } +static void tpm_tis_handle_irq_storm(struct tpm_chip *chip) +{ + struct tpm_tis_data *priv = dev_get_drvdata(&chip->dev); + int intmask = 0; + + dev_err(&chip->dev, HW_ERR + "TPM interrupt storm detected, polling instead\n"); + + tpm_tis_read32(priv, TPM_INT_ENABLE(priv->locality), &intmask); + + intmask &= ~TPM_GLOBAL_INT_ENABLE; + + tpm_tis_request_locality(chip, 0); + tpm_tis_write32(priv, TPM_INT_ENABLE(priv->locality), intmask); + tpm_tis_relinquish_locality(chip, 0); + + chip->flags &= ~TPM_CHIP_FLAG_IRQ; + + /* + * We must not call devm_free_irq() from within the interrupt handler, + * since this function waits for running interrupt handlers to finish + * and thus it would deadlock. Instead trigger a worker that does the + * unregistration. + */ + schedule_work(&priv->free_irq_work); +} + +static void tpm_tis_process_unhandled_interrupt(struct tpm_chip *chip) +{ + const unsigned int MAX_UNHANDLED_IRQS = 1000; + struct tpm_tis_data *priv = dev_get_drvdata(&chip->dev); + /* + * The worker to free the TPM interrupt (free_irq_work) may already + * be scheduled, so make sure it is not scheduled again. + */ + if (!(chip->flags & TPM_CHIP_FLAG_IRQ)) + return; + + if (time_after(jiffies, priv->last_unhandled_irq + HZ/10)) + priv->unhandled_irqs = 1; + else + priv->unhandled_irqs++; + + priv->last_unhandled_irq = jiffies; + + if (priv->unhandled_irqs > MAX_UNHANDLED_IRQS) + tpm_tis_handle_irq_storm(chip); +} + static irqreturn_t tis_int_handler(int dummy, void *dev_id) { struct tpm_chip *chip = dev_id; @@ -761,10 +810,10 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id) rc = tpm_tis_read32(priv, TPM_INT_STATUS(priv->locality), &interrupt); if (rc < 0) - return IRQ_NONE; + goto unhandled; if (interrupt == 0) - return IRQ_NONE; + goto unhandled; set_bit(TPM_TIS_IRQ_TESTED, &priv->flags); if (interrupt & TPM_INTF_DATA_AVAIL_INT) @@ -780,10 +829,14 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id) rc = tpm_tis_write32(priv, TPM_INT_STATUS(priv->locality), interrupt); tpm_tis_relinquish_locality(chip, 0); if (rc < 0) - return IRQ_NONE; + goto unhandled; tpm_tis_read32(priv, TPM_INT_STATUS(priv->locality), &interrupt); return IRQ_HANDLED; + +unhandled: + tpm_tis_process_unhandled_interrupt(chip); + return IRQ_HANDLED; } static void tpm_tis_gen_interrupt(struct tpm_chip *chip) @@ -804,6 +857,15 @@ static void tpm_tis_gen_interrupt(struct tpm_chip *chip) chip->flags &= ~TPM_CHIP_FLAG_IRQ; } +static void tpm_tis_free_irq_func(struct work_struct *work) +{ + struct tpm_tis_data *priv = container_of(work, typeof(*priv), free_irq_work); + struct tpm_chip *chip = priv->chip; + + devm_free_irq(chip->dev.parent, priv->irq, chip); + priv->irq = 0; +} + /* Register the IRQ and issue a command that will cause an interrupt. If an * irq is seen then leave the chip setup for IRQ operation, otherwise reverse * everything and leave in polling mode. Returns 0 on success. @@ -816,6 +878,7 @@ static int tpm_tis_probe_irq_single(struct tpm_chip *chip, u32 intmask, int rc; u32 int_status; + INIT_WORK(&priv->free_irq_work, tpm_tis_free_irq_func); rc = devm_request_threaded_irq(chip->dev.parent, irq, NULL, tis_int_handler, IRQF_ONESHOT | flags, @@ -918,6 +981,7 @@ void tpm_tis_remove(struct tpm_chip *chip) interrupt = 0; tpm_tis_write32(priv, reg, ~TPM_GLOBAL_INT_ENABLE & interrupt); + flush_work(&priv->free_irq_work); tpm_tis_clkrun_enable(chip, false); @@ -1021,6 +1085,7 @@ int tpm_tis_core_init(struct device *dev, struct tpm_tis_data *priv, int irq, chip->timeout_b = msecs_to_jiffies(TIS_TIMEOUT_B_MAX); chip->timeout_c = msecs_to_jiffies(TIS_TIMEOUT_C_MAX); chip->timeout_d = msecs_to_jiffies(TIS_TIMEOUT_D_MAX); + priv->chip = chip; priv->timeout_min = TPM_TIMEOUT_USECS_MIN; priv->timeout_max = TPM_TIMEOUT_USECS_MAX; priv->phy_ops = phy_ops; diff --git a/drivers/char/tpm/tpm_tis_core.h b/drivers/char/tpm/tpm_tis_core.h index e978f457fd4d..6fc86baa4398 100644 --- a/drivers/char/tpm/tpm_tis_core.h +++ b/drivers/char/tpm/tpm_tis_core.h @@ -91,12 +91,18 @@ enum tpm_tis_flags { }; struct tpm_tis_data { + struct tpm_chip *chip; u16 manufacturer_id; struct mutex locality_count_mutex; unsigned int locality_count; int locality; + /* Interrupts */ int irq; + struct work_struct free_irq_work; + unsigned long last_unhandled_irq; + unsigned int unhandled_irqs; unsigned int int_mask; + unsigned long flags; void __iomem *ilb_base_addr; u16 clkrun_enabled;