From patchwork Mon May 31 09:56:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12289107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCFC1C47080 for ; Mon, 31 May 2021 10:08:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A1630611CA for ; Mon, 31 May 2021 10:08:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A1630611CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=IoJ5iIqrjovVTfUJ9rKcrj3IvKuO8Vxgt/RyM+Oz/uQ=; b=nuC2wT9NXU8dHn RenfZrxcKrt0fXWSms6KISzSJQ+KjyyWYvdx2WvwDvObtFreDk4GoypZo30c3eG/T+KoYB3O1l/p2 8q1yjG1z4q7QnXIRa7QdlF5bggKHntYece9eMadakCAjFwV1PqpYR4cUuaFwHASYnlZ0PcFG/bNTc Ui9sSePeKJwlqVc1oh8yK08sXmcJ3++1hSTFPIINnE95MBqZF6LRyvyrZaLlYtUTVtu1NbGUCPBBD F8M7uPHPADwSsM1W5M/tSE52NFMGQoFR/+8xB0dVdRWrhgLoQ2mL9wU5D1PHCGrnK/i1jEvq0VEY/ rLn8cu31kxGDzT8yo8cg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lnep3-00Bo8y-05; Mon, 31 May 2021 10:07:01 +0000 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lneoz-00Bo8C-Am for linux-arm-kernel@lists.infradead.org; Mon, 31 May 2021 10:06:58 +0000 Received: by mail-ej1-x62e.google.com with SMTP id g20so407874ejt.0 for ; Mon, 31 May 2021 03:06:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=t/wTF3aZ4LiYhQeO0iZ2OggKy7Af73zdHd6rVMfq7tg=; b=DtQvl3aaZos10i23U1s9VJkKVoW4djjaM/fzjvHJEyCuVz8icft3/jfwyoSkP1GkE0 /Pe9dNqDw1w9NwQcxAPe07PLsC/9yzA/KGcqpuX6ie/jf4EMK1UJmi9Eyt0SpOPDKO5k eeIjtl06gSRzn7fPNSaEnL52LzmqvF8jxjyY/o0ZMx1yNvUt1zs7ynjCPPm089f94q3n z/anVALvVOf1iMlj1ZOZBQqKx78MycLif9RxzDrCJlIMM3TLkCnqXa6+962wtuidDo1S IZSglN2J8mk29YJmFyeLSOB5QUwtgZpjhXOY+Xr7MSkYQrcKidZ+CEsOCh37leFT0i6B WfPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=t/wTF3aZ4LiYhQeO0iZ2OggKy7Af73zdHd6rVMfq7tg=; b=nq+PVa8V9JFgYpLfgO8F7pGn51T7YgyzfBnUmrqfHxXeTXuBokhuZhrERxbhHkgulo auok2rZzCf8mXPAT274Dy+bHgON0MFXfPXrZ7ta4IedZC4+D8PFXeiMO5HcdhdBP/JXa UKb834o8KoieiYHBwmbOC8yJ8TVIXovifnTX7Wqk9C/Jk/4ugXDzElrPjB4bok1Tq2Yj h9x2QYrrBTbhZUJgjQKIJyDx4izRIDbeugoUI7ERjMXwKYHy84p0YePngxcwRYagYsxZ f40+r9ozu/H2Jm/Qbr8eiEs2uJvjgZUn2FFxjobXTGQQ5uWQoZC81/O5nUUHZrc4EU+L KVzw== X-Gm-Message-State: AOAM532CkHVRlPuh9dU9zKtLXyNpi07D2nXcI/tZxCYRSQeJfyVSHlcR yIElnFpLbYgIjJKrs270cMLu+A== X-Google-Smtp-Source: ABdhPJxGCry3uYPtRONdFpilkA9nOhyxcczcEsRaO2bhHJ92T+Lb29yOaQxG0ci6j6Q7zA7P6X8fYA== X-Received: by 2002:a17:906:e210:: with SMTP id gf16mr22255513ejb.472.1622455615116; Mon, 31 May 2021 03:06:55 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-106-126.adslplus.ch. [84.226.106.126]) by smtp.gmail.com with ESMTPSA id cx7sm53898edb.65.2021.05.31.03.06.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 May 2021 03:06:54 -0700 (PDT) From: Jean-Philippe Brucker To: will@kernel.org, joro@8bytes.org Cc: robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, Jean-Philippe Brucker , Aaro Koskinen Subject: [PATCH] iommu/arm-smmu-v3: Ratelimit event dump Date: Mon, 31 May 2021 11:56:50 +0200 Message-Id: <20210531095648.118282-1-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210531_030657_436930_EE022F40 X-CRM114-Status: GOOD ( 18.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When a device or driver misbehaves, it is possible to receive DMA fault events much faster than we can print them out, causing a lock up of the system and inability to cancel the source of the problem. Ratelimit printing of events to help recovery. Tested-by: Aaro Koskinen Signed-off-by: Jean-Philippe Brucker --- Aiming for v5.14 rather than 5.13, since it mainly fixes a nuisance during development/debug. Conflicts with "iommu/arm-smmu-v3: Add stall support for platform devices" currently on the list [1], because they both change arm_smmu_evtq_thread(). This patch is based onto [1]. I encountered this while developing SVA on hardware, although the problem is not specific to SVA or stall. The device driver didn't properly stop DMA, and the SMMU would flood the event queue with translation faults. Without rate limiting I was unable to even reset the device. Note that this is not a problem for normal SVA operations, since userspace cannot cause DMA to print kernel messages. Aaro Koskinen reported a similar problem [2] [1] https://lore.kernel.org/linux-iommu/20210526161927.24268-4-jean-philippe@linaro.org/ [2] https://lore.kernel.org/linux-iommu/20210528080958.GA60351@darkstar.musicnaut.iki.fi/ --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 39bdb4264248..2792382ad3bd 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1518,6 +1518,8 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) struct arm_smmu_device *smmu = dev; struct arm_smmu_queue *q = &smmu->evtq.q; struct arm_smmu_ll_queue *llq = &q->llq; + static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL, + DEFAULT_RATELIMIT_BURST); u64 evt[EVTQ_ENT_DWORDS]; do { @@ -1525,7 +1527,7 @@ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) u8 id = FIELD_GET(EVTQ_0_ID, evt[0]); ret = arm_smmu_handle_evt(smmu, evt); - if (!ret) + if (!ret || !__ratelimit(&rs)) continue; dev_info(smmu->dev, "event 0x%02x received:\n", id);