From patchwork Fri Feb 14 13:10:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 13974885 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B96423A992; Fri, 14 Feb 2025 13:11:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739538685; cv=none; b=NwOlyPyt4P09/8utpaBuuEP68gwEzcZ9p9eLHAgvOppskyfMtTluXyOgfiE9pABtz3Wjei5+baCdHDvuN+ebBnWzOkaHccLI/LRPPRTTXOWheCI1wCTnQ7YC6tBTzaSRp2MIsg6JgOe81XEDTnTDwRZon05guhgz5FIYyfM/CWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739538685; c=relaxed/simple; bh=QnE3MWbyEccydmUgWLoslOgDk1WgUiqPUae/4stuq/o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=E3aQ4s4cS/EuPFWqfyKWCBjlCZyyZeSah8RoBZfIiJgktwjJCqUtbjsMedEJbJqNyZfecrLDjsw9wtMYehnzuAAp6b7Z++ox+qrn8fuONff3bJttbv1FCIsWIyxSnpU1dP9cfaTjUR2uHMmYERyqMXfv7cyUa5chil2AE3bCUoQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Q6pUdF5K; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Q6pUdF5K" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51EAtB3K000524; Fri, 14 Feb 2025 13:11:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=nv+m3o J4hLmiasN4bXS5CzCbQltOL5gRuC2PqhirWos=; b=Q6pUdF5KSobwOAEE309b7+ 5GXTa2zQMLDPa/osPRqpDgPxuclJcxQsY/rQxT3yb753mPxlryH+i3Qu+mUTdI16 h+5/7G60j1YjE53fm1wsEprvAqbvDD/1ypNcUteIAs93CLhsTJ7T61AC9UAY7Ec2 tPEdKN02Re8cztg3osCUcYNR3TqQTN6jaQgNpLV1fCVDHa5eUROp3LNUtFgIViT4 GH69PNTUbHuvOANp3C336uS44ySRLRPkDdjDa3f68iusC7AZWuH0QzwRGlmLuHDt Xd/zslrU7YDd04trtbZ852GtUlgjghjBzMzJph6euMltP0CPC1rc6/LtuL8I4MPQ == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44ssvabbxy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:07 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 51ED9QCo027604; Fri, 14 Feb 2025 13:11:06 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44ssvabbxw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:06 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 51EBHjpw000924; Fri, 14 Feb 2025 13:11:05 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 44pjknkphc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:05 +0000 Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 51EDB4SA26608160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Feb 2025 13:11:04 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1D3E85806B; Fri, 14 Feb 2025 13:11:04 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4EC4A58067; Fri, 14 Feb 2025 13:11:01 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 14 Feb 2025 13:11:01 +0000 (GMT) From: Niklas Schnelle Date: Fri, 14 Feb 2025 14:10:52 +0100 Subject: [PATCH v6 1/3] s390/pci: Fix s390_mmio_read/write syscall page fault handling Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-vfio_pci_mmap-v6-1-6f300cb63a7e@linux.ibm.com> References: <20250214-vfio_pci_mmap-v6-0-6f300cb63a7e@linux.ibm.com> In-Reply-To: <20250214-vfio_pci_mmap-v6-0-6f300cb63a7e@linux.ibm.com> To: Bjorn Helgaas , Christoph Hellwig , Alexandra Winter , Alex Williamson , Gerd Bayer , Matthew Rosato , Jason Gunthorpe , Thorsten Winkler , Bjorn Helgaas Cc: Julian Ruess , Halil Pasic , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-pci@vger.kernel.org, Niklas Schnelle X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2001; i=schnelle@linux.ibm.com; h=from:subject:message-id; bh=QnE3MWbyEccydmUgWLoslOgDk1WgUiqPUae/4stuq/o=; b=owGbwMvMwCX2Wz534YHOJ2GMp9WSGNLXO9zfKNCZ3LWfy6qHd7uM77EjCm1e0/qmeMk+1Hw42 eHukvv/O0pZGMS4GGTFFFkWdTn7rSuYYronqL8DZg4rE8gQBi5OAZiIjiUjwwTXwtapG86ujD0o /3RKWTbP08DbDD+kHtycIjF7aqyZVSkjw8/ipbHCcas5Ns0+XcE6d63YspzXH50afY1YC3d9eNj WwgoA X-Developer-Key: i=schnelle@linux.ibm.com; a=openpgp; fpr=9DB000B2D2752030A5F72DDCAFE43F15E8C26090 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: WFaV4Qu094bww62jFQbPfuU1rdkxorfX X-Proofpoint-ORIG-GUID: a4BLbTqTnfySI6XOyiNN32nHh0Ng0Ziq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-14_05,2025-02-13_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 adultscore=0 mlxscore=0 spamscore=0 bulkscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 impostorscore=0 mlxlogscore=890 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2501170000 definitions=main-2502140096 The s390 MMIO syscalls when using the classic PCI instructions do not cause a page fault when follow_pfnmap_start() fails due to the page not being present. Besides being a general deficiency this breaks vfio-pci's mmap() handling once VFIO_PCI_MMAP gets enabled as this lazily maps on first access. Fix this by following a failed follow_pfnmap_start() with fixup_user_page() and retrying the follow_pfnmap_start(). Also fix a VM_READ vs VM_WRITE mixup in the read syscall. Signed-off-by: Niklas Schnelle --- arch/s390/pci/pci_mmio.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/s390/pci/pci_mmio.c b/arch/s390/pci/pci_mmio.c index 46f99dc164ade4ca10f170cd66bdb648f92aa904..1997d9b7965df3b9b6019c7537028cd29d52fc13 100644 --- a/arch/s390/pci/pci_mmio.c +++ b/arch/s390/pci/pci_mmio.c @@ -175,8 +175,12 @@ SYSCALL_DEFINE3(s390_pci_mmio_write, unsigned long, mmio_addr, args.address = mmio_addr; args.vma = vma; ret = follow_pfnmap_start(&args); - if (ret) - goto out_unlock_mmap; + if (ret) { + fixup_user_fault(current->mm, mmio_addr, FAULT_FLAG_WRITE, NULL); + ret = follow_pfnmap_start(&args); + if (ret) + goto out_unlock_mmap; + } io_addr = (void __iomem *)((args.pfn << PAGE_SHIFT) | (mmio_addr & ~PAGE_MASK)); @@ -315,14 +319,18 @@ SYSCALL_DEFINE3(s390_pci_mmio_read, unsigned long, mmio_addr, if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) goto out_unlock_mmap; ret = -EACCES; - if (!(vma->vm_flags & VM_WRITE)) + if (!(vma->vm_flags & VM_READ)) goto out_unlock_mmap; args.vma = vma; args.address = mmio_addr; ret = follow_pfnmap_start(&args); - if (ret) - goto out_unlock_mmap; + if (ret) { + fixup_user_fault(current->mm, mmio_addr, 0, NULL); + ret = follow_pfnmap_start(&args); + if (ret) + goto out_unlock_mmap; + } io_addr = (void __iomem *)((args.pfn << PAGE_SHIFT) | (mmio_addr & ~PAGE_MASK)); From patchwork Fri Feb 14 13:10:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 13974886 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4481B264F95; Fri, 14 Feb 2025 13:11:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739538689; cv=none; b=IfHspnNXmqvpar/2qclmeODZxwPtq5U8QTrC0n0q7RVdHbD59USAE/1wBlkdSqiqxDvHbIKuOREv1NyKxfRdjcdHGwQ4LvZlHxzy9M0OOayR/NoFGVqusxP0nS0T+kCd8xmCI0oQHj5QxiP6pylNN2SIHLX8UW+HkZrUaYrCVMQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739538689; c=relaxed/simple; bh=DLmfYThBDFVQ+hWhxQ3kMqvOqu7rxr+CGjFuDVLQYlA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BnPORnXmx8I5e39/qmGtRACv475m+xkdIS2IuPiPNywoY47QIg8akQVW7NjbdnzMGPKmIDDumNPHsJMwaXejKFLUp0+Zerjmx/9rCwkklqO0wwtmfR6X3YIccKkeBwTb1Twzv15WX21Yl0Cg/2HCufI2LgREvGsT4BMjMpRx1ik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Q1/d+Ome; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Q1/d+Ome" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51E7X52c026534; Fri, 14 Feb 2025 13:11:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=7VK1Gy fSvq9CVIM9TTtp4PeYB5rdR+xb+tX5wL0LZeE=; b=Q1/d+OmeZKdQ9iwE0Aihb2 UFzGYtcXCHeY+m066Dgy/ai6WRE5kkzrOV5eJ2QaBtwuUcabcbG2YFq9drkRU7yM HHgN8a5kD77gaG2I9/8TjbvfuVdlu6EmysAobUuVQf+igpYM91+/ZVF7kGCwjQW3 Hcfgy2fwlFn/lz4qmLCjx4NvroRRiBnABep6ZxbRgP+WN2Y9C3bfPU90VjhchG/3 svC71Fd9EB9Fz61ODmteoS5bZXOTom9kvmAFUePOUc5JuWU7P+kbnI3/fN1SUbn6 Q1qhgzc+bEMQYQ6Vh22XvO7J5pvy7gxWPn08+RYTYZtdjb7XlQOa+E7ZHM3VrPgA == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44t1hpsfrw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:09 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 51ED1As8010985; Fri, 14 Feb 2025 13:11:09 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44t1hpsfrt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:09 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 51EAkAd1028217; Fri, 14 Feb 2025 13:11:08 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 44phyyuufs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:08 +0000 Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 51EDB78q30999292 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Feb 2025 13:11:07 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 241E158052; Fri, 14 Feb 2025 13:11:07 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 574445805D; Fri, 14 Feb 2025 13:11:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 14 Feb 2025 13:11:04 +0000 (GMT) From: Niklas Schnelle Date: Fri, 14 Feb 2025 14:10:53 +0100 Subject: [PATCH v6 2/3] PCI: s390: Support mmap() of BARs and replace VFIO_PCI_MMAP by a device flag Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-vfio_pci_mmap-v6-2-6f300cb63a7e@linux.ibm.com> References: <20250214-vfio_pci_mmap-v6-0-6f300cb63a7e@linux.ibm.com> In-Reply-To: <20250214-vfio_pci_mmap-v6-0-6f300cb63a7e@linux.ibm.com> To: Bjorn Helgaas , Christoph Hellwig , Alexandra Winter , Alex Williamson , Gerd Bayer , Matthew Rosato , Jason Gunthorpe , Thorsten Winkler , Bjorn Helgaas Cc: Julian Ruess , Halil Pasic , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-pci@vger.kernel.org, Niklas Schnelle X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6480; i=schnelle@linux.ibm.com; h=from:subject:message-id; bh=DLmfYThBDFVQ+hWhxQ3kMqvOqu7rxr+CGjFuDVLQYlA=; b=owGbwMvMwCX2Wz534YHOJ2GMp9WSGNLXOzy4NW9dyIHd68xXrC14KnVSaenrgKBZ1mf3vppXe mvT6bbSfR2lLAxiXAyyYoosi7qc/dYVTDHdE9TfATOHlQlkCAMXpwBMJGwjw//K9MlnF+xYtq9Y SrR7Ii/f7OfPfmlN6Dixp/XEs/8ps6YcZGR4N/uy2I5JjzWUT154d+jLo5iZL3Q+vZp7UczAJD7 plWE9MwA= X-Developer-Key: i=schnelle@linux.ibm.com; a=openpgp; fpr=9DB000B2D2752030A5F72DDCAFE43F15E8C26090 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: aZFVCInqKdzodbj83vmo9te2V00KMDhb X-Proofpoint-GUID: m-JqxHcI-ofgLx2TVM8w6vxXXPJbyCtI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-14_05,2025-02-13_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 impostorscore=0 phishscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 clxscore=1015 malwarescore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2501170000 definitions=main-2502140096 On s390 there is a virtual PCI device called ISM which has a few peculiarities. For one, it presents a 256 TiB PCI BAR whose size leads to any attempt to ioremap() the whole BAR failing. This is problematic since mapping the whole BAR is the default behavior of for example vfio-pci in combination with QEMU and VFIO_PCI_MMAP enabled. Even if one tried to map this BAR only partially, the mapping would not be usable without extra precautions on systems with MIO support enabled. This is because of another oddity, in that this virtual PCI device does not support the newer memory I/O (MIO) PCI instructions and legacy PCI instructions are not accessible through writeq()/readq() when MIO is in use. In short the ISM device's BAR is not accessible through memory mappings. Indicate this by introducing a new non_mappable_bars flag for the ISM device and set it using a PCI quirk. Use this flag instead of the VFIO_PCI_MMAP Kconfig option to block mapping with vfio-pci. This was the only use of the Kconfig option so remove it. As s390x has no PCI quirk handling add basic support modeled after x86's arch/x86/pci/fixup.c and move the ISM device's PCI ID to the common header to make it accessible. Also enable CONFIG_PCI_QUIRKS whenever CONFIG_PCI is enabled. Acked-by: Bjorn Helgaas Signed-off-by: Niklas Schnelle --- arch/s390/Kconfig | 4 +--- arch/s390/pci/Makefile | 2 +- arch/s390/pci/pci_fixup.c | 23 +++++++++++++++++++++++ drivers/s390/net/ism_drv.c | 1 - drivers/vfio/pci/Kconfig | 4 ---- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/linux/pci.h | 1 + include/linux/pci_ids.h | 1 + 8 files changed, 28 insertions(+), 10 deletions(-) diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index 9c9ec08d78c71b4d227beeafab1b82d6434cb5c7..e48741e001476f765e8aba0037a1b386df393683 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -41,9 +41,6 @@ config AUDIT_ARCH config NO_IOPORT_MAP def_bool y -config PCI_QUIRKS - def_bool n - config ARCH_SUPPORTS_UPROBES def_bool y @@ -258,6 +255,7 @@ config S390 select PCI_DOMAINS if PCI select PCI_MSI if PCI select PCI_MSI_ARCH_FALLBACKS if PCI_MSI + select PCI_QUIRKS if PCI select SPARSE_IRQ select SWIOTLB select SYSCTL_EXCEPTION_TRACE diff --git a/arch/s390/pci/Makefile b/arch/s390/pci/Makefile index df73c5182990ad3ae4ed5a785953011feb9a093c..1810e0944a4ed9d31261788f0f6eb341e5316546 100644 --- a/arch/s390/pci/Makefile +++ b/arch/s390/pci/Makefile @@ -5,6 +5,6 @@ obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_clp.o \ pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ - pci_bus.o pci_kvm_hook.o pci_report.o + pci_bus.o pci_kvm_hook.o pci_report.o pci_fixup.o obj-$(CONFIG_PCI_IOV) += pci_iov.o obj-$(CONFIG_SYSFS) += pci_sysfs.o diff --git a/arch/s390/pci/pci_fixup.c b/arch/s390/pci/pci_fixup.c new file mode 100644 index 0000000000000000000000000000000000000000..35688b645098329f082d0c40cc8c59231c390eaa --- /dev/null +++ b/arch/s390/pci/pci_fixup.c @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Exceptions for specific devices, + * + * Copyright IBM Corp. 2025 + * + * Author(s): + * Niklas Schnelle + */ +#include + +static void zpci_ism_bar_no_mmap(struct pci_dev *pdev) +{ + /* + * ISM's BAR is special. Drivers written for ISM know + * how to handle this but others need to be aware of their + * special nature e.g. to prevent attempts to mmap() it. + */ + pdev->non_mappable_bars = 1; +} +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_IBM, + PCI_DEVICE_ID_IBM_ISM, + zpci_ism_bar_no_mmap); diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index e36e3ea165d3b2b01d68e53634676cb8c2c40220..d32633ed9fa80c1764724f493b363bfd6cb4f9cf 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -20,7 +20,6 @@ MODULE_DESCRIPTION("ISM driver for s390"); MODULE_LICENSE("GPL"); -#define PCI_DEVICE_ID_IBM_ISM 0x04ED #define DRV_NAME "ism" static const struct pci_device_id ism_device_table[] = { diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index bf50ffa10bdea9e52a9d01cc3d6ee4cade39a08c..c3bcb6911c538286f7985f9c5e938d587fc04b56 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -7,10 +7,6 @@ config VFIO_PCI_CORE select VFIO_VIRQFD select IRQ_BYPASS_MANAGER -config VFIO_PCI_MMAP - def_bool y if !S390 - depends on VFIO_PCI_CORE - config VFIO_PCI_INTX def_bool y if !S390 depends on VFIO_PCI_CORE diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 586e49efb81be32ccb50ca554a60cec684c37402..c8586d47704c74cf9a5256d65bbf888db72b2f91 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -116,7 +116,7 @@ static void vfio_pci_probe_mmaps(struct vfio_pci_core_device *vdev) res = &vdev->pdev->resource[bar]; - if (!IS_ENABLED(CONFIG_VFIO_PCI_MMAP)) + if (vdev->pdev->non_mappable_bars) goto no_mmap; if (!(res->flags & IORESOURCE_MEM)) diff --git a/include/linux/pci.h b/include/linux/pci.h index 47b31ad724fa5bf7abd7c3dc572947551b0f2148..7192b9d78d7e337ce6144190325458fe3c0f1696 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -476,6 +476,7 @@ struct pci_dev { unsigned int no_command_memory:1; /* No PCI_COMMAND_MEMORY */ unsigned int rom_bar_overlap:1; /* ROM BAR disable broken */ unsigned int rom_attr_enabled:1; /* Display of ROM attribute enabled? */ + unsigned int non_mappable_bars:1; /* BARs can't be mapped to user-space */ pci_dev_flags_t dev_flags; atomic_t enable_cnt; /* pci_enable_device has been called */ diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index de5deb1a0118fcf56570d461cbe7a501d4bd0da3..e0cdc290dff5cef39ba491d49008c4153b9b9136 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -518,6 +518,7 @@ #define PCI_DEVICE_ID_IBM_ICOM_V2_ONE_PORT_RVX_ONE_PORT_MDM 0x0251 #define PCI_DEVICE_ID_IBM_ICOM_V2_ONE_PORT_RVX_ONE_PORT_MDM_PCIE 0x0361 #define PCI_DEVICE_ID_IBM_ICOM_FOUR_PORT_MODEL 0x252 +#define PCI_DEVICE_ID_IBM_ISM 0x04ed #define PCI_SUBVENDOR_ID_IBM 0x1014 #define PCI_SUBDEVICE_ID_IBM_SATURN_SERIAL_ONE_PORT 0x03d4 From patchwork Fri Feb 14 13:10:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 13974887 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84BC123A992; Fri, 14 Feb 2025 13:11:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739538691; cv=none; b=eu7M4Gw9e9BmAPYhUP2lX/xrflv8tb1oHlyi1trDY7c74srWB6YVT2QDIGYj0XWsIxSI0pVRvWtGfOVDRWJoQlzTf/W5ZjOQq+RoJVeYT0BM762+TPzV7K1XV+Q33BTo8wVY2ohvcCY2fTJQ7VbCMGt1+XFh7y2TdNdgmfxfSTk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739538691; c=relaxed/simple; bh=oZzm7bjnXY99I6Zc3jVr5s2rPCWnBBpDlteWHQeKPAg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WGrnl+CkhgFO9RM8HcDx6gt8evUU8aqGwZyeC/FUU6kTIUUKhMdFcFtsHFOb2yFG/Vo2yx1wb+69uiLWEiRPqTfwRBZaFi3P4aei015Dd7hwy7qMj09R5HR/xn/lx8koRUBM3ZeGB1AUkrgT5tKUkEfNZ8iTOJ5bj4ZomN/9xjY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=rotjHjcz; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="rotjHjcz" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51E44kDs015865; Fri, 14 Feb 2025 13:11:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=iQNQDA TVZJVA7sMUS+nMqNiPw40LRstanI4kXZIak5Y=; b=rotjHjcztgpss0h9YnYueE fH2y9zIuCZGV/2X4lQySlVSw9Uw9MUzZxhtUW/J6wWPWVGEbqCKTU1tyEHFBOmNw RpGLCayrLEGrVMqA8RfGS6slP2Qf28tD9ORr6tBL2tf5lboa4+Tup0l1L0Z0fslG ceDoKGdLHXmhb5gWlYm3hTwWOnAPxRjS82N/vZDqv2j/wLxwfPBl1z5prePxggL3 ieNm5pG+dSefWzOKI7xiA4iqH6YUGl1o0nEdsKRlhPxgS+8sJlNAfaD7VvGDfnho FFjxb0RFVjqEkuFZW4SheItIxRCFWb8fk4MfiHScdliDFXZlM66iASvqsJcWb8KA == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44skjuwe49-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:13 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 51ED9RZW019399; Fri, 14 Feb 2025 13:11:12 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44skjuwe46-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:12 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 51EAK5cH028221; Fri, 14 Feb 2025 13:11:11 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 44phyyuug1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Feb 2025 13:11:11 +0000 Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 51EDBAnZ29426278 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Feb 2025 13:11:10 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2C19D5806B; Fri, 14 Feb 2025 13:11:10 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5E71458052; Fri, 14 Feb 2025 13:11:07 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 14 Feb 2025 13:11:07 +0000 (GMT) From: Niklas Schnelle Date: Fri, 14 Feb 2025 14:10:54 +0100 Subject: [PATCH v6 3/3] PCI: s390: Enable HAVE_PCI_MMAP on s390 and restrict mmap() of resources to mappable BARs Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250214-vfio_pci_mmap-v6-3-6f300cb63a7e@linux.ibm.com> References: <20250214-vfio_pci_mmap-v6-0-6f300cb63a7e@linux.ibm.com> In-Reply-To: <20250214-vfio_pci_mmap-v6-0-6f300cb63a7e@linux.ibm.com> To: Bjorn Helgaas , Christoph Hellwig , Alexandra Winter , Alex Williamson , Gerd Bayer , Matthew Rosato , Jason Gunthorpe , Thorsten Winkler , Bjorn Helgaas Cc: Julian Ruess , Halil Pasic , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-pci@vger.kernel.org, Niklas Schnelle X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3239; i=schnelle@linux.ibm.com; h=from:subject:message-id; bh=oZzm7bjnXY99I6Zc3jVr5s2rPCWnBBpDlteWHQeKPAg=; b=owGbwMvMwCX2Wz534YHOJ2GMp9WSGNLXOzx05VXpUpY6Pt1s8m0+ztVTdd9OSeXY+bDWV8JHg zvJPsWno5SFQYyLQVZMkWVRl7PfuoIppnuC+jtg5rAygQxh4OIUgIm8y2RkuDz5b+HScxEHFm2S 6Vt5T77nbanDil93WVZHHVfV/LD0fDDDPy3rYoVurirDs88tE1kfZ0yZ1BcnsTKE7cQX/djkxLI cVgA= X-Developer-Key: i=schnelle@linux.ibm.com; a=openpgp; fpr=9DB000B2D2752030A5F72DDCAFE43F15E8C26090 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: l-dImMFPgbWZdTA0gSjIhc-jmoW6UfA5 X-Proofpoint-ORIG-GUID: MicYpK5s-NuHQl-dwLUgCzVIoGTeeour X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-14_05,2025-02-13_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 mlxlogscore=651 lowpriorityscore=0 mlxscore=0 spamscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 malwarescore=0 adultscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2501170000 definitions=main-2502140096 So far s390 does not select HAVE_PCI_MMAP. This is partly because access to mapped PCI resources requires special PCI load/store instructions and prior to commit 71ba41c9b1d9 ("s390/pci: provide support for MIO instructions") even required use of special syscalls. This really isn't a showstopper though and in fact lack of HAVE_PCI_MMAP has previously caused extra work when testing and debugging PCI devices and drivers. Another issue when looking at HAVE_PCI_MMAP however comes from the virtual ISM devices. These present 256 TiB BARs which really can't be accessed via a mapping to user-space. Now, the newly added pdev->non_mappable_bars flag provides a way to exclude devices whose BARs can't be mapped to user-space including the s390 ISM device. So honor this flag also in the mmap() paths protected by HAVE_PCI_MMMAP and with the ISM device thus excluded enable HAVE_PCI_MMAP for s390. Note that most distributions enable CONFIG_IO_STRICT_DEVMEM=y and require unbinding drivers before resources can be mapped. This makes it extremely unlikely that any existing programs on s390 will now suddenly fail after succeeding to mmap() resources and then trying to access the mapping without use of the special PCI instructions. Link: https://lore.kernel.org/lkml/20250212132808.08dcf03c.alex.williamson@redhat.com/ Suggested-by: Alex Williamson Signed-off-by: Niklas Schnelle --- arch/s390/include/asm/pci.h | 4 ++++ drivers/pci/pci-sysfs.c | 4 ++++ drivers/pci/proc.c | 4 ++++ 3 files changed, 12 insertions(+) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 474e1f8d1d3c2fc5685b459cc68b67ac651ea3e9..518dd71a78c83c74dc7b29778e299d5c8cabcc59 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -11,6 +11,10 @@ #include #include +#define HAVE_PCI_MMAP 1 +#define ARCH_GENERIC_PCI_MMAP_RESOURCE 1 +#define arch_can_pci_mmap_wc() 1 + #define PCIBIOS_MIN_IO 0x1000 #define PCIBIOS_MIN_MEM 0x10000000 diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index b46ce1a2c5542cdea0a3f9df324434fdb7e8a4d2..7373eca0a4943bf896b4a177124e0d4572baec2b 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -1257,6 +1257,10 @@ static int pci_create_resource_files(struct pci_dev *pdev) int i; int retval; + /* Skip devices with non-mappable BARs */ + if (pdev->non_mappable_bars) + return 0; + /* Expose the PCI resources from this device as files */ for (i = 0; i < PCI_STD_NUM_BARS; i++) { diff --git a/drivers/pci/proc.c b/drivers/pci/proc.c index f967709082d654a101039091b5493b2dec5f57b4..9348a0fb808477ca9be80a8b88bbc036565bc411 100644 --- a/drivers/pci/proc.c +++ b/drivers/pci/proc.c @@ -251,6 +251,10 @@ static int proc_bus_pci_mmap(struct file *file, struct vm_area_struct *vma) security_locked_down(LOCKDOWN_PCI_ACCESS)) return -EPERM; + /* Skip devices with non-mappable BARs */ + if (dev->non_mappable_bars) + return -EINVAL; + if (fpriv->mmap_state == pci_mmap_io) { if (!arch_can_pci_mmap_io()) return -EINVAL;