From patchwork Wed Apr 1 15:46:05 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cam Macdonell X-Patchwork-Id: 15714 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n31FkD5h022707 for ; Wed, 1 Apr 2009 15:46:13 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751805AbZDAPqN (ORCPT ); Wed, 1 Apr 2009 11:46:13 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752802AbZDAPqM (ORCPT ); Wed, 1 Apr 2009 11:46:12 -0400 Received: from fleet.cs.ualberta.ca ([129.128.22.22]:50831 "EHLO fleet.cs.ualberta.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751018AbZDAPqL (ORCPT ); Wed, 1 Apr 2009 11:46:11 -0400 Received: from localhost.localdomain (st-brides.cs.ualberta.ca [129.128.23.21]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-auth.cs.ualberta.ca (Postfix) with ESMTP id 17E5A28005; Wed, 1 Apr 2009 09:46:09 -0600 (MDT) From: Cam Macdonell To: kvm@vger.kernel.org Cc: Cam Macdonell Subject: [PATCH] Guest device driver for an inter-VM shared memory PCI device that maps a shared file (from /dev/shm) on the devices memory. Date: Wed, 1 Apr 2009 09:46:05 -0600 Message-Id: <1238600765-9193-1-git-send-email-cam@cs.ualberta.ca> X-Mailer: git-send-email 1.6.0.6 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This driver corresponds to the shared memory PCI device that maps a host file into shared memory on the device. Accessing the device can be done through creating a device file on the guest num =`cat /proc/devices | grep kvm_ivshmem | awk '{print $1}` mknod --mode=666 /dev/ivshmem $num 0 read, write, lseek and mmap are supported, but mmap is the usual usage to get zero-copy communication. The driver contains the initial interrupt support, but I have not yet added the unix domain socket to support interrupts yet. --- drivers/char/Kconfig | 8 + drivers/char/Makefile | 2 + drivers/char/kvm_ivshmem.c | 370 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 380 insertions(+), 0 deletions(-) create mode 100644 drivers/char/kvm_ivshmem.c diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig index 735bbe2..afa7cb8 100644 --- a/drivers/char/Kconfig +++ b/drivers/char/Kconfig @@ -1099,6 +1099,14 @@ config DEVPORT depends on ISA || PCI default y +config KVM_IVSHMEM + tristate "Inter-VM Shared Memory Device" + depends on PCI + default m + help + This device maps a region of shared memory between the host OS and any + number of virtual machines. + source "drivers/s390/char/Kconfig" endmenu diff --git a/drivers/char/Makefile b/drivers/char/Makefile index 9caf5b5..021f06b 100644 --- a/drivers/char/Makefile +++ b/drivers/char/Makefile @@ -111,6 +111,8 @@ obj-$(CONFIG_PS3_FLASH) += ps3flash.o obj-$(CONFIG_JS_RTC) += js-rtc.o js-rtc-y = rtc.o +obj-$(CONFIG_KVM_IVSHMEM) += kvm_ivshmem.o + # Files generated that shall be removed upon make clean clean-files := consolemap_deftbl.c defkeymap.c diff --git a/drivers/char/kvm_ivshmem.c b/drivers/char/kvm_ivshmem.c new file mode 100644 index 0000000..7d46ac4 --- /dev/null +++ b/drivers/char/kvm_ivshmem.c @@ -0,0 +1,370 @@ +/* + * drivers/char/kvm_ivshmem.c - driver for KVM Inter-VM shared memory PCI device + * + * Copyright 2009 Cam Macdonell + * + * Based on cirrusfb.c and 8139cp.c: + * Copyright 1999-2001 Jeff Garzik + * Copyright 2001-2004 Jeff Garzik + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define TRUE 1 +#define FALSE 0 +#define KVM_IVSHMEM_DEVICE_MINOR_NUM 0 + +enum { + /* KVM Shmem device register offsets */ + IntrMask = 0x00, /* Interrupt Mask */ + IntrStatus = 0x10, /* Interrupt Status */ + + ShmOK = 1 /* Everything is OK */ +}; + +typedef struct kvm_ivshmem_device { + void __iomem * regs; + + void * base_addr; + + unsigned int regaddr; + unsigned int reg_size; + + unsigned int ioaddr; + unsigned int ioaddr_size; + unsigned int irq; + + bool enabled; + spinlock_t dev_spinlock; +} kvm_ivshmem_device; + +static kvm_ivshmem_device kvm_ivshmem_dev; + +static int device_major_nr; + +static int kvm_ivshmem_mmap(struct file *, struct vm_area_struct *); +static int kvm_ivshmem_open(struct inode *, struct file *); +static int kvm_ivshmem_release(struct inode *, struct file *); +static ssize_t kvm_ivshmem_read(struct file *, char *, size_t, loff_t *); +static ssize_t kvm_ivshmem_write(struct file *, const char *, size_t, loff_t *); +static loff_t kvm_ivshmem_lseek(struct file * filp, loff_t offset, int origin); + +static const struct file_operations kvm_ivshmem_ops = { + .owner = THIS_MODULE, + .open = kvm_ivshmem_open, + .mmap = kvm_ivshmem_mmap, + .read = kvm_ivshmem_read, + .write = kvm_ivshmem_write, + .llseek = kvm_ivshmem_lseek, + .release = kvm_ivshmem_release, +}; + +static struct pci_device_id kvm_ivshmem_id_table[] = { + { 0x1af4, 0x1110, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 }, + { 0 }, +}; +MODULE_DEVICE_TABLE (pci, kvm_ivshmem_id_table); + +static void kvm_ivshmem_remove_device(struct pci_dev* pdev); +static int kvm_ivshmem_probe_device (struct pci_dev *pdev, + const struct pci_device_id * ent); + +static struct pci_driver kvm_ivshmem_pci_driver = { + .name = "kvm-shmem", + .id_table = kvm_ivshmem_id_table, + .probe = kvm_ivshmem_probe_device, + .remove = kvm_ivshmem_remove_device, +}; + +static ssize_t kvm_ivshmem_read(struct file * filp, char * buffer, size_t len, + loff_t * poffset) +{ + + int bytes_read = 0; + unsigned long offset; + + offset = *poffset; + + printk(KERN_INFO "kvm_ivshmem: trying to read\n"); + if (!kvm_ivshmem_dev.base_addr) { + printk(KERN_ERR "KVM_IVSHMEM: cannot read from ioaddr (NULL)\n"); + return 0; + } + + if (len > kvm_ivshmem_dev.ioaddr_size - offset) { + len = kvm_ivshmem_dev.ioaddr_size - offset; + } + + printk(KERN_INFO "KVM_IVSHMEM: len is %u\n", (unsigned) len); + if (len == 0) return 0; + + bytes_read = copy_to_user(buffer, kvm_ivshmem_dev.base_addr+offset, len); + if (bytes_read > 0) { + return -EFAULT; + } + + printk(KERN_INFO "KVM_IVSHMEM: read %u bytes at offset %lu\n", + (unsigned) len, offset); + *poffset += len; + return len; +} + +static loff_t kvm_ivshmem_lseek(struct file * filp, loff_t offset, int origin) +{ + + loff_t retval = -1; + + switch (origin) { + case 1: + offset += filp->f_pos; + case 0: + retval = offset; + if (offset > kvm_ivshmem_dev.ioaddr_size) { + offset = kvm_ivshmem_dev.ioaddr_size; + } + filp->f_pos = offset; + } + + return retval; +} + +static ssize_t kvm_ivshmem_write(struct file * filp, const char * buffer, + size_t len, loff_t * poffset) +{ + + int bytes_written = 0; + unsigned long offset; + + offset = *poffset; + + printk(KERN_INFO "KVM_IVSHMEM: trying to write\n"); + if (!kvm_ivshmem_dev.base_addr) { + printk(KERN_ERR "KVM_IVSHMEM: cannot write to ioaddr (NULL)\n"); + return 0; + } + + if (len > kvm_ivshmem_dev.ioaddr_size - offset) { + len = kvm_ivshmem_dev.ioaddr_size - offset; + } + + printk(KERN_INFO "KVM_IVSHMEM: len is %u\n", (unsigned) len); + if (len == 0) return 0; + + bytes_written = copy_from_user(kvm_ivshmem_dev.base_addr+offset, + buffer, len); + if (bytes_written > 0) { + return -EFAULT; + } + + printk(KERN_INFO "KVM_IVSHMEM: wrote %u bytes at offset %lu\n", + (unsigned) len, offset); + *poffset += len; + return len; +} + +static irqreturn_t kvm_ivshmem_interrupt (int irq, void *dev_instance) +{ + struct kvm_ivshmem_device * dev = dev_instance; + u16 status; + + if (unlikely(dev == NULL)) + return IRQ_NONE; + + status = readw(dev->regs + IntrStatus); + if (!status || (status == 0xFFFF)) + return IRQ_NONE; + + printk(KERN_INFO "KVM_IVSHMEM: Oops, no interrupts yet (status = 0x%04x)\n", + status); + + return IRQ_HANDLED; +} + +static int kvm_ivshmem_probe_device (struct pci_dev *pdev, + const struct pci_device_id * ent) { + + int result; + + printk("KVM_IVSHMEM: Probing for KVM_IVSHMEM Device\n"); + + result = pci_enable_device(pdev); + if (result) { + printk(KERN_ERR "Cannot probe KVM_IVSHMEM device %s: error %d\n", + pci_name(pdev), result); + return result; + } + + result = pci_request_regions(pdev, "kvm_ivshmem"); + if (result < 0) { + printk(KERN_ERR "KVM_IVSHMEM: cannot request regions\n"); + goto pci_disable; + } else printk(KERN_ERR "KVM_IVSHMEM: result is %d\n", result); + + kvm_ivshmem_dev.ioaddr = pci_resource_start(pdev, 1); + kvm_ivshmem_dev.ioaddr_size = pci_resource_len(pdev, 1); + + kvm_ivshmem_dev.base_addr = pci_iomap(pdev, 1, 0); + printk(KERN_INFO "KVM_IVSHMEM: iomap base = 0x%lu \n", + (unsigned long) kvm_ivshmem_dev.base_addr); + + if (!kvm_ivshmem_dev.base_addr) { + printk(KERN_ERR "KVM_IVSHMEM: cannot iomap region of size %d\n", + kvm_ivshmem_dev.ioaddr_size); + goto pci_release; + } + + printk(KERN_INFO "KVM_IVSHMEM: ioaddr = %x ioaddr_size = %d\n", + kvm_ivshmem_dev.ioaddr, kvm_ivshmem_dev.ioaddr_size); + + kvm_ivshmem_dev.regaddr = pci_resource_start(pdev, 0); + kvm_ivshmem_dev.reg_size = pci_resource_len(pdev, 0); + kvm_ivshmem_dev.regs = pci_iomap(pdev, 0, 0x100); + + if (!kvm_ivshmem_dev.regs) { + printk(KERN_ERR "KVM_IVSHMEM: cannot ioremap registers of size %d\n", + kvm_ivshmem_dev.reg_size); + goto reg_release; + } + + printk(KERN_INFO "KVM_IVSHMEM: irq = %u regaddr = %x reg_size = %d\n", + pdev->irq, kvm_ivshmem_dev.regaddr, kvm_ivshmem_dev.reg_size); + + if (request_irq(pdev->irq, kvm_ivshmem_interrupt, IRQF_SHARED, + "kvm_ivshmem", &kvm_ivshmem_dev)) { + printk(KERN_ERR "KVM_IVSHMEM: cannot get interrupt %d\n", pdev->irq); + } + + return 0; + + +reg_release: + pci_iounmap(pdev, kvm_ivshmem_dev.base_addr); +pci_release: + pci_release_regions(pdev); +pci_disable: + pci_disable_device(pdev); + return -EBUSY; + +} + +static void kvm_ivshmem_remove_device(struct pci_dev* pdev) +{ + + printk(KERN_INFO "Unregister kvm_ivshmem device.\n"); + free_irq(pdev->irq,&kvm_ivshmem_dev); + pci_iounmap(pdev, kvm_ivshmem_dev.regs); + pci_iounmap(pdev, kvm_ivshmem_dev.base_addr); + pci_release_regions(pdev); + pci_disable_device(pdev); + +} + +static void __exit kvm_ivshmem_cleanup_module (void) +{ + pci_unregister_driver (&kvm_ivshmem_pci_driver); + unregister_chrdev(device_major_nr, "kvm_ivshmem"); +} + +static int __init kvm_ivshmem_init_module (void) +{ + + int err = -ENOMEM; + + /* Register device node ops. */ + err = register_chrdev(0, "kvm_ivshmem", &kvm_ivshmem_ops); + if (err < 0) { + printk(KERN_ERR "Unable to register kvm_ivshmem device\n"); + return err; + } + device_major_nr = err; + printk("KVM_IVSHMEM: Major device number is: %d\n", device_major_nr); + kvm_ivshmem_dev.enabled=FALSE; + + err = pci_register_driver(&kvm_ivshmem_pci_driver); + if (err < 0) { + goto error; + } + + return 0; + +error: + unregister_chrdev(device_major_nr, "kvm_ivshmem"); + return err; +} + + +static int kvm_ivshmem_open(struct inode * inode, struct file * filp) +{ + + printk(KERN_INFO "Opening kvm_ivshmem device\n"); + + if (MINOR(inode->i_rdev) != KVM_IVSHMEM_DEVICE_MINOR_NUM) { + printk(KERN_INFO "minor number is %d\n", KVM_IVSHMEM_DEVICE_MINOR_NUM); + return -ENODEV; + } + + return 0; +} + +static int kvm_ivshmem_release(struct inode * inode, struct file * filp) +{ + + return 0; +} + +static int kvm_ivshmem_mmap(struct file *filp, struct vm_area_struct * vma) +{ + + unsigned long len; + unsigned long off; + unsigned long start; + + lock_kernel(); + + off = vma->vm_pgoff << PAGE_SHIFT; + start = kvm_ivshmem_dev.ioaddr; + + len=PAGE_ALIGN((start & ~PAGE_MASK) + kvm_ivshmem_dev.ioaddr_size); + start &= PAGE_MASK; + + printk(KERN_INFO "%lu - %lu + %lu\n",vma->vm_end ,vma->vm_start, off); + printk(KERN_INFO "%lu > %lu\n",(vma->vm_end - vma->vm_start + off), len); + + if ((vma->vm_end - vma->vm_start + off) > len) { + unlock_kernel(); + return -EINVAL; + } + + off += start; + vma->vm_pgoff = off >> PAGE_SHIFT; + + vma->vm_flags |= VM_SHARED|VM_RESERVED; + + if(io_remap_pfn_range(vma, vma->vm_start, + off >> PAGE_SHIFT, vma->vm_end - vma->vm_start, + vma->vm_page_prot)) + { + printk("mmap failed\n"); + unlock_kernel(); + return -ENXIO; + } + unlock_kernel(); + + return 0; +} + +module_init(kvm_ivshmem_init_module); +module_exit(kvm_ivshmem_cleanup_module); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Cam Macdonell "); +MODULE_DESCRIPTION("KVM inter-VM shared memory module"); +MODULE_VERSION("1.0");