From patchwork Sun Jun 20 12:55:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ojaswin Mujoo X-Patchwork-Id: 12333397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CCD4C48BDF for ; Sun, 20 Jun 2021 12:57:59 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 50F636113C for ; Sun, 20 Jun 2021 12:57:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 50F636113C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+9Hm8xOjVGH4l/79tGD2aFyf2kxwECXlczcJK8DKwNA=; b=BmN9+9WsyKmNw5 U/yYHuYfmRl9456i7RNYAoCKyVHJ07Wsq3UlXwbfYzdG5NQ46DaKli5jj01JV20tB7LnnETMZjCSD cDEgEk9FhAheR0tFbN/rjVSS9oYCU8pR2rJyVPKWdkXBazzl4PButSm4rx8LiGZqlgvTMb7XMjLjJ Jt/XaWf2ecCH/RYrsmFY5WAzUuFgwWUp3kBASwzYBaH0A3r8HO1UD7VPl/3wDaKfmObrVpkQvnTkS 2TATee1ZtyKrb0qywYTVbGIGZxMkRuvjSsA7N/N77lupcfa2k1WEtE1F4bZWtT5vcg6APWFscuuXK UGt0qQ2kq21bp4syulaQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1luwzc-00117Z-5C; Sun, 20 Jun 2021 12:56:04 +0000 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luwzY-001173-E8 for linux-arm-kernel@lists.infradead.org; Sun, 20 Jun 2021 12:56:02 +0000 Received: by mail-pg1-x530.google.com with SMTP id m2so11856084pgk.7 for ; Sun, 20 Jun 2021 05:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=iML3X+LebYbUqiIruyQrVisHl+E/xyjft7hYS7VDy7I=; b=e8tvsQftsTNLUtKuaA2EpXy9VCg5x+v8pEWsQ/vBG7LLU7qU1FsYYONTdHXmSt66as oETD9VuIshmh3GLbtbZNtSBkqP3axHMFKdntbXaTDPN2GABSQHGlVnctayHu9bJ/Wndx 4EJrVCuDsN7oeQYSmauOgkc+fLu9CfHQTuEYiWTVjkUVWhNR2TTyAZ08rdP11iUZ5LMw Y38HnQWd35H7LD2ug4IKwmbCf+MwMFHj/eL7M6Oa/EHPRDUA+IaoL1S9BJTU4EZU16up sy899nw5tadH62begMVgJMuESIg48eRN8OIGZxKj7G/LfuoHbU2dL0ZcJ9wI45IRXCY+ 3SGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iML3X+LebYbUqiIruyQrVisHl+E/xyjft7hYS7VDy7I=; b=FeOY4dib6XG4oHDkna9qYKxadsfRNsQqkww50ACoiubfLaZZIYQqsqop684NRgOsWO IKCa8XZAeg/6Sr03kSOfcRUEEJVLKGB0pVeMmbv3oYCIbSyq8Kvq2cZR+q9lEZfXvorE BQHD4cJzzoyDMfhwSwkKzCZ5TdQkS0J0Is7CWTb1VjhF1xLQv1/YrN8s7iY90kjRWmD2 ObNtGcg6bDhJgHeyu4Axhdy/MNORyJGEgzWhIApb6ckx2PiLUQVVtSlTJ4mXiWgIBBs1 LWDa6sUoMbfeFcfy9FU23i3m2tAsKKfao0pbCt3hZWxvRFHuoXHKzB++DSThNpODFLRx XpYg== X-Gm-Message-State: AOAM532rRhQvXnwZzLyA3cTLOOKaAilSbQjczzNae7s37mKxXZMRzVlD uvnAZi7aDSjRhOkxSBJ4J/4= X-Google-Smtp-Source: ABdhPJzkahBPxIIzFmiVpSwNGpcOXwxm5HsGFWd8NkHOWtAiXW9R+Z0u9Al8u/OwyA8MXNVk9HO+tQ== X-Received: by 2002:a62:1c86:0:b029:2fe:b583:6418 with SMTP id c128-20020a621c860000b02902feb5836418mr14801744pfc.23.1624193759733; Sun, 20 Jun 2021 05:55:59 -0700 (PDT) Received: from ojas ([122.177.154.120]) by smtp.gmail.com with ESMTPSA id co18sm3812154pjb.37.2021.06.20.05.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 05:55:59 -0700 (PDT) Date: Sun, 20 Jun 2021 18:25:46 +0530 From: Ojaswin Mujoo To: nsaenz@kernel.org Cc: gregkh@linuxfoundation.org, stefan.wahren@i2se.com, arnd@arndb.de, dan.carpenter@oracle.com, phil@raspberrypi.com, bcm-kernel-feedback-list@broadcom.com, linux-arm-kernel@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/5] staging: vchiq: Refactor vchiq cdev code Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210620_055600_535446_47C11C66 X-CRM114-Status: GOOD ( 19.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the vchiq cdev initialization code to its own function for better code organization. Call the initialization function during probe, thus shifting the whole cdev creation logic (which was earlier split in vchiq_probe() and vchiq_driver_init()) to vchiq_probe(). Signed-off-by: Ojaswin Mujoo --- .../interface/vchiq_arm/vchiq_arm.c | 137 +++++++++++------- 1 file changed, 87 insertions(+), 50 deletions(-) diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c index a936102dbc34..f2d8116913f2 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c @@ -2207,6 +2207,81 @@ vchiq_fops = { .read = vchiq_read }; +/** + * vchiq_register_chrdev - Register the char driver for vchiq + * and create the necessary class and + * device files in userspace. + * @parent The parent of the char device. + * + * Returns 0 on success else returns the error code. + */ +static int vchiq_register_chrdev(struct device *parent) +{ + struct device *vchiq_dev; + int ret; + + vchiq_class = class_create(THIS_MODULE, DEVICE_NAME); + if (IS_ERR(vchiq_class)) { + pr_err("Failed to create vchiq class\n"); + ret = PTR_ERR(vchiq_class); + goto error_exit; + } + + ret = alloc_chrdev_region(&vchiq_devid, 0, 1, DEVICE_NAME); + if (ret) { + pr_err("vchiq: Failed to allocate vchiq's chrdev region\n"); + goto alloc_region_error; + } + + cdev_init(&vchiq_cdev, &vchiq_fops); + vchiq_cdev.owner = THIS_MODULE; + ret = cdev_add(&vchiq_cdev, vchiq_devid, 1); + if (ret) { + vchiq_log_error(vchiq_arm_log_level, + "Unable to register vchiq char device"); + goto cdev_add_error; + } + + vchiq_dev = device_create(vchiq_class, parent, vchiq_devid, NULL, + DEVICE_NAME); + if (IS_ERR(vchiq_dev)) { + vchiq_log_error(vchiq_arm_log_level, + "Failed to create vchiq char device node"); + ret = PTR_ERR(vchiq_dev); + goto device_create_error; + } + + vchiq_log_info(vchiq_arm_log_level, + "vchiq char dev initialised successfully - device %d.%d", + MAJOR(vchiq_devid), MINOR(vchiq_devid)); + + return 0; + +device_create_error: + cdev_del(&vchiq_cdev); + +cdev_add_error: + unregister_chrdev_region(vchiq_devid, 1); + +alloc_region_error: + class_destroy(vchiq_class); + +error_exit: + return ret; +} + +/** + * vchiq_deregister_chrdev - Deregister and cleanup the vchiq char + * driver and device files + */ +static void vchiq_deregister_chrdev(void) +{ + device_destroy(vchiq_class, vchiq_devid); + cdev_del(&vchiq_cdev); + unregister_chrdev_region(vchiq_devid, 1); + class_destroy(vchiq_class); +} + /* * Autosuspend related functionality */ @@ -2740,7 +2815,6 @@ static int vchiq_probe(struct platform_device *pdev) struct device_node *fw_node; const struct of_device_id *of_id; struct vchiq_drvdata *drvdata; - struct device *vchiq_dev; int err; of_id = of_match_node(vchiq_of_match, pdev->dev.of_node); @@ -2766,28 +2840,18 @@ static int vchiq_probe(struct platform_device *pdev) if (err) goto failed_platform_init; - cdev_init(&vchiq_cdev, &vchiq_fops); - vchiq_cdev.owner = THIS_MODULE; - err = cdev_add(&vchiq_cdev, vchiq_devid, 1); - if (err) { - vchiq_log_error(vchiq_arm_log_level, - "Unable to register device"); - goto failed_platform_init; - } - - vchiq_dev = device_create(vchiq_class, &pdev->dev, vchiq_devid, NULL, - "vchiq"); - if (IS_ERR(vchiq_dev)) { - err = PTR_ERR(vchiq_dev); - goto failed_device_create; - } - vchiq_debugfs_init(); vchiq_log_info(vchiq_arm_log_level, - "vchiq: initialised - version %d (min %d), device %d.%d", - VCHIQ_VERSION, VCHIQ_VERSION_MIN, - MAJOR(vchiq_devid), MINOR(vchiq_devid)); + "vchiq: platform initialised - version %d (min %d)", + VCHIQ_VERSION, VCHIQ_VERSION_MIN); + + /* + * We don't handle error here since the function handles + * cleanup in cases of failure. Further, we can proceed + * even if this function fails. + */ + vchiq_register_chrdev(&pdev->dev); vcsm_cma = vchiq_register_child(pdev, "vcsm-cma"); bcm2835_codec = vchiq_register_child(pdev, "bcm2835-codec"); @@ -2797,10 +2861,8 @@ static int vchiq_probe(struct platform_device *pdev) return 0; -failed_device_create: - cdev_del(&vchiq_cdev); failed_platform_init: - vchiq_log_warning(vchiq_arm_log_level, "could not load vchiq"); + vchiq_log_warning(vchiq_arm_log_level, "could not initialize vchiq platform"); return err; } @@ -2812,8 +2874,7 @@ static int vchiq_remove(struct platform_device *pdev) platform_device_unregister(bcm2835_codec); platform_device_unregister(vcsm_cma); vchiq_debugfs_deinit(); - device_destroy(vchiq_class, vchiq_devid); - cdev_del(&vchiq_cdev); + vchiq_deregister_chrdev(); return 0; } @@ -2831,31 +2892,9 @@ static int __init vchiq_driver_init(void) { int ret; - vchiq_class = class_create(THIS_MODULE, DEVICE_NAME); - if (IS_ERR(vchiq_class)) { - pr_err("Failed to create vchiq class\n"); - return PTR_ERR(vchiq_class); - } - - ret = alloc_chrdev_region(&vchiq_devid, 0, 1, DEVICE_NAME); - if (ret) { - pr_err("Failed to allocate vchiq's chrdev region\n"); - goto class_destroy; - } - ret = platform_driver_register(&vchiq_driver); - if (ret) { + if (ret) pr_err("Failed to register vchiq driver\n"); - goto region_unregister; - } - - return 0; - -region_unregister: - unregister_chrdev_region(vchiq_devid, 1); - -class_destroy: - class_destroy(vchiq_class); return ret; } @@ -2864,8 +2903,6 @@ module_init(vchiq_driver_init); static void __exit vchiq_driver_exit(void) { platform_driver_unregister(&vchiq_driver); - unregister_chrdev_region(vchiq_devid, 1); - class_destroy(vchiq_class); } module_exit(vchiq_driver_exit); From patchwork Sun Jun 20 12:56:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ojaswin Mujoo X-Patchwork-Id: 12333399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FE71C48BDF for ; Sun, 20 Jun 2021 12:58:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D1FD2610CA for ; Sun, 20 Jun 2021 12:58:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1FD2610CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kNU2cQlQZ3zhtyEJiY7xS+PbnELRVxEEHFo74zSkvpQ=; b=IN9FKGrIXxPf7i Y2kMHlEETghU3tuj+sHtYOS0Btl99k+VN1VYnlqpOZWb0a14vf3/mEJL/GZemvX1grC2bdMMYvM/a 2qL+7TlXBAsA/Hpf0JJxadAL2VEwK4v0yOERoWAscIIxL+nok1M9bCVlnvTQL3SZeYziJ1yES/qy8 wkieqe81qosyjGlUd7NC+HSPP2TwpFPim7QgLEexd/CkgX7lGVlKacBZFufjf9iR9KEAterKYT1G4 zY7BfRSBxAwXbGW10qb7kRUrC3Ky6zhIVTldL6OKx2o6YArTXwj2Spue9qS6cBNgxeEBzsiqnzkdS OhcYVwhvbT/WrI24i5ug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux00-0011EO-5V; Sun, 20 Jun 2021 12:56:28 +0000 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1luwzu-0011C0-GO for linux-arm-kernel@lists.infradead.org; Sun, 20 Jun 2021 12:56:24 +0000 Received: by mail-pj1-x102a.google.com with SMTP id 22-20020a17090a0c16b0290164a5354ad0so10907347pjs.2 for ; Sun, 20 Jun 2021 05:56:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=8D0iPvS5LN4cQHCvsRANY4NqRMLIH6ZhqZliXmE6Nic=; b=j9swlja8fhJH2LV6AulKGsoAFeFWrhiQJcb7ptpSVB7J2hY1MWU+3O689NaBIUIejO 1nR3FpCmUajwkV6tX6dnq9l4WKOg/76S/dotDwnfH3ew9/ceDCHdANJdnJ8+TbksALwk FdTUX3fp9koZABvZlKBoEKySzcJobWrGXilBTu+ikjGkAd0plgzDdU+/+bniB9bGQWA/ TcJCIP7k001MsXzwUB1mQPSkhrUmlZSsZjtcpWjjfEzueIAsdbaSw2xyOedf4fUepDQ8 jltme5ZqY96Sj0n+AMb7cJhMMTxZu3Jfa3y472v7kxMRvVDbDGJwGsJjQKKWMF3tnvOZ vuIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=8D0iPvS5LN4cQHCvsRANY4NqRMLIH6ZhqZliXmE6Nic=; b=dAO8BSh67knKYFlqfCshFbhNvhcU4SOHSz1Fk4CjfAQVsnG6J1DH3irf2oi2lItjaf tzWeLQO/gyNbfpZXm6XumBwyKWsNxf2L5Wx3vgpR8028AqKL1ot7PpyZ3pAvgf/ZAryn 3Z09o3GyPe3x1xyn1ZRQi1NXgiUp2wsuI36NpcyMfo8l6Q4dJCLWmdSw75N5j+6Ke9hu /1N+EGHzstmk4yM+IDBNU0ffz3POxlWzveZVAFc0Yxp2uizdJaJk49ggcuLNd1rY+sSK 8tR77UnWTCa7GipbxNz58dtN4LygPZEB1DYkapVHC7TMtHGKcyY2P/zFUfYGDMKHO2Yv vRbw== X-Gm-Message-State: AOAM532uEGpzB7DrUpwvxhoLScwaBctwSoJSzI59WAT5u2zibO5BwxN8 PCtvJWEBMOcD4iXxpUzlRv8= X-Google-Smtp-Source: ABdhPJwsu0EpaSa59gJ+a7O4tSxTxzyQdDE74aLf0wJMj8QKckeTpU5yxCxDY4bBQTLl88/y0C4bOQ== X-Received: by 2002:a17:902:6b04:b029:10d:8c9e:5f56 with SMTP id o4-20020a1709026b04b029010d8c9e5f56mr13511814plk.8.1624193781798; Sun, 20 Jun 2021 05:56:21 -0700 (PDT) Received: from ojas ([122.177.154.120]) by smtp.gmail.com with ESMTPSA id 25sm13906585pgp.51.2021.06.20.05.56.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 05:56:21 -0700 (PDT) Date: Sun, 20 Jun 2021 18:26:10 +0530 From: Ojaswin Mujoo To: nsaenz@kernel.org Cc: gregkh@linuxfoundation.org, stefan.wahren@i2se.com, arnd@arndb.de, dan.carpenter@oracle.com, phil@raspberrypi.com, bcm-kernel-feedback-list@broadcom.com, linux-arm-kernel@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/5] staging: vchiq: Move certain declarations to vchiq_arm.h Message-ID: <7f4e8cfe00db59204b41d5a594143838bf502634.1624185152.git.ojaswin98@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210620_055622_600041_6296274C X-CRM114-Status: GOOD ( 16.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move certain declarations from vchiq_arm.c to vchiq_arm.h to allow code sharing. This will be useful when we eventually separate the vchiq char driver code from platform code, into its own file. Signed-off-by: Ojaswin Mujoo --- .../interface/vchiq_arm/vchiq_arm.c | 66 ++--------------- .../interface/vchiq_arm/vchiq_arm.h | 70 +++++++++++++++++++ 2 files changed, 75 insertions(+), 61 deletions(-) diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c index f2d8116913f2..8b18f9642e6c 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c @@ -38,12 +38,6 @@ #undef MODULE_PARAM_PREFIX #define MODULE_PARAM_PREFIX DEVICE_NAME "." -/* Some per-instance constants */ -#define MAX_COMPLETIONS 128 -#define MAX_SERVICES 64 -#define MAX_ELEMENTS 8 -#define MSG_QUEUE_SIZE 128 - #define KEEPALIVE_VER 1 #define KEEPALIVE_VER_MIN KEEPALIVE_VER @@ -51,62 +45,12 @@ int vchiq_arm_log_level = VCHIQ_LOG_DEFAULT; int vchiq_susp_log_level = VCHIQ_LOG_ERROR; -struct user_service { - struct vchiq_service *service; - void __user *userdata; - struct vchiq_instance *instance; - char is_vchi; - char dequeue_pending; - char close_pending; - int message_available_pos; - int msg_insert; - int msg_remove; - struct completion insert_event; - struct completion remove_event; - struct completion close_event; - struct vchiq_header *msg_queue[MSG_QUEUE_SIZE]; -}; - -struct bulk_waiter_node { - struct bulk_waiter bulk_waiter; - int pid; - struct list_head list; -}; - -struct vchiq_instance { - struct vchiq_state *state; - struct vchiq_completion_data_kernel completions[MAX_COMPLETIONS]; - int completion_insert; - int completion_remove; - struct completion insert_event; - struct completion remove_event; - struct mutex completion_mutex; - - int connected; - int closing; - int pid; - int mark; - int use_close_delivered; - int trace; - - struct list_head bulk_waiter_list; - struct mutex bulk_waiter_list_mutex; - - struct vchiq_debugfs_node debugfs_node; -}; - -struct dump_context { - char __user *buf; - size_t actual; - size_t space; - loff_t offset; -}; +DEFINE_SPINLOCK(msg_queue_spinlock); +struct vchiq_state g_state; static struct cdev vchiq_cdev; static dev_t vchiq_devid; -static struct vchiq_state g_state; static struct class *vchiq_class; -static DEFINE_SPINLOCK(msg_queue_spinlock); static struct platform_device *bcm2835_camera; static struct platform_device *bcm2835_audio; static struct platform_device *bcm2835_codec; @@ -587,7 +531,7 @@ add_completion(struct vchiq_instance *instance, enum vchiq_reason reason, * ***************************************************************************/ -static enum vchiq_status +enum vchiq_status service_callback(enum vchiq_reason reason, struct vchiq_header *header, unsigned int handle, void *bulk_userdata) { @@ -2215,7 +2159,7 @@ vchiq_fops = { * * Returns 0 on success else returns the error code. */ -static int vchiq_register_chrdev(struct device *parent) +int vchiq_register_chrdev(struct device *parent) { struct device *vchiq_dev; int ret; @@ -2274,7 +2218,7 @@ static int vchiq_register_chrdev(struct device *parent) * vchiq_deregister_chrdev - Deregister and cleanup the vchiq char * driver and device files */ -static void vchiq_deregister_chrdev(void) +void vchiq_deregister_chrdev(void) { device_destroy(vchiq_class, vchiq_devid); cdev_del(&vchiq_cdev); diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h index f8b1c005af62..bc9af1a0c764 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h @@ -14,6 +14,12 @@ #include "vchiq_core.h" #include "vchiq_debugfs.h" +/* Some per-instance constants */ +#define MAX_COMPLETIONS 128 +#define MAX_SERVICES 64 +#define MAX_ELEMENTS 8 +#define MSG_QUEUE_SIZE 128 + enum USE_TYPE_E { USE_TYPE_SERVICE, USE_TYPE_VCHIQ @@ -56,9 +62,63 @@ struct vchiq_drvdata { struct rpi_firmware *fw; }; +struct user_service { + struct vchiq_service *service; + void __user *userdata; + struct vchiq_instance *instance; + char is_vchi; + char dequeue_pending; + char close_pending; + int message_available_pos; + int msg_insert; + int msg_remove; + struct completion insert_event; + struct completion remove_event; + struct completion close_event; + struct vchiq_header *msg_queue[MSG_QUEUE_SIZE]; +}; + +struct bulk_waiter_node { + struct bulk_waiter bulk_waiter; + int pid; + struct list_head list; +}; + +struct vchiq_instance { + struct vchiq_state *state; + struct vchiq_completion_data_kernel completions[MAX_COMPLETIONS]; + int completion_insert; + int completion_remove; + struct completion insert_event; + struct completion remove_event; + struct mutex completion_mutex; + + int connected; + int closing; + int pid; + int mark; + int use_close_delivered; + int trace; + + struct list_head bulk_waiter_list; + struct mutex bulk_waiter_list_mutex; + + struct vchiq_debugfs_node debugfs_node; +}; + +struct dump_context { + char __user *buf; + size_t actual; + size_t space; + loff_t offset; +}; + extern int vchiq_arm_log_level; extern int vchiq_susp_log_level; +extern spinlock_t msg_queue_spinlock; +extern struct vchiq_state g_state; + int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state); @@ -112,4 +172,14 @@ vchiq_instance_get_trace(struct vchiq_instance *instance); extern void vchiq_instance_set_trace(struct vchiq_instance *instance, int trace); +extern void +vchiq_deregister_chrdev(void); + +extern int +vchiq_register_chrdev(struct device *parent); + +extern enum vchiq_status +service_callback(enum vchiq_reason reason, struct vchiq_header *header, + unsigned int handle, void *bulk_userdata); + #endif /* VCHIQ_ARM_H */ From patchwork Sun Jun 20 12:56:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ojaswin Mujoo X-Patchwork-Id: 12333403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FF81C49EA2 for ; Sun, 20 Jun 2021 12:59:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E7E9A610CA for ; Sun, 20 Jun 2021 12:59:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7E9A610CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6l6VaH8skj6Peowt7fGCctIdPrQk+zZohbGh2xNoWw8=; b=xgAFHECIR3VPCs dKMtvmwW4nV/sgVdIlpBqkgFFLDifRJwRttnY2bJ8NDrGpUjvcr5XzQ4vcCZBTeA/A61xp9cANgKV +n2qvzHQ4Jy93vyWVc4hwFMiRJLA0gDkOxOGizX3bEgCaMgrqABTfbSu5Xs53v+QAxv6+2ErYodAG gyCKN4/M4Fm8jvNws7BHxIqZgSfomJynBweg/B6COG0irtsNTomE/QoeMZWEWC2dKtRfoqXVsz8AO 9zPeXKCfrraMEFZlIlgoB41WL/zOVu23dV1p1J/fukB7hyETxQUJDLizJDnSLBzewPCGZf+zbc9Ng ILeWi6DdrxHuV2SAJmzg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux0S-0011NH-0K; Sun, 20 Jun 2021 12:56:56 +0000 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux0I-0011KO-Rk for linux-arm-kernel@lists.infradead.org; Sun, 20 Jun 2021 12:56:52 +0000 Received: by mail-pj1-x1034.google.com with SMTP id 22-20020a17090a0c16b0290164a5354ad0so10907676pjs.2 for ; Sun, 20 Jun 2021 05:56:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=1z+nGpHYBhk7tupfmbSQQC8fjcSluiErB0/J1uFAnY4=; b=btZgEg2p8Ye20y3MjowGpiVUdLveKEeAAo9dWvtp3O/JrHJkq7n9PH1LrUr0oN4CUo MP2ITaFcWN+Sie4MonnWgxIqo5LDxG6NkLJTZaDzTrDL+1tdO2ZyOsDnAzU1Z5hGptd8 nQ3juWEwvSn+zOTr8+GsSQjecxhUQivsKTZGSILGMrzdXMrLCZbdIena03G1PI5pfI3L J9yGLs44vRNFWGHw39ekFyAiOdUSrvL2ZRJhaNXZLjAyyGQmc5bE5utzn60MNgkhl3kA Z98pBxA4QcNJQsy5kSwWqK2uBFRkjZGvtxdntlL4r6kP6cyM8utI6wAmBseuMHybDT/j SeGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=1z+nGpHYBhk7tupfmbSQQC8fjcSluiErB0/J1uFAnY4=; b=TUR1JtI6iohfihSeIJ/1JOfgdIOjikbYnxgGZt9vYviQRmqbXTcAJdWrVlYtvplza8 OX5czBDrcVoxuiloV9BAKytEYhr9sCPZyHW6N+AhmRdbgRMWxim02YWJj0mLAZOjKeox A4zKQDewhr/iVh06pFTXJRAIK3DW4Rbagtu/9pxv3QfpH5rvsic5NEjlHk+Oxfm9/RZU X2+EHspqwBGOoKTLtQJo8xGEC+sHlJZ+OtquvQSnOVXsrNwkOlrVq+xgUd4KB7dkcDAi 086igBJX5nLYEwgpkG842/ihb/boOb0ettloq+C/f4iMV2h5549CukCbmnHT5vWGCeXQ rm4w== X-Gm-Message-State: AOAM5333ef/B6c6zKdXOERAJF29xOo3VWEktYaYpermgWk/3A1yZx+tP SG+l6EuRp/yedMFke6b/wy4= X-Google-Smtp-Source: ABdhPJyCunSmhBjAEuK8YT/V8jSRMB1dn11iJh4/O9p/sc5dEuT96kYCYqX0qkeds8Iqh/+uc81o6Q== X-Received: by 2002:a17:902:b585:b029:f6:5cd5:f128 with SMTP id a5-20020a170902b585b02900f65cd5f128mr13159621pls.43.1624193805402; Sun, 20 Jun 2021 05:56:45 -0700 (PDT) Received: from ojas ([122.177.154.120]) by smtp.gmail.com with ESMTPSA id h18sm13773114pgl.87.2021.06.20.05.56.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 05:56:45 -0700 (PDT) Date: Sun, 20 Jun 2021 18:26:34 +0530 From: Ojaswin Mujoo To: nsaenz@kernel.org Cc: gregkh@linuxfoundation.org, stefan.wahren@i2se.com, arnd@arndb.de, dan.carpenter@oracle.com, phil@raspberrypi.com, bcm-kernel-feedback-list@broadcom.com, linux-arm-kernel@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/5] staging: vchiq: Move vchiq char driver to its own file Message-ID: <8cd9b5899f82eaf2efdc9caa8d07f719a592a94d.1624185152.git.ojaswin98@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210620_055647_196266_F2AF5440 X-CRM114-Status: GOOD ( 29.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Split the initialization code of vchiq char driver and device files from that of vchiq platform. The char driver code now resides in vchiq_dev.c and the platform code resides in the original vchiq_arm.c file. This commit focuses on separating the code into different files while maintaining the same functionality. It does not completely decouple them as the cdev init code is still called from the platform's vchiq_probe() function. Signed-off-by: Ojaswin Mujoo --- drivers/staging/vc04_services/Makefile | 1 + .../interface/vchiq_arm/vchiq_arm.c | 1675 ++--------------- .../interface/vchiq_arm/vchiq_dev.c | 1488 +++++++++++++++ 3 files changed, 1598 insertions(+), 1566 deletions(-) create mode 100644 drivers/staging/vc04_services/interface/vchiq_arm/vchiq_dev.c diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile index fc42251ec5ef..700cd62fe346 100644 --- a/drivers/staging/vc04_services/Makefile +++ b/drivers/staging/vc04_services/Makefile @@ -7,6 +7,7 @@ vchiq-objs := \ interface/vchiq_arm/vchiq_2835_arm.o \ interface/vchiq_arm/vchiq_debugfs.o \ interface/vchiq_arm/vchiq_connected.o \ + interface/vchiq_arm/vchiq_dev.o \ obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/ obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/ diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c index 8b18f9642e6c..06a19d6e2674 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c @@ -48,9 +48,6 @@ int vchiq_susp_log_level = VCHIQ_LOG_ERROR; DEFINE_SPINLOCK(msg_queue_spinlock); struct vchiq_state g_state; -static struct cdev vchiq_cdev; -static dev_t vchiq_devid; -static struct class *vchiq_class; static struct platform_device *bcm2835_camera; static struct platform_device *bcm2835_audio; static struct platform_device *bcm2835_codec; @@ -70,30 +67,6 @@ static struct vchiq_drvdata bcm2711_drvdata = { .use_36bit_addrs = true, }; -static const char *const ioctl_names[] = { - "CONNECT", - "SHUTDOWN", - "CREATE_SERVICE", - "REMOVE_SERVICE", - "QUEUE_MESSAGE", - "QUEUE_BULK_TRANSMIT", - "QUEUE_BULK_RECEIVE", - "AWAIT_COMPLETION", - "DEQUEUE_MESSAGE", - "GET_CLIENT_ID", - "GET_CONFIG", - "CLOSE_SERVICE", - "USE_SERVICE", - "RELEASE_SERVICE", - "SET_SERVICE_OPTION", - "DUMP_PHYS_MEM", - "LIB_VERSION", - "CLOSE_DELIVERED" -}; - -vchiq_static_assert(ARRAY_SIZE(ioctl_names) == - (VCHIQ_IOC_MAX + 1)); - static enum vchiq_status vchiq_blocking_bulk_transfer(unsigned int handle, void *data, unsigned int size, enum vchiq_bulk_dir dir); @@ -640,1592 +613,162 @@ service_callback(enum vchiq_reason reason, struct vchiq_header *header, /**************************************************************************** * -* user_service_free -* -***************************************************************************/ -static void -user_service_free(void *userdata) -{ - kfree(userdata); -} - -/**************************************************************************** -* -* close_delivered +* vchiq_dump * ***************************************************************************/ -static void close_delivered(struct user_service *user_service) -{ - vchiq_log_info(vchiq_arm_log_level, - "%s(handle=%x)", - __func__, user_service->service->handle); - - if (user_service->close_pending) { - /* Allow the underlying service to be culled */ - unlock_service(user_service->service); - - /* Wake the user-thread blocked in close_ or remove_service */ - complete(&user_service->close_event); - - user_service->close_pending = 0; - } -} - -struct vchiq_io_copy_callback_context { - struct vchiq_element *element; - size_t element_offset; - unsigned long elements_to_go; -}; -static ssize_t vchiq_ioc_copy_element_data(void *context, void *dest, - size_t offset, size_t maxsize) +int vchiq_dump(void *dump_context, const char *str, int len) { - struct vchiq_io_copy_callback_context *cc = context; - size_t total_bytes_copied = 0; - size_t bytes_this_round; - - while (total_bytes_copied < maxsize) { - if (!cc->elements_to_go) - return total_bytes_copied; - - if (!cc->element->size) { - cc->elements_to_go--; - cc->element++; - cc->element_offset = 0; - continue; - } - - bytes_this_round = min(cc->element->size - cc->element_offset, - maxsize - total_bytes_copied); + struct dump_context *context = (struct dump_context *)dump_context; + int copy_bytes; - if (copy_from_user(dest + total_bytes_copied, - cc->element->data + cc->element_offset, - bytes_this_round)) - return -EFAULT; + if (context->actual >= context->space) + return 0; - cc->element_offset += bytes_this_round; - total_bytes_copied += bytes_this_round; + if (context->offset > 0) { + int skip_bytes = min_t(int, len, context->offset); - if (cc->element_offset == cc->element->size) { - cc->elements_to_go--; - cc->element++; - cc->element_offset = 0; - } + str += skip_bytes; + len -= skip_bytes; + context->offset -= skip_bytes; + if (context->offset > 0) + return 0; } + copy_bytes = min_t(int, len, context->space - context->actual); + if (copy_bytes == 0) + return 0; + if (copy_to_user(context->buf + context->actual, str, + copy_bytes)) + return -EFAULT; + context->actual += copy_bytes; + len -= copy_bytes; - return maxsize; -} - -/************************************************************************** - * - * vchiq_ioc_queue_message - * - **************************************************************************/ -static int -vchiq_ioc_queue_message(unsigned int handle, - struct vchiq_element *elements, - unsigned long count) -{ - struct vchiq_io_copy_callback_context context; - enum vchiq_status status = VCHIQ_SUCCESS; - unsigned long i; - size_t total_size = 0; - - context.element = elements; - context.element_offset = 0; - context.elements_to_go = count; + /* + * If the terminating NUL is included in the length, then it + * marks the end of a line and should be replaced with a + * carriage return. + */ + if ((len == 0) && (str[copy_bytes - 1] == '\0')) { + char cr = '\n'; - for (i = 0; i < count; i++) { - if (!elements[i].data && elements[i].size != 0) + if (copy_to_user(context->buf + context->actual - 1, + &cr, 1)) return -EFAULT; - - total_size += elements[i].size; } - - status = vchiq_queue_message(handle, vchiq_ioc_copy_element_data, - &context, total_size); - - if (status == VCHIQ_ERROR) - return -EIO; - else if (status == VCHIQ_RETRY) - return -EINTR; return 0; } -static int vchiq_ioc_create_service(struct vchiq_instance *instance, - struct vchiq_create_service *args) -{ - struct user_service *user_service = NULL; - struct vchiq_service *service; - enum vchiq_status status = VCHIQ_SUCCESS; - struct vchiq_service_params_kernel params; - int srvstate; - - user_service = kmalloc(sizeof(*user_service), GFP_KERNEL); - if (!user_service) - return -ENOMEM; - - if (args->is_open) { - if (!instance->connected) { - kfree(user_service); - return -ENOTCONN; - } - srvstate = VCHIQ_SRVSTATE_OPENING; - } else { - srvstate = instance->connected ? - VCHIQ_SRVSTATE_LISTENING : VCHIQ_SRVSTATE_HIDDEN; - } - - params = (struct vchiq_service_params_kernel) { - .fourcc = args->params.fourcc, - .callback = service_callback, - .userdata = user_service, - .version = args->params.version, - .version_min = args->params.version_min, - }; - service = vchiq_add_service_internal(instance->state, ¶ms, - srvstate, instance, - user_service_free); - if (!service) { - kfree(user_service); - return -EEXIST; - } - - user_service->service = service; - user_service->userdata = args->params.userdata; - user_service->instance = instance; - user_service->is_vchi = (args->is_vchi != 0); - user_service->dequeue_pending = 0; - user_service->close_pending = 0; - user_service->message_available_pos = instance->completion_remove - 1; - user_service->msg_insert = 0; - user_service->msg_remove = 0; - init_completion(&user_service->insert_event); - init_completion(&user_service->remove_event); - init_completion(&user_service->close_event); - - if (args->is_open) { - status = vchiq_open_service_internal(service, instance->pid); - if (status != VCHIQ_SUCCESS) { - vchiq_remove_service(service->handle); - return (status == VCHIQ_RETRY) ? - -EINTR : -EIO; - } - } - args->handle = service->handle; - - return 0; -} +/**************************************************************************** +* +* vchiq_dump_platform_instance_state +* +***************************************************************************/ -static int vchiq_ioc_dequeue_message(struct vchiq_instance *instance, - struct vchiq_dequeue_message *args) +int vchiq_dump_platform_instances(void *dump_context) { - struct user_service *user_service; - struct vchiq_service *service; - struct vchiq_header *header; - int ret; - - DEBUG_INITIALISE(g_state.local) - DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); - service = find_service_for_instance(instance, args->handle); - if (!service) - return -EINVAL; + struct vchiq_state *state = vchiq_get_state(); + char buf[80]; + int len; + int i; - user_service = (struct user_service *)service->base.userdata; - if (user_service->is_vchi == 0) { - ret = -EINVAL; - goto out; - } + /* There is no list of instances, so instead scan all services, + marking those that have been dumped. */ - spin_lock(&msg_queue_spinlock); - if (user_service->msg_remove == user_service->msg_insert) { - if (!args->blocking) { - spin_unlock(&msg_queue_spinlock); - DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); - ret = -EWOULDBLOCK; - goto out; - } - user_service->dequeue_pending = 1; - ret = 0; - do { - spin_unlock(&msg_queue_spinlock); - DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); - if (wait_for_completion_interruptible( - &user_service->insert_event)) { - vchiq_log_info(vchiq_arm_log_level, - "DEQUEUE_MESSAGE interrupted"); - ret = -EINTR; - break; - } - spin_lock(&msg_queue_spinlock); - } while (user_service->msg_remove == - user_service->msg_insert); + rcu_read_lock(); + for (i = 0; i < state->unused_service; i++) { + struct vchiq_service *service; + struct vchiq_instance *instance; - if (ret) - goto out; - } + service = rcu_dereference(state->services[i]); + if (!service || service->base.callback != service_callback) + continue; - BUG_ON((int)(user_service->msg_insert - - user_service->msg_remove) < 0); - - header = user_service->msg_queue[user_service->msg_remove & - (MSG_QUEUE_SIZE - 1)]; - user_service->msg_remove++; - spin_unlock(&msg_queue_spinlock); - - complete(&user_service->remove_event); - if (!header) { - ret = -ENOTCONN; - } else if (header->size <= args->bufsize) { - /* Copy to user space if msgbuf is not NULL */ - if (!args->buf || (copy_to_user(args->buf, - header->data, header->size) == 0)) { - ret = header->size; - vchiq_release_message(service->handle, header); - } else - ret = -EFAULT; - } else { - vchiq_log_error(vchiq_arm_log_level, - "header %pK: bufsize %x < size %x", - header, args->bufsize, header->size); - WARN(1, "invalid size\n"); - ret = -EMSGSIZE; + instance = service->instance; + if (instance) + instance->mark = 0; } - DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); -out: - unlock_service(service); - return ret; -} - -static int vchiq_irq_queue_bulk_tx_rx(struct vchiq_instance *instance, - struct vchiq_queue_bulk_transfer *args, - enum vchiq_bulk_dir dir, - enum vchiq_bulk_mode __user *mode) -{ - struct vchiq_service *service; - struct bulk_waiter_node *waiter = NULL; - bool found = false; - void *userdata; - int status = 0; - int ret; - - service = find_service_for_instance(instance, args->handle); - if (!service) - return -EINVAL; + rcu_read_unlock(); - if (args->mode == VCHIQ_BULK_MODE_BLOCKING) { - waiter = kzalloc(sizeof(struct bulk_waiter_node), - GFP_KERNEL); - if (!waiter) { - ret = -ENOMEM; - goto out; - } + for (i = 0; i < state->unused_service; i++) { + struct vchiq_service *service; + struct vchiq_instance *instance; + int err; - userdata = &waiter->bulk_waiter; - } else if (args->mode == VCHIQ_BULK_MODE_WAITING) { - mutex_lock(&instance->bulk_waiter_list_mutex); - list_for_each_entry(waiter, &instance->bulk_waiter_list, - list) { - if (waiter->pid == current->pid) { - list_del(&waiter->list); - found = true; - break; - } - } - mutex_unlock(&instance->bulk_waiter_list_mutex); - if (!found) { - vchiq_log_error(vchiq_arm_log_level, - "no bulk_waiter found for pid %d", - current->pid); - ret = -ESRCH; - goto out; + rcu_read_lock(); + service = rcu_dereference(state->services[i]); + if (!service || service->base.callback != service_callback) { + rcu_read_unlock(); + continue; } - vchiq_log_info(vchiq_arm_log_level, - "found bulk_waiter %pK for pid %d", waiter, - current->pid); - userdata = &waiter->bulk_waiter; - } else { - userdata = args->userdata; - } - - /* - * FIXME address space mismatch: - * args->data may be interpreted as a kernel pointer - * in create_pagelist() called from vchiq_bulk_transfer(), - * accessing kernel data instead of user space, based on the - * address. - */ - status = vchiq_bulk_transfer(args->handle, NULL, args->data, args->size, - userdata, args->mode, dir); - - if (!waiter) { - ret = 0; - goto out; - } - if ((status != VCHIQ_RETRY) || fatal_signal_pending(current) || - !waiter->bulk_waiter.bulk) { - if (waiter->bulk_waiter.bulk) { - /* Cancel the signal when the transfer - ** completes. */ - spin_lock(&bulk_waiter_spinlock); - waiter->bulk_waiter.bulk->userdata = NULL; - spin_unlock(&bulk_waiter_spinlock); + instance = service->instance; + if (!instance || instance->mark) { + rcu_read_unlock(); + continue; } - kfree(waiter); - ret = 0; - } else { - const enum vchiq_bulk_mode mode_waiting = - VCHIQ_BULK_MODE_WAITING; - waiter->pid = current->pid; - mutex_lock(&instance->bulk_waiter_list_mutex); - list_add(&waiter->list, &instance->bulk_waiter_list); - mutex_unlock(&instance->bulk_waiter_list_mutex); - vchiq_log_info(vchiq_arm_log_level, - "saved bulk_waiter %pK for pid %d", - waiter, current->pid); + rcu_read_unlock(); - ret = put_user(mode_waiting, mode); + len = snprintf(buf, sizeof(buf), + "Instance %pK: pid %d,%s completions %d/%d", + instance, instance->pid, + instance->connected ? " connected, " : + "", + instance->completion_insert - + instance->completion_remove, + MAX_COMPLETIONS); + err = vchiq_dump(dump_context, buf, len + 1); + if (err) + return err; + instance->mark = 1; } -out: - unlock_service(service); - if (ret) - return ret; - else if (status == VCHIQ_ERROR) - return -EIO; - else if (status == VCHIQ_RETRY) - return -EINTR; return 0; } -/* read a user pointer value from an array pointers in user space */ -static inline int vchiq_get_user_ptr(void __user **buf, void __user *ubuf, int index) -{ - int ret; +/**************************************************************************** +* +* vchiq_dump_platform_service_state +* +***************************************************************************/ - if (in_compat_syscall()) { - compat_uptr_t ptr32; - compat_uptr_t __user *uptr = ubuf; - ret = get_user(ptr32, uptr + index); - *buf = compat_ptr(ptr32); - } else { - uintptr_t ptr, __user *uptr = ubuf; - ret = get_user(ptr, uptr + index); - *buf = (void __user *)ptr; - } +int vchiq_dump_platform_service_state(void *dump_context, + struct vchiq_service *service) +{ + struct user_service *user_service = + (struct user_service *)service->base.userdata; + char buf[80]; + int len; - return ret; -} + len = scnprintf(buf, sizeof(buf), " instance %pK", service->instance); -struct vchiq_completion_data32 { - enum vchiq_reason reason; - compat_uptr_t header; - compat_uptr_t service_userdata; - compat_uptr_t bulk_userdata; -}; + if ((service->base.callback == service_callback) && + user_service->is_vchi) { + len += scnprintf(buf + len, sizeof(buf) - len, + ", %d/%d messages", + user_service->msg_insert - user_service->msg_remove, + MSG_QUEUE_SIZE); -static int vchiq_put_completion(struct vchiq_completion_data __user *buf, - struct vchiq_completion_data *completion, - int index) -{ - struct vchiq_completion_data32 __user *buf32 = (void __user *)buf; - - if (in_compat_syscall()) { - struct vchiq_completion_data32 tmp = { - .reason = completion->reason, - .header = ptr_to_compat(completion->header), - .service_userdata = ptr_to_compat(completion->service_userdata), - .bulk_userdata = ptr_to_compat(completion->bulk_userdata), - }; - if (copy_to_user(&buf32[index], &tmp, sizeof(tmp))) - return -EFAULT; - } else { - if (copy_to_user(&buf[index], completion, sizeof(*completion))) - return -EFAULT; + if (user_service->dequeue_pending) + len += scnprintf(buf + len, sizeof(buf) - len, + " (dequeue pending)"); } - return 0; + return vchiq_dump(dump_context, buf, len + 1); } -static int vchiq_ioc_await_completion(struct vchiq_instance *instance, - struct vchiq_await_completion *args, - int __user *msgbufcountp) +struct vchiq_state * +vchiq_get_state(void) { - int msgbufcount; - int remove; - int ret; - - DEBUG_INITIALISE(g_state.local) - - DEBUG_TRACE(AWAIT_COMPLETION_LINE); - if (!instance->connected) { - return -ENOTCONN; - } - - mutex_lock(&instance->completion_mutex); - - DEBUG_TRACE(AWAIT_COMPLETION_LINE); - while ((instance->completion_remove == - instance->completion_insert) - && !instance->closing) { - int rc; - - DEBUG_TRACE(AWAIT_COMPLETION_LINE); - mutex_unlock(&instance->completion_mutex); - rc = wait_for_completion_interruptible( - &instance->insert_event); - mutex_lock(&instance->completion_mutex); - if (rc) { - DEBUG_TRACE(AWAIT_COMPLETION_LINE); - vchiq_log_info(vchiq_arm_log_level, - "AWAIT_COMPLETION interrupted"); - ret = -EINTR; - goto out; - } - } - DEBUG_TRACE(AWAIT_COMPLETION_LINE); - - msgbufcount = args->msgbufcount; - remove = instance->completion_remove; - - for (ret = 0; ret < args->count; ret++) { - struct vchiq_completion_data_kernel *completion; - struct vchiq_completion_data user_completion; - struct vchiq_service *service; - struct user_service *user_service; - struct vchiq_header *header; - - if (remove == instance->completion_insert) - break; - - completion = &instance->completions[ - remove & (MAX_COMPLETIONS - 1)]; - - /* - * A read memory barrier is needed to stop - * prefetch of a stale completion record - */ - rmb(); - - service = completion->service_userdata; - user_service = service->base.userdata; - - memset(&user_completion, 0, sizeof(user_completion)); - user_completion = (struct vchiq_completion_data) { - .reason = completion->reason, - .service_userdata = user_service->userdata, - }; - - header = completion->header; - if (header) { - void __user *msgbuf; - int msglen; - - msglen = header->size + sizeof(struct vchiq_header); - /* This must be a VCHIQ-style service */ - if (args->msgbufsize < msglen) { - vchiq_log_error(vchiq_arm_log_level, - "header %pK: msgbufsize %x < msglen %x", - header, args->msgbufsize, msglen); - WARN(1, "invalid message size\n"); - if (ret == 0) - ret = -EMSGSIZE; - break; - } - if (msgbufcount <= 0) - /* Stall here for lack of a - ** buffer for the message. */ - break; - /* Get the pointer from user space */ - msgbufcount--; - if (vchiq_get_user_ptr(&msgbuf, args->msgbufs, - msgbufcount)) { - if (ret == 0) - ret = -EFAULT; - break; - } - - /* Copy the message to user space */ - if (copy_to_user(msgbuf, header, msglen)) { - if (ret == 0) - ret = -EFAULT; - break; - } - /* Now it has been copied, the message - ** can be released. */ - vchiq_release_message(service->handle, header); - - /* The completion must point to the - ** msgbuf. */ - user_completion.header = msgbuf; - } - - if ((completion->reason == VCHIQ_SERVICE_CLOSED) && - !instance->use_close_delivered) - unlock_service(service); - - /* - * FIXME: address space mismatch, does bulk_userdata - * actually point to user or kernel memory? - */ - user_completion.bulk_userdata = completion->bulk_userdata; - - if (vchiq_put_completion(args->buf, &user_completion, ret)) { - if (ret == 0) - ret = -EFAULT; - break; - } - - /* - * Ensure that the above copy has completed - * before advancing the remove pointer. - */ - mb(); - remove++; - instance->completion_remove = remove; - } - - if (msgbufcount != args->msgbufcount) { - if (put_user(msgbufcount, msgbufcountp)) - ret = -EFAULT; - } -out: - if (ret) - complete(&instance->remove_event); - mutex_unlock(&instance->completion_mutex); - DEBUG_TRACE(AWAIT_COMPLETION_LINE); - - return ret; -} - -/**************************************************************************** -* -* vchiq_ioctl -* -***************************************************************************/ -static long -vchiq_ioctl(struct file *file, unsigned int cmd, unsigned long arg) -{ - struct vchiq_instance *instance = file->private_data; - enum vchiq_status status = VCHIQ_SUCCESS; - struct vchiq_service *service = NULL; - long ret = 0; - int i, rc; - - vchiq_log_trace(vchiq_arm_log_level, - "%s - instance %pK, cmd %s, arg %lx", - __func__, instance, - ((_IOC_TYPE(cmd) == VCHIQ_IOC_MAGIC) && - (_IOC_NR(cmd) <= VCHIQ_IOC_MAX)) ? - ioctl_names[_IOC_NR(cmd)] : "", arg); - - switch (cmd) { - case VCHIQ_IOC_SHUTDOWN: - if (!instance->connected) - break; - - /* Remove all services */ - i = 0; - while ((service = next_service_by_instance(instance->state, - instance, &i))) { - status = vchiq_remove_service(service->handle); - unlock_service(service); - if (status != VCHIQ_SUCCESS) - break; - } - service = NULL; - - if (status == VCHIQ_SUCCESS) { - /* Wake the completion thread and ask it to exit */ - instance->closing = 1; - complete(&instance->insert_event); - } - - break; - - case VCHIQ_IOC_CONNECT: - if (instance->connected) { - ret = -EINVAL; - break; - } - rc = mutex_lock_killable(&instance->state->mutex); - if (rc) { - vchiq_log_error(vchiq_arm_log_level, - "vchiq: connect: could not lock mutex for " - "state %d: %d", - instance->state->id, rc); - ret = -EINTR; - break; - } - status = vchiq_connect_internal(instance->state, instance); - mutex_unlock(&instance->state->mutex); - - if (status == VCHIQ_SUCCESS) - instance->connected = 1; - else - vchiq_log_error(vchiq_arm_log_level, - "vchiq: could not connect: %d", status); - break; - - case VCHIQ_IOC_CREATE_SERVICE: { - struct vchiq_create_service __user *argp; - struct vchiq_create_service args; - - argp = (void __user *)arg; - if (copy_from_user(&args, argp, sizeof(args))) { - ret = -EFAULT; - break; - } - - ret = vchiq_ioc_create_service(instance, &args); - if (ret < 0) - break; - - if (put_user(args.handle, &argp->handle)) { - vchiq_remove_service(args.handle); - ret = -EFAULT; - } - } break; - - case VCHIQ_IOC_CLOSE_SERVICE: - case VCHIQ_IOC_REMOVE_SERVICE: { - unsigned int handle = (unsigned int)arg; - struct user_service *user_service; - - service = find_service_for_instance(instance, handle); - if (!service) { - ret = -EINVAL; - break; - } - - user_service = service->base.userdata; - - /* close_pending is false on first entry, and when the - wait in vchiq_close_service has been interrupted. */ - if (!user_service->close_pending) { - status = (cmd == VCHIQ_IOC_CLOSE_SERVICE) ? - vchiq_close_service(service->handle) : - vchiq_remove_service(service->handle); - if (status != VCHIQ_SUCCESS) - break; - } - - /* close_pending is true once the underlying service - has been closed until the client library calls the - CLOSE_DELIVERED ioctl, signalling close_event. */ - if (user_service->close_pending && - wait_for_completion_interruptible( - &user_service->close_event)) - status = VCHIQ_RETRY; - break; - } - - case VCHIQ_IOC_USE_SERVICE: - case VCHIQ_IOC_RELEASE_SERVICE: { - unsigned int handle = (unsigned int)arg; - - service = find_service_for_instance(instance, handle); - if (service) { - status = (cmd == VCHIQ_IOC_USE_SERVICE) ? - vchiq_use_service_internal(service) : - vchiq_release_service_internal(service); - if (status != VCHIQ_SUCCESS) { - vchiq_log_error(vchiq_susp_log_level, - "%s: cmd %s returned error %d for " - "service %c%c%c%c:%03d", - __func__, - (cmd == VCHIQ_IOC_USE_SERVICE) ? - "VCHIQ_IOC_USE_SERVICE" : - "VCHIQ_IOC_RELEASE_SERVICE", - status, - VCHIQ_FOURCC_AS_4CHARS( - service->base.fourcc), - service->client_id); - ret = -EINVAL; - } - } else - ret = -EINVAL; - } break; - - case VCHIQ_IOC_QUEUE_MESSAGE: { - struct vchiq_queue_message args; - - if (copy_from_user(&args, (const void __user *)arg, - sizeof(args))) { - ret = -EFAULT; - break; - } - - service = find_service_for_instance(instance, args.handle); - - if (service && (args.count <= MAX_ELEMENTS)) { - /* Copy elements into kernel space */ - struct vchiq_element elements[MAX_ELEMENTS]; - - if (copy_from_user(elements, args.elements, - args.count * sizeof(struct vchiq_element)) == 0) - ret = vchiq_ioc_queue_message(args.handle, elements, - args.count); - else - ret = -EFAULT; - } else { - ret = -EINVAL; - } - } break; - - case VCHIQ_IOC_QUEUE_BULK_TRANSMIT: - case VCHIQ_IOC_QUEUE_BULK_RECEIVE: { - struct vchiq_queue_bulk_transfer args; - struct vchiq_queue_bulk_transfer __user *argp; - - enum vchiq_bulk_dir dir = - (cmd == VCHIQ_IOC_QUEUE_BULK_TRANSMIT) ? - VCHIQ_BULK_TRANSMIT : VCHIQ_BULK_RECEIVE; - - argp = (void __user *)arg; - if (copy_from_user(&args, argp, sizeof(args))) { - ret = -EFAULT; - break; - } - - ret = vchiq_irq_queue_bulk_tx_rx(instance, &args, - dir, &argp->mode); - } break; - - case VCHIQ_IOC_AWAIT_COMPLETION: { - struct vchiq_await_completion args; - struct vchiq_await_completion __user *argp; - - argp = (void __user *)arg; - if (copy_from_user(&args, argp, sizeof(args))) { - ret = -EFAULT; - break; - } - - ret = vchiq_ioc_await_completion(instance, &args, - &argp->msgbufcount); - } break; - - case VCHIQ_IOC_DEQUEUE_MESSAGE: { - struct vchiq_dequeue_message args; - - if (copy_from_user(&args, (const void __user *)arg, - sizeof(args))) { - ret = -EFAULT; - break; - } - - ret = vchiq_ioc_dequeue_message(instance, &args); - } break; - - case VCHIQ_IOC_GET_CLIENT_ID: { - unsigned int handle = (unsigned int)arg; - - ret = vchiq_get_client_id(handle); - } break; - - case VCHIQ_IOC_GET_CONFIG: { - struct vchiq_get_config args; - struct vchiq_config config; - - if (copy_from_user(&args, (const void __user *)arg, - sizeof(args))) { - ret = -EFAULT; - break; - } - if (args.config_size > sizeof(config)) { - ret = -EINVAL; - break; - } - - vchiq_get_config(&config); - if (copy_to_user(args.pconfig, &config, args.config_size)) { - ret = -EFAULT; - break; - } - } break; - - case VCHIQ_IOC_SET_SERVICE_OPTION: { - struct vchiq_set_service_option args; - - if (copy_from_user(&args, (const void __user *)arg, - sizeof(args))) { - ret = -EFAULT; - break; - } - - service = find_service_for_instance(instance, args.handle); - if (!service) { - ret = -EINVAL; - break; - } - - status = vchiq_set_service_option( - args.handle, args.option, args.value); - } break; - - case VCHIQ_IOC_LIB_VERSION: { - unsigned int lib_version = (unsigned int)arg; - - if (lib_version < VCHIQ_VERSION_MIN) - ret = -EINVAL; - else if (lib_version >= VCHIQ_VERSION_CLOSE_DELIVERED) - instance->use_close_delivered = 1; - } break; - - case VCHIQ_IOC_CLOSE_DELIVERED: { - unsigned int handle = (unsigned int)arg; - - service = find_closed_service_for_instance(instance, handle); - if (service) { - struct user_service *user_service = - (struct user_service *)service->base.userdata; - close_delivered(user_service); - } else - ret = -EINVAL; - } break; - - default: - ret = -ENOTTY; - break; - } - - if (service) - unlock_service(service); - - if (ret == 0) { - if (status == VCHIQ_ERROR) - ret = -EIO; - else if (status == VCHIQ_RETRY) - ret = -EINTR; - } - - if ((status == VCHIQ_SUCCESS) && (ret < 0) && (ret != -EINTR) && - (ret != -EWOULDBLOCK)) - vchiq_log_info(vchiq_arm_log_level, - " ioctl instance %pK, cmd %s -> status %d, %ld", - instance, - (_IOC_NR(cmd) <= VCHIQ_IOC_MAX) ? - ioctl_names[_IOC_NR(cmd)] : - "", - status, ret); - else - vchiq_log_trace(vchiq_arm_log_level, - " ioctl instance %pK, cmd %s -> status %d, %ld", - instance, - (_IOC_NR(cmd) <= VCHIQ_IOC_MAX) ? - ioctl_names[_IOC_NR(cmd)] : - "", - status, ret); - - return ret; -} - -#if defined(CONFIG_COMPAT) - -struct vchiq_service_params32 { - int fourcc; - compat_uptr_t callback; - compat_uptr_t userdata; - short version; /* Increment for non-trivial changes */ - short version_min; /* Update for incompatible changes */ -}; - -struct vchiq_create_service32 { - struct vchiq_service_params32 params; - int is_open; - int is_vchi; - unsigned int handle; /* OUT */ -}; - -#define VCHIQ_IOC_CREATE_SERVICE32 \ - _IOWR(VCHIQ_IOC_MAGIC, 2, struct vchiq_create_service32) - -static long -vchiq_compat_ioctl_create_service( - struct file *file, - unsigned int cmd, - struct vchiq_create_service32 __user *ptrargs32) -{ - struct vchiq_create_service args; - struct vchiq_create_service32 args32; - long ret; - - if (copy_from_user(&args32, ptrargs32, sizeof(args32))) - return -EFAULT; - - args = (struct vchiq_create_service) { - .params = { - .fourcc = args32.params.fourcc, - .callback = compat_ptr(args32.params.callback), - .userdata = compat_ptr(args32.params.userdata), - .version = args32.params.version, - .version_min = args32.params.version_min, - }, - .is_open = args32.is_open, - .is_vchi = args32.is_vchi, - .handle = args32.handle, - }; - - ret = vchiq_ioc_create_service(file->private_data, &args); - if (ret < 0) - return ret; - - if (put_user(args.handle, &ptrargs32->handle)) { - vchiq_remove_service(args.handle); - return -EFAULT; - } - - return 0; -} - -struct vchiq_element32 { - compat_uptr_t data; - unsigned int size; -}; - -struct vchiq_queue_message32 { - unsigned int handle; - unsigned int count; - compat_uptr_t elements; -}; - -#define VCHIQ_IOC_QUEUE_MESSAGE32 \ - _IOW(VCHIQ_IOC_MAGIC, 4, struct vchiq_queue_message32) - -static long -vchiq_compat_ioctl_queue_message(struct file *file, - unsigned int cmd, - struct vchiq_queue_message32 __user *arg) -{ - struct vchiq_queue_message args; - struct vchiq_queue_message32 args32; - struct vchiq_service *service; - int ret; - - if (copy_from_user(&args32, arg, sizeof(args32))) - return -EFAULT; - - args = (struct vchiq_queue_message) { - .handle = args32.handle, - .count = args32.count, - .elements = compat_ptr(args32.elements), - }; - - if (args32.count > MAX_ELEMENTS) - return -EINVAL; - - service = find_service_for_instance(file->private_data, args.handle); - if (!service) - return -EINVAL; - - if (args32.elements && args32.count) { - struct vchiq_element32 element32[MAX_ELEMENTS]; - struct vchiq_element elements[MAX_ELEMENTS]; - unsigned int count; - - if (copy_from_user(&element32, args.elements, - sizeof(element32))) { - unlock_service(service); - return -EFAULT; - } - - for (count = 0; count < args32.count; count++) { - elements[count].data = - compat_ptr(element32[count].data); - elements[count].size = element32[count].size; - } - ret = vchiq_ioc_queue_message(args.handle, elements, - args.count); - } else { - ret = -EINVAL; - } - unlock_service(service); - - return ret; -} - -struct vchiq_queue_bulk_transfer32 { - unsigned int handle; - compat_uptr_t data; - unsigned int size; - compat_uptr_t userdata; - enum vchiq_bulk_mode mode; -}; - -#define VCHIQ_IOC_QUEUE_BULK_TRANSMIT32 \ - _IOWR(VCHIQ_IOC_MAGIC, 5, struct vchiq_queue_bulk_transfer32) -#define VCHIQ_IOC_QUEUE_BULK_RECEIVE32 \ - _IOWR(VCHIQ_IOC_MAGIC, 6, struct vchiq_queue_bulk_transfer32) - -static long -vchiq_compat_ioctl_queue_bulk(struct file *file, - unsigned int cmd, - struct vchiq_queue_bulk_transfer32 __user *argp) -{ - struct vchiq_queue_bulk_transfer32 args32; - struct vchiq_queue_bulk_transfer args; - enum vchiq_bulk_dir dir = (cmd == VCHIQ_IOC_QUEUE_BULK_TRANSMIT32) ? - VCHIQ_BULK_TRANSMIT : VCHIQ_BULK_RECEIVE; - - if (copy_from_user(&args32, argp, sizeof(args32))) - return -EFAULT; - - args = (struct vchiq_queue_bulk_transfer) { - .handle = args32.handle, - .data = compat_ptr(args32.data), - .size = args32.size, - .userdata = compat_ptr(args32.userdata), - .mode = args32.mode, - }; - - return vchiq_irq_queue_bulk_tx_rx(file->private_data, &args, - dir, &argp->mode); -} - -struct vchiq_await_completion32 { - unsigned int count; - compat_uptr_t buf; - unsigned int msgbufsize; - unsigned int msgbufcount; /* IN/OUT */ - compat_uptr_t msgbufs; -}; - -#define VCHIQ_IOC_AWAIT_COMPLETION32 \ - _IOWR(VCHIQ_IOC_MAGIC, 7, struct vchiq_await_completion32) - -static long -vchiq_compat_ioctl_await_completion(struct file *file, - unsigned int cmd, - struct vchiq_await_completion32 __user *argp) -{ - struct vchiq_await_completion args; - struct vchiq_await_completion32 args32; - - if (copy_from_user(&args32, argp, sizeof(args32))) - return -EFAULT; - - args = (struct vchiq_await_completion) { - .count = args32.count, - .buf = compat_ptr(args32.buf), - .msgbufsize = args32.msgbufsize, - .msgbufcount = args32.msgbufcount, - .msgbufs = compat_ptr(args32.msgbufs), - }; - - return vchiq_ioc_await_completion(file->private_data, &args, - &argp->msgbufcount); -} - -struct vchiq_dequeue_message32 { - unsigned int handle; - int blocking; - unsigned int bufsize; - compat_uptr_t buf; -}; - -#define VCHIQ_IOC_DEQUEUE_MESSAGE32 \ - _IOWR(VCHIQ_IOC_MAGIC, 8, struct vchiq_dequeue_message32) - -static long -vchiq_compat_ioctl_dequeue_message(struct file *file, - unsigned int cmd, - struct vchiq_dequeue_message32 __user *arg) -{ - struct vchiq_dequeue_message32 args32; - struct vchiq_dequeue_message args; - - if (copy_from_user(&args32, arg, sizeof(args32))) - return -EFAULT; - - args = (struct vchiq_dequeue_message) { - .handle = args32.handle, - .blocking = args32.blocking, - .bufsize = args32.bufsize, - .buf = compat_ptr(args32.buf), - }; - - return vchiq_ioc_dequeue_message(file->private_data, &args); -} - -struct vchiq_get_config32 { - unsigned int config_size; - compat_uptr_t pconfig; -}; - -#define VCHIQ_IOC_GET_CONFIG32 \ - _IOWR(VCHIQ_IOC_MAGIC, 10, struct vchiq_get_config32) - -static long -vchiq_compat_ioctl_get_config(struct file *file, - unsigned int cmd, - struct vchiq_get_config32 __user *arg) -{ - struct vchiq_get_config32 args32; - struct vchiq_config config; - void __user *ptr; - - if (copy_from_user(&args32, arg, sizeof(args32))) - return -EFAULT; - if (args32.config_size > sizeof(config)) - return -EINVAL; - - vchiq_get_config(&config); - ptr = compat_ptr(args32.pconfig); - if (copy_to_user(ptr, &config, args32.config_size)) - return -EFAULT; - - return 0; -} - -static long -vchiq_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg) -{ - void __user *argp = compat_ptr(arg); - switch (cmd) { - case VCHIQ_IOC_CREATE_SERVICE32: - return vchiq_compat_ioctl_create_service(file, cmd, argp); - case VCHIQ_IOC_QUEUE_MESSAGE32: - return vchiq_compat_ioctl_queue_message(file, cmd, argp); - case VCHIQ_IOC_QUEUE_BULK_TRANSMIT32: - case VCHIQ_IOC_QUEUE_BULK_RECEIVE32: - return vchiq_compat_ioctl_queue_bulk(file, cmd, argp); - case VCHIQ_IOC_AWAIT_COMPLETION32: - return vchiq_compat_ioctl_await_completion(file, cmd, argp); - case VCHIQ_IOC_DEQUEUE_MESSAGE32: - return vchiq_compat_ioctl_dequeue_message(file, cmd, argp); - case VCHIQ_IOC_GET_CONFIG32: - return vchiq_compat_ioctl_get_config(file, cmd, argp); - default: - return vchiq_ioctl(file, cmd, (unsigned long)argp); - } -} - -#endif - -static int vchiq_open(struct inode *inode, struct file *file) -{ - struct vchiq_state *state = vchiq_get_state(); - struct vchiq_instance *instance; - - vchiq_log_info(vchiq_arm_log_level, "vchiq_open"); - - if (!state) { - vchiq_log_error(vchiq_arm_log_level, - "vchiq has no connection to VideoCore"); - return -ENOTCONN; - } - - instance = kzalloc(sizeof(*instance), GFP_KERNEL); - if (!instance) - return -ENOMEM; - - instance->state = state; - instance->pid = current->tgid; - - vchiq_debugfs_add_instance(instance); - - init_completion(&instance->insert_event); - init_completion(&instance->remove_event); - mutex_init(&instance->completion_mutex); - mutex_init(&instance->bulk_waiter_list_mutex); - INIT_LIST_HEAD(&instance->bulk_waiter_list); - - file->private_data = instance; - - return 0; -} - -static int vchiq_release(struct inode *inode, struct file *file) -{ - struct vchiq_instance *instance = file->private_data; - struct vchiq_state *state = vchiq_get_state(); - struct vchiq_service *service; - int ret = 0; - int i; - - vchiq_log_info(vchiq_arm_log_level, "%s: instance=%lx", __func__, - (unsigned long)instance); - - if (!state) { - ret = -EPERM; - goto out; - } - - /* Ensure videocore is awake to allow termination. */ - vchiq_use_internal(instance->state, NULL, USE_TYPE_VCHIQ); - - mutex_lock(&instance->completion_mutex); - - /* Wake the completion thread and ask it to exit */ - instance->closing = 1; - complete(&instance->insert_event); - - mutex_unlock(&instance->completion_mutex); - - /* Wake the slot handler if the completion queue is full. */ - complete(&instance->remove_event); - - /* Mark all services for termination... */ - i = 0; - while ((service = next_service_by_instance(state, instance, &i))) { - struct user_service *user_service = service->base.userdata; - - /* Wake the slot handler if the msg queue is full. */ - complete(&user_service->remove_event); - - vchiq_terminate_service_internal(service); - unlock_service(service); - } - - /* ...and wait for them to die */ - i = 0; - while ((service = next_service_by_instance(state, instance, &i))) { - struct user_service *user_service = service->base.userdata; - - wait_for_completion(&service->remove_event); - - BUG_ON(service->srvstate != VCHIQ_SRVSTATE_FREE); - - spin_lock(&msg_queue_spinlock); - - while (user_service->msg_remove != user_service->msg_insert) { - struct vchiq_header *header; - int m = user_service->msg_remove & (MSG_QUEUE_SIZE - 1); - - header = user_service->msg_queue[m]; - user_service->msg_remove++; - spin_unlock(&msg_queue_spinlock); - - if (header) - vchiq_release_message(service->handle, header); - spin_lock(&msg_queue_spinlock); - } - - spin_unlock(&msg_queue_spinlock); - - unlock_service(service); - } - - /* Release any closed services */ - while (instance->completion_remove != - instance->completion_insert) { - struct vchiq_completion_data_kernel *completion; - struct vchiq_service *service; - - completion = &instance->completions[ - instance->completion_remove & (MAX_COMPLETIONS - 1)]; - service = completion->service_userdata; - if (completion->reason == VCHIQ_SERVICE_CLOSED) { - struct user_service *user_service = - service->base.userdata; - - /* Wake any blocked user-thread */ - if (instance->use_close_delivered) - complete(&user_service->close_event); - unlock_service(service); - } - instance->completion_remove++; - } - - /* Release the PEER service count. */ - vchiq_release_internal(instance->state, NULL); - - { - struct bulk_waiter_node *waiter, *next; - - list_for_each_entry_safe(waiter, next, - &instance->bulk_waiter_list, list) { - list_del(&waiter->list); - vchiq_log_info(vchiq_arm_log_level, - "bulk_waiter - cleaned up %pK for pid %d", - waiter, waiter->pid); - kfree(waiter); - } - } - - vchiq_debugfs_remove_instance(instance); - - kfree(instance); - file->private_data = NULL; - -out: - return ret; -} - -/**************************************************************************** -* -* vchiq_dump -* -***************************************************************************/ - -int vchiq_dump(void *dump_context, const char *str, int len) -{ - struct dump_context *context = (struct dump_context *)dump_context; - int copy_bytes; - - if (context->actual >= context->space) - return 0; - - if (context->offset > 0) { - int skip_bytes = min_t(int, len, context->offset); - - str += skip_bytes; - len -= skip_bytes; - context->offset -= skip_bytes; - if (context->offset > 0) - return 0; - } - copy_bytes = min_t(int, len, context->space - context->actual); - if (copy_bytes == 0) - return 0; - if (copy_to_user(context->buf + context->actual, str, - copy_bytes)) - return -EFAULT; - context->actual += copy_bytes; - len -= copy_bytes; - - /* - * If the terminating NUL is included in the length, then it - * marks the end of a line and should be replaced with a - * carriage return. - */ - if ((len == 0) && (str[copy_bytes - 1] == '\0')) { - char cr = '\n'; - - if (copy_to_user(context->buf + context->actual - 1, - &cr, 1)) - return -EFAULT; - } - return 0; -} - -/**************************************************************************** -* -* vchiq_dump_platform_instance_state -* -***************************************************************************/ - -int vchiq_dump_platform_instances(void *dump_context) -{ - struct vchiq_state *state = vchiq_get_state(); - char buf[80]; - int len; - int i; - - /* There is no list of instances, so instead scan all services, - marking those that have been dumped. */ - - rcu_read_lock(); - for (i = 0; i < state->unused_service; i++) { - struct vchiq_service *service; - struct vchiq_instance *instance; - - service = rcu_dereference(state->services[i]); - if (!service || service->base.callback != service_callback) - continue; - - instance = service->instance; - if (instance) - instance->mark = 0; - } - rcu_read_unlock(); - - for (i = 0; i < state->unused_service; i++) { - struct vchiq_service *service; - struct vchiq_instance *instance; - int err; - - rcu_read_lock(); - service = rcu_dereference(state->services[i]); - if (!service || service->base.callback != service_callback) { - rcu_read_unlock(); - continue; - } - - instance = service->instance; - if (!instance || instance->mark) { - rcu_read_unlock(); - continue; - } - rcu_read_unlock(); - - len = snprintf(buf, sizeof(buf), - "Instance %pK: pid %d,%s completions %d/%d", - instance, instance->pid, - instance->connected ? " connected, " : - "", - instance->completion_insert - - instance->completion_remove, - MAX_COMPLETIONS); - err = vchiq_dump(dump_context, buf, len + 1); - if (err) - return err; - instance->mark = 1; - } - return 0; -} - -/**************************************************************************** -* -* vchiq_dump_platform_service_state -* -***************************************************************************/ - -int vchiq_dump_platform_service_state(void *dump_context, - struct vchiq_service *service) -{ - struct user_service *user_service = - (struct user_service *)service->base.userdata; - char buf[80]; - int len; - - len = scnprintf(buf, sizeof(buf), " instance %pK", service->instance); - - if ((service->base.callback == service_callback) && - user_service->is_vchi) { - len += scnprintf(buf + len, sizeof(buf) - len, - ", %d/%d messages", - user_service->msg_insert - user_service->msg_remove, - MSG_QUEUE_SIZE); - - if (user_service->dequeue_pending) - len += scnprintf(buf + len, sizeof(buf) - len, - " (dequeue pending)"); - } - - return vchiq_dump(dump_context, buf, len + 1); -} - -/**************************************************************************** -* -* vchiq_read -* -***************************************************************************/ - -static ssize_t -vchiq_read(struct file *file, char __user *buf, - size_t count, loff_t *ppos) -{ - struct dump_context context; - int err; - - context.buf = buf; - context.actual = 0; - context.space = count; - context.offset = *ppos; - - err = vchiq_dump_state(&context, &g_state); - if (err) - return err; - - *ppos += context.actual; - - return context.actual; -} - -struct vchiq_state * -vchiq_get_state(void) -{ - - if (!g_state.remote) - printk(KERN_ERR "%s: g_state.remote == NULL\n", __func__); - else if (g_state.remote->initialised != 1) - printk(KERN_NOTICE "%s: g_state.remote->initialised != 1 (%d)\n", - __func__, g_state.remote->initialised); + if (!g_state.remote) + printk(KERN_ERR "%s: g_state.remote == NULL\n", __func__); + else if (g_state.remote->initialised != 1) + printk(KERN_NOTICE "%s: g_state.remote->initialised != 1 (%d)\n", + __func__, g_state.remote->initialised); return (g_state.remote && (g_state.remote->initialised == 1)) ? &g_state : NULL; } -static const struct file_operations -vchiq_fops = { - .owner = THIS_MODULE, - .unlocked_ioctl = vchiq_ioctl, -#if defined(CONFIG_COMPAT) - .compat_ioctl = vchiq_compat_ioctl, -#endif - .open = vchiq_open, - .release = vchiq_release, - .read = vchiq_read -}; - -/** - * vchiq_register_chrdev - Register the char driver for vchiq - * and create the necessary class and - * device files in userspace. - * @parent The parent of the char device. - * - * Returns 0 on success else returns the error code. - */ -int vchiq_register_chrdev(struct device *parent) -{ - struct device *vchiq_dev; - int ret; - - vchiq_class = class_create(THIS_MODULE, DEVICE_NAME); - if (IS_ERR(vchiq_class)) { - pr_err("Failed to create vchiq class\n"); - ret = PTR_ERR(vchiq_class); - goto error_exit; - } - - ret = alloc_chrdev_region(&vchiq_devid, 0, 1, DEVICE_NAME); - if (ret) { - pr_err("vchiq: Failed to allocate vchiq's chrdev region\n"); - goto alloc_region_error; - } - - cdev_init(&vchiq_cdev, &vchiq_fops); - vchiq_cdev.owner = THIS_MODULE; - ret = cdev_add(&vchiq_cdev, vchiq_devid, 1); - if (ret) { - vchiq_log_error(vchiq_arm_log_level, - "Unable to register vchiq char device"); - goto cdev_add_error; - } - - vchiq_dev = device_create(vchiq_class, parent, vchiq_devid, NULL, - DEVICE_NAME); - if (IS_ERR(vchiq_dev)) { - vchiq_log_error(vchiq_arm_log_level, - "Failed to create vchiq char device node"); - ret = PTR_ERR(vchiq_dev); - goto device_create_error; - } - - vchiq_log_info(vchiq_arm_log_level, - "vchiq char dev initialised successfully - device %d.%d", - MAJOR(vchiq_devid), MINOR(vchiq_devid)); - - return 0; - -device_create_error: - cdev_del(&vchiq_cdev); - -cdev_add_error: - unregister_chrdev_region(vchiq_devid, 1); - -alloc_region_error: - class_destroy(vchiq_class); - -error_exit: - return ret; -} - -/** - * vchiq_deregister_chrdev - Deregister and cleanup the vchiq char - * driver and device files - */ -void vchiq_deregister_chrdev(void) -{ - device_destroy(vchiq_class, vchiq_devid); - cdev_del(&vchiq_cdev); - unregister_chrdev_region(vchiq_devid, 1); - class_destroy(vchiq_class); -} - /* * Autosuspend related functionality */ diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_dev.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_dev.c new file mode 100644 index 000000000000..034dc7de0656 --- /dev/null +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_dev.c @@ -0,0 +1,1488 @@ +#include +#include +#include +#include +#include + +#include "vchiq_core.h" +#include "vchiq_ioctl.h" +#include "vchiq_arm.h" +#include "vchiq_debugfs.h" + +#define DEVICE_NAME "vchiq" + +static struct cdev vchiq_cdev; +static dev_t vchiq_devid; +static struct class *vchiq_class; + +static const char *const ioctl_names[] = { + "CONNECT", + "SHUTDOWN", + "CREATE_SERVICE", + "REMOVE_SERVICE", + "QUEUE_MESSAGE", + "QUEUE_BULK_TRANSMIT", + "QUEUE_BULK_RECEIVE", + "AWAIT_COMPLETION", + "DEQUEUE_MESSAGE", + "GET_CLIENT_ID", + "GET_CONFIG", + "CLOSE_SERVICE", + "USE_SERVICE", + "RELEASE_SERVICE", + "SET_SERVICE_OPTION", + "DUMP_PHYS_MEM", + "LIB_VERSION", + "CLOSE_DELIVERED" +}; + +vchiq_static_assert(ARRAY_SIZE(ioctl_names) == + (VCHIQ_IOC_MAX + 1)); + +static void +user_service_free(void *userdata) +{ + kfree(userdata); +} + +static void close_delivered(struct user_service *user_service) +{ + vchiq_log_info(vchiq_arm_log_level, + "%s(handle=%x)", + __func__, user_service->service->handle); + + if (user_service->close_pending) { + /* Allow the underlying service to be culled */ + unlock_service(user_service->service); + + /* Wake the user-thread blocked in close_ or remove_service */ + complete(&user_service->close_event); + + user_service->close_pending = 0; + } +} + +struct vchiq_io_copy_callback_context { + struct vchiq_element *element; + size_t element_offset; + unsigned long elements_to_go; +}; + +static ssize_t vchiq_ioc_copy_element_data(void *context, void *dest, + size_t offset, size_t maxsize) +{ + struct vchiq_io_copy_callback_context *cc = context; + size_t total_bytes_copied = 0; + size_t bytes_this_round; + + while (total_bytes_copied < maxsize) { + if (!cc->elements_to_go) + return total_bytes_copied; + + if (!cc->element->size) { + cc->elements_to_go--; + cc->element++; + cc->element_offset = 0; + continue; + } + + bytes_this_round = min(cc->element->size - cc->element_offset, + maxsize - total_bytes_copied); + + if (copy_from_user(dest + total_bytes_copied, + cc->element->data + cc->element_offset, + bytes_this_round)) + return -EFAULT; + + cc->element_offset += bytes_this_round; + total_bytes_copied += bytes_this_round; + + if (cc->element_offset == cc->element->size) { + cc->elements_to_go--; + cc->element++; + cc->element_offset = 0; + } + } + + return maxsize; +} + +static int +vchiq_ioc_queue_message(unsigned int handle, + struct vchiq_element *elements, + unsigned long count) +{ + struct vchiq_io_copy_callback_context context; + enum vchiq_status status = VCHIQ_SUCCESS; + unsigned long i; + size_t total_size = 0; + + context.element = elements; + context.element_offset = 0; + context.elements_to_go = count; + + for (i = 0; i < count; i++) { + if (!elements[i].data && elements[i].size != 0) + return -EFAULT; + + total_size += elements[i].size; + } + + status = vchiq_queue_message(handle, vchiq_ioc_copy_element_data, + &context, total_size); + + if (status == VCHIQ_ERROR) + return -EIO; + else if (status == VCHIQ_RETRY) + return -EINTR; + return 0; +} + +/* read a user pointer value from an array pointers in user space */ +static inline int vchiq_get_user_ptr(void __user **buf, void __user *ubuf, int index) +{ + int ret; + + if (in_compat_syscall()) { + compat_uptr_t ptr32; + + compat_uptr_t __user *uptr = ubuf; + ret = get_user(ptr32, uptr + index); + *buf = compat_ptr(ptr32); + } else { + uintptr_t ptr, __user *uptr = ubuf; + + ret = get_user(ptr, uptr + index); + *buf = (void __user *)ptr; + } + + return ret; +} + +struct vchiq_completion_data32 { + enum vchiq_reason reason; + compat_uptr_t header; + compat_uptr_t service_userdata; + compat_uptr_t bulk_userdata; +}; + +static int vchiq_put_completion(struct vchiq_completion_data __user *buf, + struct vchiq_completion_data *completion, + int index) +{ + struct vchiq_completion_data32 __user *buf32 = (void __user *)buf; + + if (in_compat_syscall()) { + struct vchiq_completion_data32 tmp = { + .reason = completion->reason, + .header = ptr_to_compat(completion->header), + .service_userdata = ptr_to_compat(completion->service_userdata), + .bulk_userdata = ptr_to_compat(completion->bulk_userdata), + }; + if (copy_to_user(&buf32[index], &tmp, sizeof(tmp))) + return -EFAULT; + } else { + if (copy_to_user(&buf[index], completion, sizeof(*completion))) + return -EFAULT; + } + + return 0; +} + +static int vchiq_ioc_await_completion(struct vchiq_instance *instance, + struct vchiq_await_completion *args, + int __user *msgbufcountp) +{ + int msgbufcount; + int remove; + int ret; + + DEBUG_INITIALISE(g_state.local) + + DEBUG_TRACE(AWAIT_COMPLETION_LINE); + if (!instance->connected) + return -ENOTCONN; + + mutex_lock(&instance->completion_mutex); + + DEBUG_TRACE(AWAIT_COMPLETION_LINE); + while ((instance->completion_remove == instance->completion_insert) + && !instance->closing) { + int rc; + + DEBUG_TRACE(AWAIT_COMPLETION_LINE); + mutex_unlock(&instance->completion_mutex); + rc = wait_for_completion_interruptible( + &instance->insert_event); + mutex_lock(&instance->completion_mutex); + if (rc) { + DEBUG_TRACE(AWAIT_COMPLETION_LINE); + vchiq_log_info(vchiq_arm_log_level, + "AWAIT_COMPLETION interrupted"); + ret = -EINTR; + goto out; + } + } + DEBUG_TRACE(AWAIT_COMPLETION_LINE); + + msgbufcount = args->msgbufcount; + remove = instance->completion_remove; + + for (ret = 0; ret < args->count; ret++) { + struct vchiq_completion_data_kernel *completion; + struct vchiq_completion_data user_completion; + struct vchiq_service *service; + struct user_service *user_service; + struct vchiq_header *header; + + if (remove == instance->completion_insert) + break; + + completion = &instance->completions[ + remove & (MAX_COMPLETIONS - 1)]; + + /* + * A read memory barrier is needed to stop + * prefetch of a stale completion record + */ + rmb(); + + service = completion->service_userdata; + user_service = service->base.userdata; + + memset(&user_completion, 0, sizeof(user_completion)); + user_completion = (struct vchiq_completion_data) { + .reason = completion->reason, + .service_userdata = user_service->userdata, + }; + + header = completion->header; + if (header) { + void __user *msgbuf; + int msglen; + + msglen = header->size + sizeof(struct vchiq_header); + /* This must be a VCHIQ-style service */ + if (args->msgbufsize < msglen) { + vchiq_log_error(vchiq_arm_log_level, + "header %pK: msgbufsize %x < msglen %x", + header, args->msgbufsize, msglen); + WARN(1, "invalid message size\n"); + if (ret == 0) + ret = -EMSGSIZE; + break; + } + if (msgbufcount <= 0) + /* + * Stall here for lack of a + * buffer for the message. + */ + break; + /* Get the pointer from user space */ + msgbufcount--; + if (vchiq_get_user_ptr(&msgbuf, args->msgbufs, + msgbufcount)) { + if (ret == 0) + ret = -EFAULT; + break; + } + + /* Copy the message to user space */ + if (copy_to_user(msgbuf, header, msglen)) { + if (ret == 0) + ret = -EFAULT; + break; + } + + /* Now it has been copied, the message + * can be released. + */ + vchiq_release_message(service->handle, header); + + /* The completion must point to the + * msgbuf. + */ + user_completion.header = msgbuf; + } + + if ((completion->reason == VCHIQ_SERVICE_CLOSED) && + !instance->use_close_delivered) + unlock_service(service); + + /* + * FIXME: address space mismatch, does bulk_userdata + * actually point to user or kernel memory? + */ + user_completion.bulk_userdata = completion->bulk_userdata; + + if (vchiq_put_completion(args->buf, &user_completion, ret)) { + if (ret == 0) + ret = -EFAULT; + break; + } + + /* + * Ensure that the above copy has completed + * before advancing the remove pointer. + */ + mb(); + remove++; + instance->completion_remove = remove; + } + + if (msgbufcount != args->msgbufcount) { + if (put_user(msgbufcount, msgbufcountp)) + ret = -EFAULT; + } +out: + if (ret) + complete(&instance->remove_event); + mutex_unlock(&instance->completion_mutex); + DEBUG_TRACE(AWAIT_COMPLETION_LINE); + + return ret; +} + +static int vchiq_ioc_create_service(struct vchiq_instance *instance, + struct vchiq_create_service *args) +{ + struct user_service *user_service = NULL; + struct vchiq_service *service; + enum vchiq_status status = VCHIQ_SUCCESS; + struct vchiq_service_params_kernel params; + int srvstate; + + user_service = kmalloc(sizeof(*user_service), GFP_KERNEL); + if (!user_service) + return -ENOMEM; + + if (args->is_open) { + if (!instance->connected) { + kfree(user_service); + return -ENOTCONN; + } + srvstate = VCHIQ_SRVSTATE_OPENING; + } else { + srvstate = instance->connected ? + VCHIQ_SRVSTATE_LISTENING : VCHIQ_SRVSTATE_HIDDEN; + } + + params = (struct vchiq_service_params_kernel) { + .fourcc = args->params.fourcc, + .callback = service_callback, + .userdata = user_service, + .version = args->params.version, + .version_min = args->params.version_min, + }; + service = vchiq_add_service_internal(instance->state, ¶ms, + srvstate, instance, + user_service_free); + if (!service) { + kfree(user_service); + return -EEXIST; + } + + user_service->service = service; + user_service->userdata = args->params.userdata; + user_service->instance = instance; + user_service->is_vchi = (args->is_vchi != 0); + user_service->dequeue_pending = 0; + user_service->close_pending = 0; + user_service->message_available_pos = instance->completion_remove - 1; + user_service->msg_insert = 0; + user_service->msg_remove = 0; + init_completion(&user_service->insert_event); + init_completion(&user_service->remove_event); + init_completion(&user_service->close_event); + + if (args->is_open) { + status = vchiq_open_service_internal(service, instance->pid); + if (status != VCHIQ_SUCCESS) { + vchiq_remove_service(service->handle); + return (status == VCHIQ_RETRY) ? + -EINTR : -EIO; + } + } + args->handle = service->handle; + + return 0; +} + +static int vchiq_ioc_dequeue_message(struct vchiq_instance *instance, + struct vchiq_dequeue_message *args) +{ + struct user_service *user_service; + struct vchiq_service *service; + struct vchiq_header *header; + int ret; + + DEBUG_INITIALISE(g_state.local) + DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); + service = find_service_for_instance(instance, args->handle); + if (!service) + return -EINVAL; + + user_service = (struct user_service *)service->base.userdata; + if (user_service->is_vchi == 0) { + ret = -EINVAL; + goto out; + } + + spin_lock(&msg_queue_spinlock); + if (user_service->msg_remove == user_service->msg_insert) { + if (!args->blocking) { + spin_unlock(&msg_queue_spinlock); + DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); + ret = -EWOULDBLOCK; + goto out; + } + user_service->dequeue_pending = 1; + ret = 0; + do { + spin_unlock(&msg_queue_spinlock); + DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); + if (wait_for_completion_interruptible( + &user_service->insert_event)) { + vchiq_log_info(vchiq_arm_log_level, + "DEQUEUE_MESSAGE interrupted"); + ret = -EINTR; + break; + } + spin_lock(&msg_queue_spinlock); + } while (user_service->msg_remove == + user_service->msg_insert); + + if (ret) + goto out; + } + + BUG_ON((int)(user_service->msg_insert - + user_service->msg_remove) < 0); + + header = user_service->msg_queue[user_service->msg_remove & + (MSG_QUEUE_SIZE - 1)]; + user_service->msg_remove++; + spin_unlock(&msg_queue_spinlock); + + complete(&user_service->remove_event); + if (!header) { + ret = -ENOTCONN; + } else if (header->size <= args->bufsize) { + /* Copy to user space if msgbuf is not NULL */ + if (!args->buf || (copy_to_user(args->buf, + header->data, header->size) == 0)) { + ret = header->size; + vchiq_release_message(service->handle, header); + } else { + ret = -EFAULT; + } + } else { + vchiq_log_error(vchiq_arm_log_level, + "header %pK: bufsize %x < size %x", + header, args->bufsize, header->size); + WARN(1, "invalid size\n"); + ret = -EMSGSIZE; + } + DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); +out: + unlock_service(service); + return ret; +} + +static int vchiq_irq_queue_bulk_tx_rx(struct vchiq_instance *instance, + struct vchiq_queue_bulk_transfer *args, + enum vchiq_bulk_dir dir, + enum vchiq_bulk_mode __user *mode) +{ + struct vchiq_service *service; + struct bulk_waiter_node *waiter = NULL; + bool found = false; + void *userdata; + int status = 0; + int ret; + + service = find_service_for_instance(instance, args->handle); + if (!service) + return -EINVAL; + + if (args->mode == VCHIQ_BULK_MODE_BLOCKING) { + waiter = kzalloc(sizeof(struct bulk_waiter_node), + GFP_KERNEL); + if (!waiter) { + ret = -ENOMEM; + goto out; + } + + userdata = &waiter->bulk_waiter; + } else if (args->mode == VCHIQ_BULK_MODE_WAITING) { + mutex_lock(&instance->bulk_waiter_list_mutex); + list_for_each_entry(waiter, &instance->bulk_waiter_list, + list) { + if (waiter->pid == current->pid) { + list_del(&waiter->list); + found = true; + break; + } + } + mutex_unlock(&instance->bulk_waiter_list_mutex); + if (!found) { + vchiq_log_error(vchiq_arm_log_level, + "no bulk_waiter found for pid %d", + current->pid); + ret = -ESRCH; + goto out; + } + vchiq_log_info(vchiq_arm_log_level, + "found bulk_waiter %pK for pid %d", waiter, + current->pid); + userdata = &waiter->bulk_waiter; + } else { + userdata = args->userdata; + } + + /* + * FIXME address space mismatch: + * args->data may be interpreted as a kernel pointer + * in create_pagelist() called from vchiq_bulk_transfer(), + * accessing kernel data instead of user space, based on the + * address. + */ + status = vchiq_bulk_transfer(args->handle, NULL, args->data, args->size, + userdata, args->mode, dir); + + if (!waiter) { + ret = 0; + goto out; + } + + if ((status != VCHIQ_RETRY) || fatal_signal_pending(current) || + !waiter->bulk_waiter.bulk) { + if (waiter->bulk_waiter.bulk) { + /* Cancel the signal when the transfer + * completes. + */ + spin_lock(&bulk_waiter_spinlock); + waiter->bulk_waiter.bulk->userdata = NULL; + spin_unlock(&bulk_waiter_spinlock); + } + kfree(waiter); + ret = 0; + } else { + const enum vchiq_bulk_mode mode_waiting = + VCHIQ_BULK_MODE_WAITING; + waiter->pid = current->pid; + mutex_lock(&instance->bulk_waiter_list_mutex); + list_add(&waiter->list, &instance->bulk_waiter_list); + mutex_unlock(&instance->bulk_waiter_list_mutex); + vchiq_log_info(vchiq_arm_log_level, + "saved bulk_waiter %pK for pid %d", + waiter, current->pid); + + ret = put_user(mode_waiting, mode); + } +out: + unlock_service(service); + if (ret) + return ret; + else if (status == VCHIQ_ERROR) + return -EIO; + else if (status == VCHIQ_RETRY) + return -EINTR; + return 0; +} + +/**************************************************************************** + * + * vchiq_read + * + ************************************************************************** + */ +static ssize_t +vchiq_read(struct file *file, char __user *buf, + size_t count, loff_t *ppos) +{ + struct dump_context context; + int err; + + context.buf = buf; + context.actual = 0; + context.space = count; + context.offset = *ppos; + + err = vchiq_dump_state(&context, &g_state); + if (err) + return err; + + *ppos += context.actual; + + return context.actual; +} + +/**************************************************************************** +* +* vchiq_open +* +***************************************************************************/ +static int vchiq_open(struct inode *inode, struct file *file) +{ + struct vchiq_state *state = vchiq_get_state(); + struct vchiq_instance *instance; + + vchiq_log_info(vchiq_arm_log_level, "vchiq_open"); + + if (!state) { + vchiq_log_error(vchiq_arm_log_level, + "vchiq has no connection to VideoCore"); + return -ENOTCONN; + } + + instance = kzalloc(sizeof(*instance), GFP_KERNEL); + if (!instance) + return -ENOMEM; + + instance->state = state; + instance->pid = current->tgid; + + vchiq_debugfs_add_instance(instance); + + init_completion(&instance->insert_event); + init_completion(&instance->remove_event); + mutex_init(&instance->completion_mutex); + mutex_init(&instance->bulk_waiter_list_mutex); + INIT_LIST_HEAD(&instance->bulk_waiter_list); + + file->private_data = instance; + + return 0; +} + +/**************************************************************************** + * + * vchiq_release + * + ************************************************************************** + */ +static int vchiq_release(struct inode *inode, struct file *file) +{ + struct vchiq_instance *instance = file->private_data; + struct vchiq_state *state = vchiq_get_state(); + struct vchiq_service *service; + int ret = 0; + int i; + + vchiq_log_info(vchiq_arm_log_level, "%s: instance=%lx", __func__, + (unsigned long)instance); + + if (!state) { + ret = -EPERM; + goto out; + } + + /* Ensure videocore is awake to allow termination. */ + vchiq_use_internal(instance->state, NULL, USE_TYPE_VCHIQ); + + mutex_lock(&instance->completion_mutex); + + /* Wake the completion thread and ask it to exit */ + instance->closing = 1; + complete(&instance->insert_event); + + mutex_unlock(&instance->completion_mutex); + + /* Wake the slot handler if the completion queue is full. */ + complete(&instance->remove_event); + + /* Mark all services for termination... */ + i = 0; + while ((service = next_service_by_instance(state, instance, &i))) { + struct user_service *user_service = service->base.userdata; + + /* Wake the slot handler if the msg queue is full. */ + complete(&user_service->remove_event); + + vchiq_terminate_service_internal(service); + unlock_service(service); + } + + /* ...and wait for them to die */ + i = 0; + while ((service = next_service_by_instance(state, instance, &i))) { + struct user_service *user_service = service->base.userdata; + + wait_for_completion(&service->remove_event); + + BUG_ON(service->srvstate != VCHIQ_SRVSTATE_FREE); + + spin_lock(&msg_queue_spinlock); + + while (user_service->msg_remove != user_service->msg_insert) { + struct vchiq_header *header; + int m = user_service->msg_remove & (MSG_QUEUE_SIZE - 1); + + header = user_service->msg_queue[m]; + user_service->msg_remove++; + spin_unlock(&msg_queue_spinlock); + + if (header) + vchiq_release_message(service->handle, header); + spin_lock(&msg_queue_spinlock); + } + + spin_unlock(&msg_queue_spinlock); + + unlock_service(service); + } + + /* Release any closed services */ + while (instance->completion_remove != + instance->completion_insert) { + struct vchiq_completion_data_kernel *completion; + struct vchiq_service *service; + + completion = &instance->completions[ + instance->completion_remove & (MAX_COMPLETIONS - 1)]; + service = completion->service_userdata; + if (completion->reason == VCHIQ_SERVICE_CLOSED) { + struct user_service *user_service = + service->base.userdata; + + /* Wake any blocked user-thread */ + if (instance->use_close_delivered) + complete(&user_service->close_event); + unlock_service(service); + } + instance->completion_remove++; + } + + /* Release the PEER service count. */ + vchiq_release_internal(instance->state, NULL); + + { + struct bulk_waiter_node *waiter, *next; + + list_for_each_entry_safe(waiter, next, + &instance->bulk_waiter_list, list) { + list_del(&waiter->list); + vchiq_log_info(vchiq_arm_log_level, + "bulk_waiter - cleaned up %pK for pid %d", + waiter, waiter->pid); + kfree(waiter); + } + } + + vchiq_debugfs_remove_instance(instance); + + kfree(instance); + file->private_data = NULL; + +out: + return ret; +} + +/**************************************************************************** + * + * vchiq_ioctl + * + ************************************************************************** + */ +static long +vchiq_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + struct vchiq_instance *instance = file->private_data; + enum vchiq_status status = VCHIQ_SUCCESS; + struct vchiq_service *service = NULL; + long ret = 0; + int i, rc; + + vchiq_log_trace(vchiq_arm_log_level, + "%s - instance %pK, cmd %s, arg %lx", + __func__, instance, + ((_IOC_TYPE(cmd) == VCHIQ_IOC_MAGIC) && + (_IOC_NR(cmd) <= VCHIQ_IOC_MAX)) ? + ioctl_names[_IOC_NR(cmd)] : "", arg); + + switch (cmd) { + case VCHIQ_IOC_SHUTDOWN: + if (!instance->connected) + break; + + /* Remove all services */ + i = 0; + while ((service = next_service_by_instance(instance->state, + instance, &i))) { + status = vchiq_remove_service(service->handle); + unlock_service(service); + if (status != VCHIQ_SUCCESS) + break; + } + service = NULL; + + if (status == VCHIQ_SUCCESS) { + /* Wake the completion thread and ask it to exit */ + instance->closing = 1; + complete(&instance->insert_event); + } + + break; + + case VCHIQ_IOC_CONNECT: + if (instance->connected) { + ret = -EINVAL; + break; + } + rc = mutex_lock_killable(&instance->state->mutex); + if (rc) { + vchiq_log_error(vchiq_arm_log_level, + "vchiq: connect: could not lock mutex for " + "state %d: %d", + instance->state->id, rc); + ret = -EINTR; + break; + } + status = vchiq_connect_internal(instance->state, instance); + mutex_unlock(&instance->state->mutex); + + if (status == VCHIQ_SUCCESS) + instance->connected = 1; + else + vchiq_log_error(vchiq_arm_log_level, + "vchiq: could not connect: %d", status); + break; + + case VCHIQ_IOC_CREATE_SERVICE: { + struct vchiq_create_service __user *argp; + struct vchiq_create_service args; + + argp = (void __user *)arg; + if (copy_from_user(&args, argp, sizeof(args))) { + ret = -EFAULT; + break; + } + + ret = vchiq_ioc_create_service(instance, &args); + if (ret < 0) + break; + + if (put_user(args.handle, &argp->handle)) { + vchiq_remove_service(args.handle); + ret = -EFAULT; + } + } break; + + case VCHIQ_IOC_CLOSE_SERVICE: + case VCHIQ_IOC_REMOVE_SERVICE: { + unsigned int handle = (unsigned int)arg; + struct user_service *user_service; + + service = find_service_for_instance(instance, handle); + if (!service) { + ret = -EINVAL; + break; + } + + user_service = service->base.userdata; + + /* + * close_pending is false on first entry, and when the + * wait in vchiq_close_service has been interrupted. + */ + if (!user_service->close_pending) { + status = (cmd == VCHIQ_IOC_CLOSE_SERVICE) ? + vchiq_close_service(service->handle) : + vchiq_remove_service(service->handle); + if (status != VCHIQ_SUCCESS) + break; + } + + /* + * close_pending is true once the underlying service + * has been closed until the client library calls the + * CLOSE_DELIVERED ioctl, signalling close_event. + */ + if (user_service->close_pending && + wait_for_completion_interruptible( + &user_service->close_event)) + status = VCHIQ_RETRY; + break; + } + + case VCHIQ_IOC_USE_SERVICE: + case VCHIQ_IOC_RELEASE_SERVICE: { + unsigned int handle = (unsigned int)arg; + + service = find_service_for_instance(instance, handle); + if (service) { + status = (cmd == VCHIQ_IOC_USE_SERVICE) ? + vchiq_use_service_internal(service) : + vchiq_release_service_internal(service); + if (status != VCHIQ_SUCCESS) { + vchiq_log_error(vchiq_susp_log_level, + "%s: cmd %s returned error %d for " + "service %c%c%c%c:%03d", + __func__, + (cmd == VCHIQ_IOC_USE_SERVICE) ? + "VCHIQ_IOC_USE_SERVICE" : + "VCHIQ_IOC_RELEASE_SERVICE", + status, + VCHIQ_FOURCC_AS_4CHARS( + service->base.fourcc), + service->client_id); + ret = -EINVAL; + } + } else + ret = -EINVAL; + } break; + + case VCHIQ_IOC_QUEUE_MESSAGE: { + struct vchiq_queue_message args; + + if (copy_from_user(&args, (const void __user *)arg, + sizeof(args))) { + ret = -EFAULT; + break; + } + + service = find_service_for_instance(instance, args.handle); + + if (service && (args.count <= MAX_ELEMENTS)) { + /* Copy elements into kernel space */ + struct vchiq_element elements[MAX_ELEMENTS]; + + if (copy_from_user(elements, args.elements, + args.count * sizeof(struct vchiq_element)) == 0) + ret = vchiq_ioc_queue_message(args.handle, elements, + args.count); + else + ret = -EFAULT; + } else { + ret = -EINVAL; + } + } break; + + case VCHIQ_IOC_QUEUE_BULK_TRANSMIT: + case VCHIQ_IOC_QUEUE_BULK_RECEIVE: { + struct vchiq_queue_bulk_transfer args; + struct vchiq_queue_bulk_transfer __user *argp; + + enum vchiq_bulk_dir dir = + (cmd == VCHIQ_IOC_QUEUE_BULK_TRANSMIT) ? + VCHIQ_BULK_TRANSMIT : VCHIQ_BULK_RECEIVE; + + argp = (void __user *)arg; + if (copy_from_user(&args, argp, sizeof(args))) { + ret = -EFAULT; + break; + } + + ret = vchiq_irq_queue_bulk_tx_rx(instance, &args, + dir, &argp->mode); + } break; + + case VCHIQ_IOC_AWAIT_COMPLETION: { + struct vchiq_await_completion args; + struct vchiq_await_completion __user *argp; + + argp = (void __user *)arg; + if (copy_from_user(&args, argp, sizeof(args))) { + ret = -EFAULT; + break; + } + + ret = vchiq_ioc_await_completion(instance, &args, + &argp->msgbufcount); + } break; + + case VCHIQ_IOC_DEQUEUE_MESSAGE: { + struct vchiq_dequeue_message args; + + if (copy_from_user(&args, (const void __user *)arg, + sizeof(args))) { + ret = -EFAULT; + break; + } + + ret = vchiq_ioc_dequeue_message(instance, &args); + } break; + + case VCHIQ_IOC_GET_CLIENT_ID: { + unsigned int handle = (unsigned int)arg; + + ret = vchiq_get_client_id(handle); + } break; + + case VCHIQ_IOC_GET_CONFIG: { + struct vchiq_get_config args; + struct vchiq_config config; + + if (copy_from_user(&args, (const void __user *)arg, + sizeof(args))) { + ret = -EFAULT; + break; + } + if (args.config_size > sizeof(config)) { + ret = -EINVAL; + break; + } + + vchiq_get_config(&config); + if (copy_to_user(args.pconfig, &config, args.config_size)) { + ret = -EFAULT; + break; + } + } break; + + case VCHIQ_IOC_SET_SERVICE_OPTION: { + struct vchiq_set_service_option args; + + if (copy_from_user(&args, (const void __user *)arg, + sizeof(args))) { + ret = -EFAULT; + break; + } + + service = find_service_for_instance(instance, args.handle); + if (!service) { + ret = -EINVAL; + break; + } + + status = vchiq_set_service_option( + args.handle, args.option, args.value); + } break; + + case VCHIQ_IOC_LIB_VERSION: { + unsigned int lib_version = (unsigned int)arg; + + if (lib_version < VCHIQ_VERSION_MIN) + ret = -EINVAL; + else if (lib_version >= VCHIQ_VERSION_CLOSE_DELIVERED) + instance->use_close_delivered = 1; + } break; + + case VCHIQ_IOC_CLOSE_DELIVERED: { + unsigned int handle = (unsigned int)arg; + + service = find_closed_service_for_instance(instance, handle); + if (service) { + struct user_service *user_service = + (struct user_service *)service->base.userdata; + close_delivered(user_service); + } else + ret = -EINVAL; + } break; + + default: + ret = -ENOTTY; + break; + } + + if (service) + unlock_service(service); + + if (ret == 0) { + if (status == VCHIQ_ERROR) + ret = -EIO; + else if (status == VCHIQ_RETRY) + ret = -EINTR; + } + + if ((status == VCHIQ_SUCCESS) && (ret < 0) && (ret != -EINTR) && + (ret != -EWOULDBLOCK)) + vchiq_log_info(vchiq_arm_log_level, + " ioctl instance %pK, cmd %s -> status %d, %ld", + instance, + (_IOC_NR(cmd) <= VCHIQ_IOC_MAX) ? + ioctl_names[_IOC_NR(cmd)] : + "", + status, ret); + else + vchiq_log_trace(vchiq_arm_log_level, + " ioctl instance %pK, cmd %s -> status %d, %ld", + instance, + (_IOC_NR(cmd) <= VCHIQ_IOC_MAX) ? + ioctl_names[_IOC_NR(cmd)] : + "", + status, ret); + + return ret; +} + +/**************************************************************************** + * + * compat IOCTL related functions + * + ****************************************************************************/ +#if defined(CONFIG_COMPAT) + +struct vchiq_service_params32 { + int fourcc; + compat_uptr_t callback; + compat_uptr_t userdata; + short version; /* Increment for non-trivial changes */ + short version_min; /* Update for incompatible changes */ +}; + +struct vchiq_create_service32 { + struct vchiq_service_params32 params; + int is_open; + int is_vchi; + unsigned int handle; /* OUT */ +}; + +#define VCHIQ_IOC_CREATE_SERVICE32 \ + _IOWR(VCHIQ_IOC_MAGIC, 2, struct vchiq_create_service32) + +static long +vchiq_compat_ioctl_create_service( + struct file *file, + unsigned int cmd, + struct vchiq_create_service32 __user *ptrargs32) +{ + struct vchiq_create_service args; + struct vchiq_create_service32 args32; + long ret; + + if (copy_from_user(&args32, ptrargs32, sizeof(args32))) + return -EFAULT; + + args = (struct vchiq_create_service) { + .params = { + .fourcc = args32.params.fourcc, + .callback = compat_ptr(args32.params.callback), + .userdata = compat_ptr(args32.params.userdata), + .version = args32.params.version, + .version_min = args32.params.version_min, + }, + .is_open = args32.is_open, + .is_vchi = args32.is_vchi, + .handle = args32.handle, + }; + + ret = vchiq_ioc_create_service(file->private_data, &args); + if (ret < 0) + return ret; + + if (put_user(args.handle, &ptrargs32->handle)) { + vchiq_remove_service(args.handle); + return -EFAULT; + } + + return 0; +} + +struct vchiq_element32 { + compat_uptr_t data; + unsigned int size; +}; + +struct vchiq_queue_message32 { + unsigned int handle; + unsigned int count; + compat_uptr_t elements; +}; + +#define VCHIQ_IOC_QUEUE_MESSAGE32 \ + _IOW(VCHIQ_IOC_MAGIC, 4, struct vchiq_queue_message32) + +static long +vchiq_compat_ioctl_queue_message(struct file *file, + unsigned int cmd, + struct vchiq_queue_message32 __user *arg) +{ + struct vchiq_queue_message args; + struct vchiq_queue_message32 args32; + struct vchiq_service *service; + int ret; + + if (copy_from_user(&args32, arg, sizeof(args32))) + return -EFAULT; + + args = (struct vchiq_queue_message) { + .handle = args32.handle, + .count = args32.count, + .elements = compat_ptr(args32.elements), + }; + + if (args32.count > MAX_ELEMENTS) + return -EINVAL; + + service = find_service_for_instance(file->private_data, args.handle); + if (!service) + return -EINVAL; + + if (args32.elements && args32.count) { + struct vchiq_element32 element32[MAX_ELEMENTS]; + struct vchiq_element elements[MAX_ELEMENTS]; + unsigned int count; + + if (copy_from_user(&element32, args.elements, + sizeof(element32))) { + unlock_service(service); + return -EFAULT; + } + + for (count = 0; count < args32.count; count++) { + elements[count].data = + compat_ptr(element32[count].data); + elements[count].size = element32[count].size; + } + ret = vchiq_ioc_queue_message(args.handle, elements, + args.count); + } else { + ret = -EINVAL; + } + unlock_service(service); + + return ret; +} + +struct vchiq_queue_bulk_transfer32 { + unsigned int handle; + compat_uptr_t data; + unsigned int size; + compat_uptr_t userdata; + enum vchiq_bulk_mode mode; +}; + +#define VCHIQ_IOC_QUEUE_BULK_TRANSMIT32 \ + _IOWR(VCHIQ_IOC_MAGIC, 5, struct vchiq_queue_bulk_transfer32) +#define VCHIQ_IOC_QUEUE_BULK_RECEIVE32 \ + _IOWR(VCHIQ_IOC_MAGIC, 6, struct vchiq_queue_bulk_transfer32) + +static long +vchiq_compat_ioctl_queue_bulk(struct file *file, + unsigned int cmd, + struct vchiq_queue_bulk_transfer32 __user *argp) +{ + struct vchiq_queue_bulk_transfer32 args32; + struct vchiq_queue_bulk_transfer args; + enum vchiq_bulk_dir dir = (cmd == VCHIQ_IOC_QUEUE_BULK_TRANSMIT32) ? + VCHIQ_BULK_TRANSMIT : VCHIQ_BULK_RECEIVE; + + if (copy_from_user(&args32, argp, sizeof(args32))) + return -EFAULT; + + args = (struct vchiq_queue_bulk_transfer) { + .handle = args32.handle, + .data = compat_ptr(args32.data), + .size = args32.size, + .userdata = compat_ptr(args32.userdata), + .mode = args32.mode, + }; + + return vchiq_irq_queue_bulk_tx_rx(file->private_data, &args, + dir, &argp->mode); +} + +struct vchiq_await_completion32 { + unsigned int count; + compat_uptr_t buf; + unsigned int msgbufsize; + unsigned int msgbufcount; /* IN/OUT */ + compat_uptr_t msgbufs; +}; + +#define VCHIQ_IOC_AWAIT_COMPLETION32 \ + _IOWR(VCHIQ_IOC_MAGIC, 7, struct vchiq_await_completion32) + +static long +vchiq_compat_ioctl_await_completion(struct file *file, + unsigned int cmd, + struct vchiq_await_completion32 __user *argp) +{ + struct vchiq_await_completion args; + struct vchiq_await_completion32 args32; + + if (copy_from_user(&args32, argp, sizeof(args32))) + return -EFAULT; + + args = (struct vchiq_await_completion) { + .count = args32.count, + .buf = compat_ptr(args32.buf), + .msgbufsize = args32.msgbufsize, + .msgbufcount = args32.msgbufcount, + .msgbufs = compat_ptr(args32.msgbufs), + }; + + return vchiq_ioc_await_completion(file->private_data, &args, + &argp->msgbufcount); +} + +struct vchiq_dequeue_message32 { + unsigned int handle; + int blocking; + unsigned int bufsize; + compat_uptr_t buf; +}; + +#define VCHIQ_IOC_DEQUEUE_MESSAGE32 \ + _IOWR(VCHIQ_IOC_MAGIC, 8, struct vchiq_dequeue_message32) + +static long +vchiq_compat_ioctl_dequeue_message(struct file *file, + unsigned int cmd, + struct vchiq_dequeue_message32 __user *arg) +{ + struct vchiq_dequeue_message32 args32; + struct vchiq_dequeue_message args; + + if (copy_from_user(&args32, arg, sizeof(args32))) + return -EFAULT; + + args = (struct vchiq_dequeue_message) { + .handle = args32.handle, + .blocking = args32.blocking, + .bufsize = args32.bufsize, + .buf = compat_ptr(args32.buf), + }; + + return vchiq_ioc_dequeue_message(file->private_data, &args); +} + +struct vchiq_get_config32 { + unsigned int config_size; + compat_uptr_t pconfig; +}; + +#define VCHIQ_IOC_GET_CONFIG32 \ + _IOWR(VCHIQ_IOC_MAGIC, 10, struct vchiq_get_config32) + +static long +vchiq_compat_ioctl_get_config(struct file *file, + unsigned int cmd, + struct vchiq_get_config32 __user *arg) +{ + struct vchiq_get_config32 args32; + struct vchiq_config config; + void __user *ptr; + + if (copy_from_user(&args32, arg, sizeof(args32))) + return -EFAULT; + if (args32.config_size > sizeof(config)) + return -EINVAL; + + vchiq_get_config(&config); + ptr = compat_ptr(args32.pconfig); + if (copy_to_user(ptr, &config, args32.config_size)) + return -EFAULT; + + return 0; +} + +/** + * + * vchiq_compat_ioctl + * + */ +static long +vchiq_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg) +{ + void __user *argp = compat_ptr(arg); + switch (cmd) { + case VCHIQ_IOC_CREATE_SERVICE32: + return vchiq_compat_ioctl_create_service(file, cmd, argp); + case VCHIQ_IOC_QUEUE_MESSAGE32: + return vchiq_compat_ioctl_queue_message(file, cmd, argp); + case VCHIQ_IOC_QUEUE_BULK_TRANSMIT32: + case VCHIQ_IOC_QUEUE_BULK_RECEIVE32: + return vchiq_compat_ioctl_queue_bulk(file, cmd, argp); + case VCHIQ_IOC_AWAIT_COMPLETION32: + return vchiq_compat_ioctl_await_completion(file, cmd, argp); + case VCHIQ_IOC_DEQUEUE_MESSAGE32: + return vchiq_compat_ioctl_dequeue_message(file, cmd, argp); + case VCHIQ_IOC_GET_CONFIG32: + return vchiq_compat_ioctl_get_config(file, cmd, argp); + default: + return vchiq_ioctl(file, cmd, (unsigned long)argp); + } +} + +/* compat IOCTL end */ +#endif + +static const struct file_operations +vchiq_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = vchiq_ioctl, +#if defined(CONFIG_COMPAT) + .compat_ioctl = vchiq_compat_ioctl, +#endif + .open = vchiq_open, + .release = vchiq_release, + .read = vchiq_read +}; + +/** + * vchiq_register_chrdev - Register the char driver for vchiq + * and create the necessary class and + * device files in userspace. + * @parent The parent of the char device. + * + * Returns 0 on success else returns the error code. + */ +int vchiq_register_chrdev(struct device *parent) +{ + struct device *vchiq_dev; + int ret; + + vchiq_class = class_create(THIS_MODULE, DEVICE_NAME); + if (IS_ERR(vchiq_class)) { + pr_err("Failed to create vchiq class\n"); + ret = PTR_ERR(vchiq_class); + goto error_exit; + } + + ret = alloc_chrdev_region(&vchiq_devid, 0, 1, DEVICE_NAME); + if (ret) { + pr_err("vchiq: Failed to allocate vchiq's chrdev region\n"); + goto alloc_region_error; + } + + cdev_init(&vchiq_cdev, &vchiq_fops); + vchiq_cdev.owner = THIS_MODULE; + ret = cdev_add(&vchiq_cdev, vchiq_devid, 1); + if (ret) { + vchiq_log_error(vchiq_arm_log_level, + "Unable to register vchiq char device"); + goto cdev_add_error; + } + + vchiq_dev = device_create(vchiq_class, parent, vchiq_devid, NULL, + DEVICE_NAME); + if (IS_ERR(vchiq_dev)) { + vchiq_log_error(vchiq_arm_log_level, + "Failed to create vchiq char device node"); + ret = PTR_ERR(vchiq_dev); + goto device_create_error; + } + + vchiq_log_info(vchiq_arm_log_level, + "vchiq char dev initialised successfully - device %d.%d", + MAJOR(vchiq_devid), MINOR(vchiq_devid)); + + return 0; + +device_create_error: + cdev_del(&vchiq_cdev); + +cdev_add_error: + unregister_chrdev_region(vchiq_devid, 1); + +alloc_region_error: + class_destroy(vchiq_class); + +error_exit: + return ret; +} + +/** + * vchiq_deregister_chrdev - Deregister and cleanup the vchiq char + * driver and device files + */ +void vchiq_deregister_chrdev(void) +{ + device_destroy(vchiq_class, vchiq_devid); + cdev_del(&vchiq_cdev); + unregister_chrdev_region(vchiq_devid, 1); + class_destroy(vchiq_class); +} From patchwork Sun Jun 20 12:57:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ojaswin Mujoo X-Patchwork-Id: 12333401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F950C48BDF for ; Sun, 20 Jun 2021 12:59:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6792F610CA for ; Sun, 20 Jun 2021 12:59:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6792F610CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8DxUMCGTVBoJ6/k7UCMiSjJEvwNzS8ziYeqbZFApNgg=; b=fGb1n6gL/MvYjG xgfpJgHWK7v6ZHOTnP6GpuIX0hAE2k/3kIheDlbPw1sSdt96nOdJG3E4rO3ox5aTdV5p7cBDROwqT z/2xN44Ve+Se+Twn/YMUqtZaRhT8pv1aH4Zq9qw6lsIW5jA2Zy28eC2lMIFb55J2EHT3d1oHIE18p XSG+UhHuBhQWvWVdc3TwMLllnGhLS4QeYb0p5TcyCOW2yu3SZSn3j1/1urwo4a9DUAJzHGHKVveVH ECM4bwWwoG6dH4vG16p37LoUKkLQnHnXVkYFuYFixmmKfkHjMddZ/gAFb6zgszGzFpxzkM3JhUA1j zHY0ML+rfeh7TGMO9FdA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux14-0011cn-Vn; Sun, 20 Jun 2021 12:57:35 +0000 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux0i-0011TK-4x for linux-arm-kernel@lists.infradead.org; Sun, 20 Jun 2021 12:57:13 +0000 Received: by mail-pl1-x636.google.com with SMTP id o21so7066900pll.6 for ; Sun, 20 Jun 2021 05:57:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=NEMvFyS18P5cftlbD6eQ1AwuZhrAFjZ8m++eEi7Zvp4=; b=NB3Tnxdo9kDLe2Bi09gNLIU9WWb7jtpM3y2BQQznCYCVA9ylsjvzN4N179Ogz0A7BH qDNnnE370C8YRZIOO0LWTSsiG/poEZ1VXlGO0pr+O5qiG1ZaUddPQ1D1klKRPY0UoxiT zWwBa/xvCwufx3zpfzW1OtoXEA4tEhk0JvZHmTmNR7ZVtcPuwgNz8A6rhjiwax6qunId 3wJpFtLj+g2lOFuk2AsxdsBk1ABg3KvW4H0IqK2zButA7EuVgmoRWQC/q3xbXrh5NlfU K6/5c0wHW090cAZqKdmZYeua13njLnzCqMCL+lSIMHmkVdea5WG190EyTrR+LlU+vsQ2 8Ovw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=NEMvFyS18P5cftlbD6eQ1AwuZhrAFjZ8m++eEi7Zvp4=; b=sTVOuSjVlU6Pk+vnAV0M3ysC7DEt7Kk9kP2S/GBvTQTAeCBagI7eapQ4vYIvUIytR9 uzHbimFIyiZR6E7SF0HqeVBK+BpK8WAICXgwbO6L2BLL18Z09abHpl3r39wKHA7hqaNw CXbsx3zlU16s76Gmfv9nhBph99WZIknTn7vt7r0yBbgjSsN2kprqGiFhM14XdZ+tAvLd M4iMe6E3dDNSnsWutn9TyYCb7scTLnnsqY7BFknwmaKvDF7c7XiHAMgHnoTVu20KwXON kMGRY1y5fhtS5vRQnhp8DpjT0/zUZix46rZPqA97BxBF5D4zMuvI5ilju0SVXvqVugkS bgjQ== X-Gm-Message-State: AOAM531MsNyDXJ6u7XgILlRWERznXplqtnPp8ogIVA6VFgSzH/vJ5cEJ 6GQd95GbJXtoPNa485xWSoIC0r/iaxg= X-Google-Smtp-Source: ABdhPJxq2skko6hT7WAdZ8XJ4MFeYCt4USuct+irJN/hebx2+vSZZO8601KL5ql3Pmo1gN7RSlQ/TQ== X-Received: by 2002:a17:902:ac81:b029:122:1809:40e with SMTP id h1-20020a170902ac81b02901221809040emr9498905plr.79.1624193831403; Sun, 20 Jun 2021 05:57:11 -0700 (PDT) Received: from ojas ([122.177.154.120]) by smtp.gmail.com with ESMTPSA id b1sm13215203pgb.91.2021.06.20.05.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 05:57:11 -0700 (PDT) Date: Sun, 20 Jun 2021 18:27:01 +0530 From: Ojaswin Mujoo To: nsaenz@kernel.org Cc: gregkh@linuxfoundation.org, stefan.wahren@i2se.com, arnd@arndb.de, dan.carpenter@oracle.com, phil@raspberrypi.com, bcm-kernel-feedback-list@broadcom.com, linux-arm-kernel@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 4/5] staging: vchiq: Make creation of vchiq cdev optional Message-ID: <80d4ad6cb3129d7c6e7e66f3d358c6dea82c509b.1624185152.git.ojaswin98@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210620_055712_243519_F9A91354 X-CRM114-Status: GOOD ( 14.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Before this commit, vchiq cdev (/dev/vchiq) was always created during platform initialization. Introduce a new Kconfig option CONFIG_VCHIQ_CDEV which determines the cdev needs to be created or not. Also modify the predefined config files to have this option set to "yes". Signed-off-by: Ojaswin Mujoo --- arch/arm/configs/bcm2709_defconfig | 1 + arch/arm/configs/bcm2711_defconfig | 1 + arch/arm/configs/bcmrpi_defconfig | 1 + drivers/staging/vc04_services/Kconfig | 10 ++++++++++ drivers/staging/vc04_services/Makefile | 5 ++++- .../vc04_services/interface/vchiq_arm/vchiq_arm.h | 9 +++++++++ 6 files changed, 26 insertions(+), 1 deletion(-) diff --git a/arch/arm/configs/bcm2709_defconfig b/arch/arm/configs/bcm2709_defconfig index 7eb4418d7fea..444c80b97e2c 100644 --- a/arch/arm/configs/bcm2709_defconfig +++ b/arch/arm/configs/bcm2709_defconfig @@ -1314,6 +1314,7 @@ CONFIG_FB_TFT_UC1701=m CONFIG_FB_TFT_UPD161704=m CONFIG_FB_TFT_WATTEROTT=m CONFIG_BCM2835_VCHIQ=y +CONFIG_VCHIQ_CDEV=y CONFIG_SND_BCM2835=m CONFIG_VIDEO_BCM2835=m CONFIG_VIDEO_CODEC_BCM2835=m diff --git a/arch/arm/configs/bcm2711_defconfig b/arch/arm/configs/bcm2711_defconfig index 299e4d95e4ca..f323fb7f29e2 100644 --- a/arch/arm/configs/bcm2711_defconfig +++ b/arch/arm/configs/bcm2711_defconfig @@ -1352,6 +1352,7 @@ CONFIG_FB_TFT_UC1701=m CONFIG_FB_TFT_UPD161704=m CONFIG_FB_TFT_WATTEROTT=m CONFIG_BCM2835_VCHIQ=y +CONFIG_VCHIQ_CDEV=y CONFIG_SND_BCM2835=m CONFIG_VIDEO_BCM2835=m CONFIG_VIDEO_CODEC_BCM2835=m diff --git a/arch/arm/configs/bcmrpi_defconfig b/arch/arm/configs/bcmrpi_defconfig index e270865d89cc..92216ce77d11 100644 --- a/arch/arm/configs/bcmrpi_defconfig +++ b/arch/arm/configs/bcmrpi_defconfig @@ -1325,6 +1325,7 @@ CONFIG_FB_TFT_UC1701=m CONFIG_FB_TFT_UPD161704=m CONFIG_FB_TFT_WATTEROTT=m CONFIG_BCM2835_VCHIQ=y +CONFIG_VCHIQ_CDEV=y CONFIG_SND_BCM2835=m CONFIG_VIDEO_BCM2835=m CONFIG_VIDEO_CODEC_BCM2835=m diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig index 8b912617bfec..7c22554cdd20 100644 --- a/drivers/staging/vc04_services/Kconfig +++ b/drivers/staging/vc04_services/Kconfig @@ -19,6 +19,16 @@ config BCM2835_VCHIQ Defaults to Y when the Broadcom Videocore services are included in the build, N otherwise. +if BCM2835_VCHIQ + +config VCHIQ_CDEV + bool "VCHIQ Character Driver" + help + Enable the creation of VCHIQ character driver to help + communicate with the Videocore platform. + +endif + source "drivers/staging/vc04_services/bcm2835-audio/Kconfig" source "drivers/staging/vc04_services/bcm2835-camera/Kconfig" diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile index 700cd62fe346..cc6371386a62 100644 --- a/drivers/staging/vc04_services/Makefile +++ b/drivers/staging/vc04_services/Makefile @@ -7,7 +7,10 @@ vchiq-objs := \ interface/vchiq_arm/vchiq_2835_arm.o \ interface/vchiq_arm/vchiq_debugfs.o \ interface/vchiq_arm/vchiq_connected.o \ - interface/vchiq_arm/vchiq_dev.o \ + +ifdef CONFIG_VCHIQ_CDEV +vchiq-objs += interface/vchiq_arm/vchiq_dev.o +endif obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/ obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/ diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h index bc9af1a0c764..a99cca0cb23b 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h @@ -172,12 +172,21 @@ vchiq_instance_get_trace(struct vchiq_instance *instance); extern void vchiq_instance_set_trace(struct vchiq_instance *instance, int trace); +#if IS_ENABLED(CONFIG_VCHIQ_CDEV) + extern void vchiq_deregister_chrdev(void); extern int vchiq_register_chrdev(struct device *parent); +#else + +static inline void vchiq_deregister_chrdev(void) { } +static inline int vchiq_register_chrdev(struct device *parent) { return 0; } + +#endif /* IS_ENABLED(CONFIG_VCHIQ_CDEV) */ + extern enum vchiq_status service_callback(enum vchiq_reason reason, struct vchiq_header *header, unsigned int handle, void *bulk_userdata); From patchwork Sun Jun 20 12:57:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ojaswin Mujoo X-Patchwork-Id: 12333405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DC9FC48BDF for ; Sun, 20 Jun 2021 12:59:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0E6EE610CA for ; Sun, 20 Jun 2021 12:59:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E6EE610CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IzhH0NOfdbsShl7XVc7YUY+QvcfKk6jDaltC8WlluGU=; b=THIoqH2Q0xO0I2 hbcW1SjSZ+oAb9LvkeKzI175C7e7MrIQhZaVhp3ouNamDWlxBcWbvk13yZX45cnjV6TPDI0XRKoNV dIAi9TJnuwN3aY0VXET7ZfFASJ2WZY7hWXbfQc0XklXWck2yJcHAyaYxooFIdPnWXCn1V4cBpStFD TejBbDSwoQU+ui1zfeOwYTURPHSoVChz0yUedEX/P5/OFZMpEOEFan6GW7bBzKK2N83F5LoY7VK1a wMNlxqht2Iv20t4JfSyWAn9l/IxL5InDZyZvKa5yf8AeyURk6dw3B4Y5HX5pmWHffi/AaS8kS2j5w llTaGsdbMJ6OlsJJWvNQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux1R-0011mP-1t; Sun, 20 Jun 2021 12:57:57 +0000 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lux19-0011et-6E for linux-arm-kernel@lists.infradead.org; Sun, 20 Jun 2021 12:57:43 +0000 Received: by mail-pj1-x102d.google.com with SMTP id h16so8360108pjv.2 for ; Sun, 20 Jun 2021 05:57:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=bLdCas7X3LkNs4ttDItS1W5jgGf0j45KNW4xOoMdFCs=; b=TND25wscFbac2J933iYeikhbRdw2RUW1/KJgF0tGguL3eytUbWa2uidfjHfTCd6zrj e8F1BZRUbnrd/2Bl3tbOYSvvjHNkQF/eOIy0Z5oO7lztuE2SCmvvrfMCNQqte91oCMyH Wv044OV0q3tOjKJWD7IvPgNua2tLLcH/Ng5tO48x9SDFwkFEaE7eDH78BaePHrd27ESv KeFqO5W8H8T+Mhi26aGLaQSUOCyR0s5DPX2q7kJAiRI41AvOcHs9lQw8wVUqsE2t4jC+ Qz9TuXioZh7P7GfXD0f9fI59mjwNZJM0imeFSxU5k/bRvfRcWLR9fdCOMDSTWf1mA3dr PLig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=bLdCas7X3LkNs4ttDItS1W5jgGf0j45KNW4xOoMdFCs=; b=RYJuz2hQIym3j4g6Cvbw109EhJIgAelQShZYpy3BiQEl/XCKcRVvO43r0/23rWQU2+ XsKrf4LUi9AHP+HP+CFAmmTHVr9hvuLKadC0iHRPsmpjF1Fy6r6K0gYtz7dGQrsRewF1 xD0K9H8q6CndQIro3G6kYdQKNoJsHm3HzfrbHTrgNS3TJ9BJaGDnaZbtsJ9Am34ahO8O wGAgCFP54+h8jCccRHJ+VCKSs3vFTbBkV/+Qhk3pzZnL7APpt1+x6f0IlsvNMHkQ1AvQ uUL94u0nqpgFZoO5/SneHuBFwb1QuDBdqPQqbQOP+3wFIz845mSsznXnXr4uZdeIUjfK 8a/w== X-Gm-Message-State: AOAM532qFNRn+OmuriXUDXv00671rPjrTsgjVilb8wEXnI/0mxQIGTf3 GPelSNIAK+BjUO6+XFwGQYU= X-Google-Smtp-Source: ABdhPJyfLv+oSPcgXygQx51RcC9mQ3b83kNRCDMi0Af0qglylchIZLlbeM1RU/ClG2Ov0FHiwovHzQ== X-Received: by 2002:a17:90a:4404:: with SMTP id s4mr32931717pjg.218.1624193858309; Sun, 20 Jun 2021 05:57:38 -0700 (PDT) Received: from ojas ([122.177.154.120]) by smtp.gmail.com with ESMTPSA id q12sm14236861pgc.25.2021.06.20.05.57.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 05:57:38 -0700 (PDT) Date: Sun, 20 Jun 2021 18:27:25 +0530 From: Ojaswin Mujoo To: nsaenz@kernel.org Cc: gregkh@linuxfoundation.org, stefan.wahren@i2se.com, arnd@arndb.de, dan.carpenter@oracle.com, phil@raspberrypi.com, bcm-kernel-feedback-list@broadcom.com, linux-arm-kernel@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v2 5/5] staging: vchiq: Combine vchiq platform code into single file Message-ID: <03145b153c7d1b1a285ccdefb60abc2377dc5558.1624185152.git.ojaswin98@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210620_055739_373943_5AAB7CCF X-CRM114-Status: GOOD ( 30.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Combine the vchiq platform initialization code into a single file. The file vchiq_2835_arm.c is now merged in vchiq_arm.c Signed-off-by: Ojaswin Mujoo --- drivers/staging/vc04_services/Makefile | 1 - .../interface/vchiq_arm/vchiq_2835_arm.c | 651 ------------------ .../interface/vchiq_arm/vchiq_arm.c | 635 +++++++++++++++++ 3 files changed, 635 insertions(+), 652 deletions(-) delete mode 100644 drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile index cc6371386a62..cc0a81996eda 100644 --- a/drivers/staging/vc04_services/Makefile +++ b/drivers/staging/vc04_services/Makefile @@ -4,7 +4,6 @@ obj-$(CONFIG_BCM2835_VCHIQ) += vchiq.o vchiq-objs := \ interface/vchiq_arm/vchiq_core.o \ interface/vchiq_arm/vchiq_arm.o \ - interface/vchiq_arm/vchiq_2835_arm.o \ interface/vchiq_arm/vchiq_debugfs.o \ interface/vchiq_arm/vchiq_connected.o \ diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c deleted file mode 100644 index 2a1d8d6541b2..000000000000 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c +++ /dev/null @@ -1,651 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause -/* Copyright (c) 2010-2012 Broadcom. All rights reserved. */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32) - -#include "vchiq_arm.h" -#include "vchiq_connected.h" -#include "vchiq_pagelist.h" - -#define MAX_FRAGMENTS (VCHIQ_NUM_CURRENT_BULKS * 2) - -#define VCHIQ_PLATFORM_FRAGMENTS_OFFSET_IDX 0 -#define VCHIQ_PLATFORM_FRAGMENTS_COUNT_IDX 1 - -#define BELL0 0x00 -#define BELL2 0x08 - -#define VCHIQ_DMA_POOL_SIZE PAGE_SIZE - -struct vchiq_2835_state { - int inited; - struct vchiq_arm_state arm_state; -}; - -struct vchiq_pagelist_info { - struct pagelist *pagelist; - size_t pagelist_buffer_size; - dma_addr_t dma_addr; - bool is_from_pool; - enum dma_data_direction dma_dir; - unsigned int num_pages; - unsigned int pages_need_release; - struct page **pages; - struct scatterlist *scatterlist; - unsigned int scatterlist_mapped; -}; - -static void __iomem *g_regs; -/* This value is the size of the L2 cache lines as understood by the - * VPU firmware, which determines the required alignment of the - * offsets/sizes in pagelists. - * - * Modern VPU firmware looks for a DT "cache-line-size" property in - * the VCHIQ node and will overwrite it with the actual L2 cache size, - * which the kernel must then respect. That property was rejected - * upstream, so we have to use the VPU firmware's compatibility value - * of 32. - */ -static unsigned int g_cache_line_size = 32; -static struct dma_pool *g_dma_pool; -static unsigned int g_use_36bit_addrs = 0; -static unsigned int g_fragments_size; -static char *g_fragments_base; -static char *g_free_fragments; -static struct semaphore g_free_fragments_sema; -static struct device *g_dev; -static struct device *g_dma_dev; - -static DEFINE_SEMAPHORE(g_free_fragments_mutex); - -static irqreturn_t -vchiq_doorbell_irq(int irq, void *dev_id); - -static struct vchiq_pagelist_info * -create_pagelist(char *buf, char __user *ubuf, size_t count, unsigned short type); - -static void -free_pagelist(struct vchiq_pagelist_info *pagelistinfo, - int actual); - -int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state) -{ - struct device *dev = &pdev->dev; - struct device *dma_dev = NULL; - struct vchiq_drvdata *drvdata = platform_get_drvdata(pdev); - struct rpi_firmware *fw = drvdata->fw; - struct vchiq_slot_zero *vchiq_slot_zero; - void *slot_mem; - dma_addr_t slot_phys; - u32 channelbase; - int slot_mem_size, frag_mem_size; - int err, irq, i; - - /* - * VCHI messages between the CPU and firmware use - * 32-bit bus addresses. - */ - err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); - - if (err < 0) - return err; - - g_cache_line_size = drvdata->cache_line_size; - g_fragments_size = 2 * g_cache_line_size; - - if (drvdata->use_36bit_addrs) { - struct device_node *dma_node = - of_find_compatible_node(NULL, NULL, "brcm,bcm2711-dma"); - - if (dma_node) { - struct platform_device *pdev; - - pdev = of_find_device_by_node(dma_node); - if (pdev) - dma_dev = &pdev->dev; - of_node_put(dma_node); - g_use_36bit_addrs = true; - } else { - dev_err(dev, "40-bit DMA controller not found\n"); - return -EINVAL; - } - } - - /* Allocate space for the channels in coherent memory */ - slot_mem_size = PAGE_ALIGN(TOTAL_SLOTS * VCHIQ_SLOT_SIZE); - frag_mem_size = PAGE_ALIGN(g_fragments_size * MAX_FRAGMENTS); - - slot_mem = dmam_alloc_coherent(dev, slot_mem_size + frag_mem_size, - &slot_phys, GFP_KERNEL); - if (!slot_mem) { - dev_err(dev, "could not allocate DMA memory\n"); - return -ENOMEM; - } - - WARN_ON(((unsigned long)slot_mem & (PAGE_SIZE - 1)) != 0); - channelbase = slot_phys; - - vchiq_slot_zero = vchiq_init_slots(slot_mem, slot_mem_size); - if (!vchiq_slot_zero) - return -EINVAL; - - vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_OFFSET_IDX] = - channelbase + slot_mem_size; - vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_COUNT_IDX] = - MAX_FRAGMENTS; - - g_fragments_base = (char *)slot_mem + slot_mem_size; - - g_free_fragments = g_fragments_base; - for (i = 0; i < (MAX_FRAGMENTS - 1); i++) { - *(char **)&g_fragments_base[i*g_fragments_size] = - &g_fragments_base[(i + 1)*g_fragments_size]; - } - *(char **)&g_fragments_base[i * g_fragments_size] = NULL; - sema_init(&g_free_fragments_sema, MAX_FRAGMENTS); - - if (vchiq_init_state(state, vchiq_slot_zero) != VCHIQ_SUCCESS) - return -EINVAL; - - g_regs = devm_platform_ioremap_resource(pdev, 0); - if (IS_ERR(g_regs)) - return PTR_ERR(g_regs); - - irq = platform_get_irq(pdev, 0); - if (irq <= 0) - return irq; - - err = devm_request_irq(dev, irq, vchiq_doorbell_irq, IRQF_IRQPOLL, - "VCHIQ doorbell", state); - if (err) { - dev_err(dev, "failed to register irq=%d\n", irq); - return err; - } - - /* Send the base address of the slots to VideoCore */ - err = rpi_firmware_property(fw, RPI_FIRMWARE_VCHIQ_INIT, - &channelbase, sizeof(channelbase)); - if (err || channelbase) { - dev_err(dev, "failed to set channelbase\n"); - return err ? : -ENXIO; - } - - g_dev = dev; - g_dma_dev = dma_dev ?: dev; - g_dma_pool = dmam_pool_create("vchiq_scatter_pool", dev, - VCHIQ_DMA_POOL_SIZE, g_cache_line_size, - 0); - if (!g_dma_pool) { - dev_err(dev, "failed to create dma pool"); - return -ENOMEM; - } - - vchiq_log_info(vchiq_arm_log_level, - "vchiq_init - done (slots %pK, phys %pad)", - vchiq_slot_zero, &slot_phys); - - vchiq_call_connected_callbacks(); - - return 0; -} - -enum vchiq_status -vchiq_platform_init_state(struct vchiq_state *state) -{ - enum vchiq_status status = VCHIQ_SUCCESS; - struct vchiq_2835_state *platform_state; - - state->platform_state = kzalloc(sizeof(*platform_state), GFP_KERNEL); - if (!state->platform_state) - return VCHIQ_ERROR; - - platform_state = (struct vchiq_2835_state *)state->platform_state; - - platform_state->inited = 1; - status = vchiq_arm_init_state(state, &platform_state->arm_state); - - if (status != VCHIQ_SUCCESS) - platform_state->inited = 0; - - return status; -} - -struct vchiq_arm_state* -vchiq_platform_get_arm_state(struct vchiq_state *state) -{ - struct vchiq_2835_state *platform_state; - - platform_state = (struct vchiq_2835_state *)state->platform_state; - - WARN_ON_ONCE(!platform_state->inited); - - return &platform_state->arm_state; -} - -void -remote_event_signal(struct remote_event *event) -{ - wmb(); - - event->fired = 1; - - dsb(sy); /* data barrier operation */ - - if (event->armed) - writel(0, g_regs + BELL2); /* trigger vc interrupt */ -} - -enum vchiq_status -vchiq_prepare_bulk_data(struct vchiq_bulk *bulk, void *offset, - void __user *uoffset, int size, int dir) -{ - struct vchiq_pagelist_info *pagelistinfo; - - pagelistinfo = create_pagelist(offset, uoffset, size, - (dir == VCHIQ_BULK_RECEIVE) - ? PAGELIST_READ - : PAGELIST_WRITE); - - if (!pagelistinfo) - return VCHIQ_ERROR; - - bulk->data = pagelistinfo->dma_addr; - - /* - * Store the pagelistinfo address in remote_data, - * which isn't used by the slave. - */ - bulk->remote_data = pagelistinfo; - - return VCHIQ_SUCCESS; -} - -void -vchiq_complete_bulk(struct vchiq_bulk *bulk) -{ - if (bulk && bulk->remote_data && bulk->actual) - free_pagelist((struct vchiq_pagelist_info *)bulk->remote_data, - bulk->actual); -} - -int vchiq_dump_platform_state(void *dump_context) -{ - char buf[80]; - int len; - - len = snprintf(buf, sizeof(buf), - " Platform: 2835 (VC master)"); - return vchiq_dump(dump_context, buf, len + 1); -} - -/* - * Local functions - */ - -static irqreturn_t -vchiq_doorbell_irq(int irq, void *dev_id) -{ - struct vchiq_state *state = dev_id; - irqreturn_t ret = IRQ_NONE; - unsigned int status; - - /* Read (and clear) the doorbell */ - status = readl(g_regs + BELL0); - - if (status & 0x4) { /* Was the doorbell rung? */ - remote_event_pollall(state); - ret = IRQ_HANDLED; - } - - return ret; -} - -static void -cleanup_pagelistinfo(struct vchiq_pagelist_info *pagelistinfo) -{ - if (pagelistinfo->scatterlist_mapped) { - dma_unmap_sg(g_dma_dev, pagelistinfo->scatterlist, - pagelistinfo->num_pages, pagelistinfo->dma_dir); - } - - if (pagelistinfo->pages_need_release) - unpin_user_pages(pagelistinfo->pages, pagelistinfo->num_pages); - - if (pagelistinfo->is_from_pool) { - dma_pool_free(g_dma_pool, pagelistinfo->pagelist, - pagelistinfo->dma_addr); - } else { - dma_free_coherent(g_dev, pagelistinfo->pagelist_buffer_size, - pagelistinfo->pagelist, - pagelistinfo->dma_addr); - } -} - -/* There is a potential problem with partial cache lines (pages?) - * at the ends of the block when reading. If the CPU accessed anything in - * the same line (page?) then it may have pulled old data into the cache, - * obscuring the new data underneath. We can solve this by transferring the - * partial cache lines separately, and allowing the ARM to copy into the - * cached area. - */ - -static struct vchiq_pagelist_info * -create_pagelist(char *buf, char __user *ubuf, - size_t count, unsigned short type) -{ - struct pagelist *pagelist; - struct vchiq_pagelist_info *pagelistinfo; - struct page **pages; - u32 *addrs; - unsigned int num_pages, offset, i, k; - int actual_pages; - bool is_from_pool; - size_t pagelist_size; - struct scatterlist *scatterlist, *sg; - int dma_buffers; - dma_addr_t dma_addr; - - if (count >= INT_MAX - PAGE_SIZE) - return NULL; - - if (buf) - offset = (uintptr_t)buf & (PAGE_SIZE - 1); - else - offset = (uintptr_t)ubuf & (PAGE_SIZE - 1); - num_pages = DIV_ROUND_UP(count + offset, PAGE_SIZE); - - if (num_pages > (SIZE_MAX - sizeof(struct pagelist) - - sizeof(struct vchiq_pagelist_info)) / - (sizeof(u32) + sizeof(pages[0]) + - sizeof(struct scatterlist))) - return NULL; - - pagelist_size = sizeof(struct pagelist) + - (num_pages * sizeof(u32)) + - (num_pages * sizeof(pages[0]) + - (num_pages * sizeof(struct scatterlist))) + - sizeof(struct vchiq_pagelist_info); - - /* Allocate enough storage to hold the page pointers and the page - * list - */ - if (pagelist_size > VCHIQ_DMA_POOL_SIZE) { - pagelist = dma_alloc_coherent(g_dev, - pagelist_size, - &dma_addr, - GFP_KERNEL); - is_from_pool = false; - } else { - pagelist = dma_pool_alloc(g_dma_pool, GFP_KERNEL, &dma_addr); - is_from_pool = true; - } - - vchiq_log_trace(vchiq_arm_log_level, "%s - %pK", __func__, pagelist); - - if (!pagelist) - return NULL; - - addrs = pagelist->addrs; - pages = (struct page **)(addrs + num_pages); - scatterlist = (struct scatterlist *)(pages + num_pages); - pagelistinfo = (struct vchiq_pagelist_info *) - (scatterlist + num_pages); - - pagelist->length = count; - pagelist->type = type; - pagelist->offset = offset; - - /* Populate the fields of the pagelistinfo structure */ - pagelistinfo->pagelist = pagelist; - pagelistinfo->pagelist_buffer_size = pagelist_size; - pagelistinfo->dma_addr = dma_addr; - pagelistinfo->is_from_pool = is_from_pool; - pagelistinfo->dma_dir = (type == PAGELIST_WRITE) ? - DMA_TO_DEVICE : DMA_FROM_DEVICE; - pagelistinfo->num_pages = num_pages; - pagelistinfo->pages_need_release = 0; - pagelistinfo->pages = pages; - pagelistinfo->scatterlist = scatterlist; - pagelistinfo->scatterlist_mapped = 0; - - if (buf) { - unsigned long length = count; - unsigned int off = offset; - - for (actual_pages = 0; actual_pages < num_pages; - actual_pages++) { - struct page *pg = - vmalloc_to_page((buf + - (actual_pages * PAGE_SIZE))); - size_t bytes = PAGE_SIZE - off; - - if (!pg) { - cleanup_pagelistinfo(pagelistinfo); - return NULL; - } - - if (bytes > length) - bytes = length; - pages[actual_pages] = pg; - length -= bytes; - off = 0; - } - /* do not try and release vmalloc pages */ - } else { - actual_pages = pin_user_pages_fast( - (unsigned long)ubuf & PAGE_MASK, - num_pages, - type == PAGELIST_READ, - pages); - - if (actual_pages != num_pages) { - vchiq_log_info(vchiq_arm_log_level, - "%s - only %d/%d pages locked", - __func__, actual_pages, num_pages); - - /* This is probably due to the process being killed */ - if (actual_pages > 0) - unpin_user_pages(pages, actual_pages); - cleanup_pagelistinfo(pagelistinfo); - return NULL; - } - /* release user pages */ - pagelistinfo->pages_need_release = 1; - } - - /* - * Initialize the scatterlist so that the magic cookie - * is filled if debugging is enabled - */ - sg_init_table(scatterlist, num_pages); - /* Now set the pages for each scatterlist */ - for (i = 0; i < num_pages; i++) { - unsigned int len = PAGE_SIZE - offset; - - if (len > count) - len = count; - sg_set_page(scatterlist + i, pages[i], len, offset); - offset = 0; - count -= len; - } - - dma_buffers = dma_map_sg(g_dma_dev, - scatterlist, - num_pages, - pagelistinfo->dma_dir); - - if (dma_buffers == 0) { - cleanup_pagelistinfo(pagelistinfo); - return NULL; - } - - pagelistinfo->scatterlist_mapped = 1; - - /* Combine adjacent blocks for performance */ - k = 0; - if (g_use_36bit_addrs) { - for_each_sg(scatterlist, sg, dma_buffers, i) { - u32 len = sg_dma_len(sg); - u64 addr = sg_dma_address(sg); - u32 page_id = (u32)((addr >> 4) & ~0xff); - u32 sg_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; - - /* Note: addrs is the address + page_count - 1 - * The firmware expects blocks after the first to be page- - * aligned and a multiple of the page size - */ - WARN_ON(len == 0); - WARN_ON(i && - (i != (dma_buffers - 1)) && (len & ~PAGE_MASK)); - WARN_ON(i && (addr & ~PAGE_MASK)); - WARN_ON(upper_32_bits(addr) > 0xf); - if (k > 0 && - ((addrs[k - 1] & ~0xff) + - (((addrs[k - 1] & 0xff) + 1) << 8) - == page_id)) { - u32 inc_pages = min(sg_pages, - 0xff - (addrs[k - 1] & 0xff)); - addrs[k - 1] += inc_pages; - page_id += inc_pages << 8; - sg_pages -= inc_pages; - } - while (sg_pages) { - u32 inc_pages = min(sg_pages, 0x100u); - addrs[k++] = page_id | (inc_pages - 1); - page_id += inc_pages << 8; - sg_pages -= inc_pages; - } - } - } else { - for_each_sg(scatterlist, sg, dma_buffers, i) { - u32 len = sg_dma_len(sg); - u32 addr = sg_dma_address(sg); - u32 new_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; - - /* Note: addrs is the address + page_count - 1 - * The firmware expects blocks after the first to be page- - * aligned and a multiple of the page size - */ - WARN_ON(len == 0); - WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK)); - WARN_ON(i && (addr & ~PAGE_MASK)); - if (k > 0 && - ((addrs[k - 1] & PAGE_MASK) + - (((addrs[k - 1] & ~PAGE_MASK) + 1) << PAGE_SHIFT)) - == (addr & PAGE_MASK)) - addrs[k - 1] += new_pages; - else - addrs[k++] = (addr & PAGE_MASK) | (new_pages - 1); - } - } - - /* Partial cache lines (fragments) require special measures */ - if ((type == PAGELIST_READ) && - ((pagelist->offset & (g_cache_line_size - 1)) || - ((pagelist->offset + pagelist->length) & - (g_cache_line_size - 1)))) { - char *fragments; - - if (down_interruptible(&g_free_fragments_sema)) { - cleanup_pagelistinfo(pagelistinfo); - return NULL; - } - - WARN_ON(!g_free_fragments); - - down(&g_free_fragments_mutex); - fragments = g_free_fragments; - WARN_ON(!fragments); - g_free_fragments = *(char **) g_free_fragments; - up(&g_free_fragments_mutex); - pagelist->type = PAGELIST_READ_WITH_FRAGMENTS + - (fragments - g_fragments_base) / g_fragments_size; - } - - return pagelistinfo; -} - -static void -free_pagelist(struct vchiq_pagelist_info *pagelistinfo, - int actual) -{ - struct pagelist *pagelist = pagelistinfo->pagelist; - struct page **pages = pagelistinfo->pages; - unsigned int num_pages = pagelistinfo->num_pages; - - vchiq_log_trace(vchiq_arm_log_level, "%s - %pK, %d", - __func__, pagelistinfo->pagelist, actual); - - /* - * NOTE: dma_unmap_sg must be called before the - * cpu can touch any of the data/pages. - */ - dma_unmap_sg(g_dma_dev, pagelistinfo->scatterlist, - pagelistinfo->num_pages, pagelistinfo->dma_dir); - pagelistinfo->scatterlist_mapped = 0; - - /* Deal with any partial cache lines (fragments) */ - if (pagelist->type >= PAGELIST_READ_WITH_FRAGMENTS) { - char *fragments = g_fragments_base + - (pagelist->type - PAGELIST_READ_WITH_FRAGMENTS) * - g_fragments_size; - int head_bytes, tail_bytes; - - head_bytes = (g_cache_line_size - pagelist->offset) & - (g_cache_line_size - 1); - tail_bytes = (pagelist->offset + actual) & - (g_cache_line_size - 1); - - if ((actual >= 0) && (head_bytes != 0)) { - if (head_bytes > actual) - head_bytes = actual; - - memcpy((char *)kmap(pages[0]) + - pagelist->offset, - fragments, - head_bytes); - kunmap(pages[0]); - } - if ((actual >= 0) && (head_bytes < actual) && - (tail_bytes != 0)) { - memcpy((char *)kmap(pages[num_pages - 1]) + - ((pagelist->offset + actual) & - (PAGE_SIZE - 1) & ~(g_cache_line_size - 1)), - fragments + g_cache_line_size, - tail_bytes); - kunmap(pages[num_pages - 1]); - } - - down(&g_free_fragments_mutex); - *(char **)fragments = g_free_fragments; - g_free_fragments = fragments; - up(&g_free_fragments_mutex); - up(&g_free_fragments_sema); - } - - /* Need to mark all the pages dirty. */ - if (pagelist->type != PAGELIST_WRITE && - pagelistinfo->pages_need_release) { - unsigned int i; - - for (i = 0; i < num_pages; i++) - set_page_dirty(pages[i]); - } - - cleanup_pagelistinfo(pagelistinfo); -} diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c index 06a19d6e2674..6e082ab29c1f 100644 --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c @@ -25,15 +25,32 @@ #include #include #include +#include +#include +#include #include #include "vchiq_core.h" #include "vchiq_ioctl.h" #include "vchiq_arm.h" #include "vchiq_debugfs.h" +#include "vchiq_connected.h" +#include "vchiq_pagelist.h" #define DEVICE_NAME "vchiq" +#define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32) + +#define MAX_FRAGMENTS (VCHIQ_NUM_CURRENT_BULKS * 2) + +#define VCHIQ_PLATFORM_FRAGMENTS_OFFSET_IDX 0 +#define VCHIQ_PLATFORM_FRAGMENTS_COUNT_IDX 1 + +#define BELL0 0x00 +#define BELL2 0x08 + +#define VCHIQ_DMA_POOL_SIZE PAGE_SIZE + /* Override the default prefix, which would be vchiq_arm (from the filename) */ #undef MODULE_PARAM_PREFIX #define MODULE_PARAM_PREFIX DEVICE_NAME "." @@ -67,10 +84,628 @@ static struct vchiq_drvdata bcm2711_drvdata = { .use_36bit_addrs = true, }; +struct vchiq_2835_state { + int inited; + struct vchiq_arm_state arm_state; +}; + +struct vchiq_pagelist_info { + struct pagelist *pagelist; + size_t pagelist_buffer_size; + dma_addr_t dma_addr; + bool is_from_pool; + enum dma_data_direction dma_dir; + unsigned int num_pages; + unsigned int pages_need_release; + struct page **pages; + struct scatterlist *scatterlist; + unsigned int scatterlist_mapped; +}; + +static void __iomem *g_regs; +/* This value is the size of the L2 cache lines as understood by the + * VPU firmware, which determines the required alignment of the + * offsets/sizes in pagelists. + * + * Modern VPU firmware looks for a DT "cache-line-size" property in + * the VCHIQ node and will overwrite it with the actual L2 cache size, + * which the kernel must then respect. That property was rejected + * upstream, so we have to use the VPU firmware's compatibility value + * of 32. + */ +static unsigned int g_cache_line_size = 32; +static struct dma_pool *g_dma_pool; +static unsigned int g_use_36bit_addrs = 0; +static unsigned int g_fragments_size; +static char *g_fragments_base; +static char *g_free_fragments; +static struct semaphore g_free_fragments_sema; +static struct device *g_dev; +static struct device *g_dma_dev; + +static DEFINE_SEMAPHORE(g_free_fragments_mutex); + +static irqreturn_t +vchiq_doorbell_irq(int irq, void *dev_id); + +static struct vchiq_pagelist_info * +create_pagelist(char *buf, char __user *ubuf, size_t count, unsigned short type); + +static void +free_pagelist(struct vchiq_pagelist_info *pagelistinfo, + int actual); + static enum vchiq_status vchiq_blocking_bulk_transfer(unsigned int handle, void *data, unsigned int size, enum vchiq_bulk_dir dir); +int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state) +{ + struct device *dev = &pdev->dev; + struct device *dma_dev = NULL; + struct vchiq_drvdata *drvdata = platform_get_drvdata(pdev); + struct rpi_firmware *fw = drvdata->fw; + struct vchiq_slot_zero *vchiq_slot_zero; + void *slot_mem; + dma_addr_t slot_phys; + u32 channelbase; + int slot_mem_size, frag_mem_size; + int err, irq, i; + + /* + * VCHI messages between the CPU and firmware use + * 32-bit bus addresses. + */ + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); + + if (err < 0) + return err; + + g_cache_line_size = drvdata->cache_line_size; + g_fragments_size = 2 * g_cache_line_size; + + if (drvdata->use_36bit_addrs) { + struct device_node *dma_node = + of_find_compatible_node(NULL, NULL, "brcm,bcm2711-dma"); + + if (dma_node) { + struct platform_device *pdev; + + pdev = of_find_device_by_node(dma_node); + if (pdev) + dma_dev = &pdev->dev; + of_node_put(dma_node); + g_use_36bit_addrs = true; + } else { + dev_err(dev, "40-bit DMA controller not found\n"); + return -EINVAL; + } + } + + /* Allocate space for the channels in coherent memory */ + slot_mem_size = PAGE_ALIGN(TOTAL_SLOTS * VCHIQ_SLOT_SIZE); + frag_mem_size = PAGE_ALIGN(g_fragments_size * MAX_FRAGMENTS); + + slot_mem = dmam_alloc_coherent(dev, slot_mem_size + frag_mem_size, + &slot_phys, GFP_KERNEL); + if (!slot_mem) { + dev_err(dev, "could not allocate DMA memory\n"); + return -ENOMEM; + } + + WARN_ON(((unsigned long)slot_mem & (PAGE_SIZE - 1)) != 0); + channelbase = slot_phys; + + vchiq_slot_zero = vchiq_init_slots(slot_mem, slot_mem_size); + if (!vchiq_slot_zero) + return -EINVAL; + + vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_OFFSET_IDX] = + channelbase + slot_mem_size; + vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_COUNT_IDX] = + MAX_FRAGMENTS; + + g_fragments_base = (char *)slot_mem + slot_mem_size; + + g_free_fragments = g_fragments_base; + for (i = 0; i < (MAX_FRAGMENTS - 1); i++) { + *(char **)&g_fragments_base[i*g_fragments_size] = + &g_fragments_base[(i + 1)*g_fragments_size]; + } + *(char **)&g_fragments_base[i * g_fragments_size] = NULL; + sema_init(&g_free_fragments_sema, MAX_FRAGMENTS); + + if (vchiq_init_state(state, vchiq_slot_zero) != VCHIQ_SUCCESS) + return -EINVAL; + + g_regs = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(g_regs)) + return PTR_ERR(g_regs); + + irq = platform_get_irq(pdev, 0); + if (irq <= 0) + return irq; + + err = devm_request_irq(dev, irq, vchiq_doorbell_irq, IRQF_IRQPOLL, + "VCHIQ doorbell", state); + if (err) { + dev_err(dev, "failed to register irq=%d\n", irq); + return err; + } + + /* Send the base address of the slots to VideoCore */ + err = rpi_firmware_property(fw, RPI_FIRMWARE_VCHIQ_INIT, + &channelbase, sizeof(channelbase)); + if (err || channelbase) { + dev_err(dev, "failed to set channelbase\n"); + return err ? : -ENXIO; + } + + g_dev = dev; + g_dma_dev = dma_dev ?: dev; + g_dma_pool = dmam_pool_create("vchiq_scatter_pool", dev, + VCHIQ_DMA_POOL_SIZE, g_cache_line_size, + 0); + if (!g_dma_pool) { + dev_err(dev, "failed to create dma pool"); + return -ENOMEM; + } + + vchiq_log_info(vchiq_arm_log_level, + "vchiq_init - done (slots %pK, phys %pad)", + vchiq_slot_zero, &slot_phys); + + vchiq_call_connected_callbacks(); + + return 0; +} + +enum vchiq_status +vchiq_platform_init_state(struct vchiq_state *state) +{ + enum vchiq_status status = VCHIQ_SUCCESS; + struct vchiq_2835_state *platform_state; + + state->platform_state = kzalloc(sizeof(*platform_state), GFP_KERNEL); + if (!state->platform_state) + return VCHIQ_ERROR; + + platform_state = (struct vchiq_2835_state *)state->platform_state; + + platform_state->inited = 1; + status = vchiq_arm_init_state(state, &platform_state->arm_state); + + if (status != VCHIQ_SUCCESS) + platform_state->inited = 0; + + return status; +} + +struct vchiq_arm_state* +vchiq_platform_get_arm_state(struct vchiq_state *state) +{ + struct vchiq_2835_state *platform_state; + + platform_state = (struct vchiq_2835_state *)state->platform_state; + + WARN_ON_ONCE(!platform_state->inited); + + return &platform_state->arm_state; +} + +void +remote_event_signal(struct remote_event *event) +{ + wmb(); + + event->fired = 1; + + dsb(sy); /* data barrier operation */ + + if (event->armed) + writel(0, g_regs + BELL2); /* trigger vc interrupt */ +} + +enum vchiq_status +vchiq_prepare_bulk_data(struct vchiq_bulk *bulk, void *offset, + void __user *uoffset, int size, int dir) +{ + struct vchiq_pagelist_info *pagelistinfo; + + pagelistinfo = create_pagelist(offset, uoffset, size, + (dir == VCHIQ_BULK_RECEIVE) + ? PAGELIST_READ + : PAGELIST_WRITE); + + if (!pagelistinfo) + return VCHIQ_ERROR; + + bulk->data = pagelistinfo->dma_addr; + + /* + * Store the pagelistinfo address in remote_data, + * which isn't used by the slave. + */ + bulk->remote_data = pagelistinfo; + + return VCHIQ_SUCCESS; +} + +void +vchiq_complete_bulk(struct vchiq_bulk *bulk) +{ + if (bulk && bulk->remote_data && bulk->actual) + free_pagelist((struct vchiq_pagelist_info *)bulk->remote_data, + bulk->actual); +} + +int vchiq_dump_platform_state(void *dump_context) +{ + char buf[80]; + int len; + + len = snprintf(buf, sizeof(buf), + " Platform: 2835 (VC master)"); + return vchiq_dump(dump_context, buf, len + 1); +} + +/* + * Local functions + */ + +static irqreturn_t +vchiq_doorbell_irq(int irq, void *dev_id) +{ + struct vchiq_state *state = dev_id; + irqreturn_t ret = IRQ_NONE; + unsigned int status; + + /* Read (and clear) the doorbell */ + status = readl(g_regs + BELL0); + + if (status & 0x4) { /* Was the doorbell rung? */ + remote_event_pollall(state); + ret = IRQ_HANDLED; + } + + return ret; +} + +static void +cleanup_pagelistinfo(struct vchiq_pagelist_info *pagelistinfo) +{ + if (pagelistinfo->scatterlist_mapped) { + dma_unmap_sg(g_dma_dev, pagelistinfo->scatterlist, + pagelistinfo->num_pages, pagelistinfo->dma_dir); + } + + if (pagelistinfo->pages_need_release) + unpin_user_pages(pagelistinfo->pages, pagelistinfo->num_pages); + + if (pagelistinfo->is_from_pool) { + dma_pool_free(g_dma_pool, pagelistinfo->pagelist, + pagelistinfo->dma_addr); + } else { + dma_free_coherent(g_dev, pagelistinfo->pagelist_buffer_size, + pagelistinfo->pagelist, + pagelistinfo->dma_addr); + } +} + +/* There is a potential problem with partial cache lines (pages?) + * at the ends of the block when reading. If the CPU accessed anything in + * the same line (page?) then it may have pulled old data into the cache, + * obscuring the new data underneath. We can solve this by transferring the + * partial cache lines separately, and allowing the ARM to copy into the + * cached area. + */ + +static struct vchiq_pagelist_info * +create_pagelist(char *buf, char __user *ubuf, + size_t count, unsigned short type) +{ + struct pagelist *pagelist; + struct vchiq_pagelist_info *pagelistinfo; + struct page **pages; + u32 *addrs; + unsigned int num_pages, offset, i, k; + int actual_pages; + bool is_from_pool; + size_t pagelist_size; + struct scatterlist *scatterlist, *sg; + int dma_buffers; + dma_addr_t dma_addr; + + if (count >= INT_MAX - PAGE_SIZE) + return NULL; + + if (buf) + offset = (uintptr_t)buf & (PAGE_SIZE - 1); + else + offset = (uintptr_t)ubuf & (PAGE_SIZE - 1); + num_pages = DIV_ROUND_UP(count + offset, PAGE_SIZE); + + if (num_pages > (SIZE_MAX - sizeof(struct pagelist) - + sizeof(struct vchiq_pagelist_info)) / + (sizeof(u32) + sizeof(pages[0]) + + sizeof(struct scatterlist))) + return NULL; + + pagelist_size = sizeof(struct pagelist) + + (num_pages * sizeof(u32)) + + (num_pages * sizeof(pages[0]) + + (num_pages * sizeof(struct scatterlist))) + + sizeof(struct vchiq_pagelist_info); + + /* Allocate enough storage to hold the page pointers and the page + * list + */ + if (pagelist_size > VCHIQ_DMA_POOL_SIZE) { + pagelist = dma_alloc_coherent(g_dev, + pagelist_size, + &dma_addr, + GFP_KERNEL); + is_from_pool = false; + } else { + pagelist = dma_pool_alloc(g_dma_pool, GFP_KERNEL, &dma_addr); + is_from_pool = true; + } + + vchiq_log_trace(vchiq_arm_log_level, "%s - %pK", __func__, pagelist); + + if (!pagelist) + return NULL; + + addrs = pagelist->addrs; + pages = (struct page **)(addrs + num_pages); + scatterlist = (struct scatterlist *)(pages + num_pages); + pagelistinfo = (struct vchiq_pagelist_info *) + (scatterlist + num_pages); + + pagelist->length = count; + pagelist->type = type; + pagelist->offset = offset; + + /* Populate the fields of the pagelistinfo structure */ + pagelistinfo->pagelist = pagelist; + pagelistinfo->pagelist_buffer_size = pagelist_size; + pagelistinfo->dma_addr = dma_addr; + pagelistinfo->is_from_pool = is_from_pool; + pagelistinfo->dma_dir = (type == PAGELIST_WRITE) ? + DMA_TO_DEVICE : DMA_FROM_DEVICE; + pagelistinfo->num_pages = num_pages; + pagelistinfo->pages_need_release = 0; + pagelistinfo->pages = pages; + pagelistinfo->scatterlist = scatterlist; + pagelistinfo->scatterlist_mapped = 0; + + if (buf) { + unsigned long length = count; + unsigned int off = offset; + + for (actual_pages = 0; actual_pages < num_pages; + actual_pages++) { + struct page *pg = + vmalloc_to_page((buf + + (actual_pages * PAGE_SIZE))); + size_t bytes = PAGE_SIZE - off; + + if (!pg) { + cleanup_pagelistinfo(pagelistinfo); + return NULL; + } + + if (bytes > length) + bytes = length; + pages[actual_pages] = pg; + length -= bytes; + off = 0; + } + /* do not try and release vmalloc pages */ + } else { + actual_pages = pin_user_pages_fast( + (unsigned long)ubuf & PAGE_MASK, + num_pages, + type == PAGELIST_READ, + pages); + + if (actual_pages != num_pages) { + vchiq_log_info(vchiq_arm_log_level, + "%s - only %d/%d pages locked", + __func__, actual_pages, num_pages); + + /* This is probably due to the process being killed */ + if (actual_pages > 0) + unpin_user_pages(pages, actual_pages); + cleanup_pagelistinfo(pagelistinfo); + return NULL; + } + /* release user pages */ + pagelistinfo->pages_need_release = 1; + } + + /* + * Initialize the scatterlist so that the magic cookie + * is filled if debugging is enabled + */ + sg_init_table(scatterlist, num_pages); + /* Now set the pages for each scatterlist */ + for (i = 0; i < num_pages; i++) { + unsigned int len = PAGE_SIZE - offset; + + if (len > count) + len = count; + sg_set_page(scatterlist + i, pages[i], len, offset); + offset = 0; + count -= len; + } + + dma_buffers = dma_map_sg(g_dma_dev, + scatterlist, + num_pages, + pagelistinfo->dma_dir); + + if (dma_buffers == 0) { + cleanup_pagelistinfo(pagelistinfo); + return NULL; + } + + pagelistinfo->scatterlist_mapped = 1; + + /* Combine adjacent blocks for performance */ + k = 0; + if (g_use_36bit_addrs) { + for_each_sg(scatterlist, sg, dma_buffers, i) { + u32 len = sg_dma_len(sg); + u64 addr = sg_dma_address(sg); + u32 page_id = (u32)((addr >> 4) & ~0xff); + u32 sg_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; + + /* Note: addrs is the address + page_count - 1 + * The firmware expects blocks after the first to be page- + * aligned and a multiple of the page size + */ + WARN_ON(len == 0); + WARN_ON(i && + (i != (dma_buffers - 1)) && (len & ~PAGE_MASK)); + WARN_ON(i && (addr & ~PAGE_MASK)); + WARN_ON(upper_32_bits(addr) > 0xf); + if (k > 0 && + ((addrs[k - 1] & ~0xff) + + (((addrs[k - 1] & 0xff) + 1) << 8) + == page_id)) { + u32 inc_pages = min(sg_pages, + 0xff - (addrs[k - 1] & 0xff)); + addrs[k - 1] += inc_pages; + page_id += inc_pages << 8; + sg_pages -= inc_pages; + } + while (sg_pages) { + u32 inc_pages = min(sg_pages, 0x100u); + addrs[k++] = page_id | (inc_pages - 1); + page_id += inc_pages << 8; + sg_pages -= inc_pages; + } + } + } else { + for_each_sg(scatterlist, sg, dma_buffers, i) { + u32 len = sg_dma_len(sg); + u32 addr = sg_dma_address(sg); + u32 new_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; + + /* Note: addrs is the address + page_count - 1 + * The firmware expects blocks after the first to be page- + * aligned and a multiple of the page size + */ + WARN_ON(len == 0); + WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK)); + WARN_ON(i && (addr & ~PAGE_MASK)); + if (k > 0 && + ((addrs[k - 1] & PAGE_MASK) + + (((addrs[k - 1] & ~PAGE_MASK) + 1) << PAGE_SHIFT)) + == (addr & PAGE_MASK)) + addrs[k - 1] += new_pages; + else + addrs[k++] = (addr & PAGE_MASK) | (new_pages - 1); + } + } + + /* Partial cache lines (fragments) require special measures */ + if ((type == PAGELIST_READ) && + ((pagelist->offset & (g_cache_line_size - 1)) || + ((pagelist->offset + pagelist->length) & + (g_cache_line_size - 1)))) { + char *fragments; + + if (down_interruptible(&g_free_fragments_sema)) { + cleanup_pagelistinfo(pagelistinfo); + return NULL; + } + + WARN_ON(!g_free_fragments); + + down(&g_free_fragments_mutex); + fragments = g_free_fragments; + WARN_ON(!fragments); + g_free_fragments = *(char **) g_free_fragments; + up(&g_free_fragments_mutex); + pagelist->type = PAGELIST_READ_WITH_FRAGMENTS + + (fragments - g_fragments_base) / g_fragments_size; + } + + return pagelistinfo; +} + +static void +free_pagelist(struct vchiq_pagelist_info *pagelistinfo, + int actual) +{ + struct pagelist *pagelist = pagelistinfo->pagelist; + struct page **pages = pagelistinfo->pages; + unsigned int num_pages = pagelistinfo->num_pages; + + vchiq_log_trace(vchiq_arm_log_level, "%s - %pK, %d", + __func__, pagelistinfo->pagelist, actual); + + /* + * NOTE: dma_unmap_sg must be called before the + * cpu can touch any of the data/pages. + */ + dma_unmap_sg(g_dma_dev, pagelistinfo->scatterlist, + pagelistinfo->num_pages, pagelistinfo->dma_dir); + pagelistinfo->scatterlist_mapped = 0; + + /* Deal with any partial cache lines (fragments) */ + if (pagelist->type >= PAGELIST_READ_WITH_FRAGMENTS) { + char *fragments = g_fragments_base + + (pagelist->type - PAGELIST_READ_WITH_FRAGMENTS) * + g_fragments_size; + int head_bytes, tail_bytes; + + head_bytes = (g_cache_line_size - pagelist->offset) & + (g_cache_line_size - 1); + tail_bytes = (pagelist->offset + actual) & + (g_cache_line_size - 1); + + if ((actual >= 0) && (head_bytes != 0)) { + if (head_bytes > actual) + head_bytes = actual; + + memcpy((char *)kmap(pages[0]) + + pagelist->offset, + fragments, + head_bytes); + kunmap(pages[0]); + } + if ((actual >= 0) && (head_bytes < actual) && + (tail_bytes != 0)) { + memcpy((char *)kmap(pages[num_pages - 1]) + + ((pagelist->offset + actual) & + (PAGE_SIZE - 1) & ~(g_cache_line_size - 1)), + fragments + g_cache_line_size, + tail_bytes); + kunmap(pages[num_pages - 1]); + } + + down(&g_free_fragments_mutex); + *(char **)fragments = g_free_fragments; + g_free_fragments = fragments; + up(&g_free_fragments_mutex); + up(&g_free_fragments_sema); + } + + /* Need to mark all the pages dirty. */ + if (pagelist->type != PAGELIST_WRITE && + pagelistinfo->pages_need_release) { + unsigned int i; + + for (i = 0; i < num_pages; i++) + set_page_dirty(pages[i]); + } + + cleanup_pagelistinfo(pagelistinfo); +} + #define VCHIQ_INIT_RETRIES 10 enum vchiq_status vchiq_initialise(struct vchiq_instance **instance_out) {