[v3] fpga manager: Add Altera CvP driver

Message ID	1492700308-10511-1-git-send-email-agust@denx.de (mailing list archive)
State	Superseded, archived
Headers	show Return-Path: <linux-fpga-owner@kernel.org> From: Anatolij Gustschin <agust@denx.de> To: linux-fpga@vger.kernel.org Cc: Alan Tull <atull@opensource.altera.com>, matthew.gerlach@linux.intel.com, Yi Li <yi1.li@linux.intel.com>, Moritz Fischer <moritz.fischer@ettus.com>, linux-kernel@vger.kernel.org Subject: [PATCH v3] fpga manager: Add Altera CvP driver Date: Thu, 20 Apr 2017 16:58:28 +0200 Message-Id: <1492700308-10511-1-git-send-email-agust@denx.de> Sender: linux-fpga-owner@vger.kernel.org Precedence: bulk

Anatolij Gustschin April 20, 2017, 2:58 p.m. UTC

Add FPGA manager driver for loading Arria/Cyclone/Stratix
FPGAs via CvP.

Signed-off-by: Anatolij Gustschin <agust@denx.de>
---
Changes in v3:

  - removed V-series from description (since the driver works
    also with Arria-10). Also renamed functions, config option
    and driver file name. Changed module description in Kconfig

  - dropped 'firmware=' module option (does not allow to bypass
    the fpga framework any more)

  - fixed build warning with newer gcc

  - changed to use hex numbers for PCI bus/device in registered
    fpga manager name. Use more generic fpga manager name (no
    V-series in the name any more)

  - moved steps 16 to 18 from teardown func to write_complete()
    as suggested by Yi

  - rebased on linux-next and tested with Arria-10/Stratix-V
    devices

Changes in v2:

  - rebase for v4.10, change subject ("Stratix V" to "V series")
    and add GPL header

  - use BIT() in register bit macro definitions

  - change .state() to return status or FPGA_MGR_STATE_UNKNOWN

  - use unique name for registered FPGA manager (append "@B:D.F")

  - use PCI_ANY_ID for device ID, users can set custom ID using
    new_id sysfs interface

  - remove map_base/map_len init and use pci_iomap() instead

  - simplify to use pci_read_config_word() instead of
    pci_bus_read_config_word()

  - run fpga_mgr_put() after usage, so the module can be removed
    without unbinding the driver

  - access CVP_STATUS as a 32-bit register, update CVP_STATUS
    bits macros

  - check CONFIG_READY bit in write_init and perform a teardown
    if this bit is set. Move teardown code to separate function
    as suggested

  - change function names to altera_v_*() as we can use
    the driver with other V-series FPGAs. Also change
    the driver file and Kconfig option names accordingly

  - pad written firmware data if the file size is not a multiple of 4

  - when config error checking is enabled, run checks after a 4KiB
    chunk is written

  - remove single built-in firmware string, add module parameter
    to allow setting default firmware for multiple cards as a
    Bus:Device.Function specific strings (optional)

  - remove polling of non-existing data bits DATA_ENC, DATA_COMP.
    Instead, add module parameter for bitstream specific clock to
    data ratio setting

 drivers/fpga/Kconfig      |   7 +
 drivers/fpga/Makefile     |   1 +
 drivers/fpga/altera-cvp.c | 587 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 595 insertions(+)
 create mode 100644 drivers/fpga/altera-cvp.c

Matthew Gerlach April 20, 2017, 5:29 p.m. UTC | #1

On Thu, 20 Apr 2017, Anatolij Gustschin wrote:

> Add FPGA manager driver for loading Arria/Cyclone/Stratix
> FPGAs via CvP.
>
> Signed-off-by: Anatolij Gustschin <agust@denx.de>
> ---

Hi Anatolij,

Since you say the driver works with Arria-10, I thought I would give it 
a try with the Altera Arria10 PCIe DevKit I am using.  I successfully
compiled your patch as an out of tree module against a 3.10 kernel.  The 
module successfully loaded and created instances for both boards in the 
host.

Now that I have the driver instances running, I'm not sure howto actually 
perform CvP.  Do you use a debugfs interface or something else?  Do you 
use the sof or an rbf file?

Thanks,
Matthew Gerlach

> Changes in v3:
>
>  - removed V-series from description (since the driver works
>    also with Arria-10). Also renamed functions, config option
>    and driver file name. Changed module description in Kconfig
>
>  - dropped 'firmware=' module option (does not allow to bypass
>    the fpga framework any more)
>
>  - fixed build warning with newer gcc
>
>  - changed to use hex numbers for PCI bus/device in registered
>    fpga manager name. Use more generic fpga manager name (no
>    V-series in the name any more)
>
>  - moved steps 16 to 18 from teardown func to write_complete()
>    as suggested by Yi
>
>  - rebased on linux-next and tested with Arria-10/Stratix-V
>    devices
>
> Changes in v2:
>
>  - rebase for v4.10, change subject ("Stratix V" to "V series")
>    and add GPL header
>
>  - use BIT() in register bit macro definitions
>
>  - change .state() to return status or FPGA_MGR_STATE_UNKNOWN
>
>  - use unique name for registered FPGA manager (append "@B:D.F")
>
>  - use PCI_ANY_ID for device ID, users can set custom ID using
>    new_id sysfs interface
>
>  - remove map_base/map_len init and use pci_iomap() instead
>
>  - simplify to use pci_read_config_word() instead of
>    pci_bus_read_config_word()
>
>  - run fpga_mgr_put() after usage, so the module can be removed
>    without unbinding the driver
>
>  - access CVP_STATUS as a 32-bit register, update CVP_STATUS
>    bits macros
>
>  - check CONFIG_READY bit in write_init and perform a teardown
>    if this bit is set. Move teardown code to separate function
>    as suggested
>
>  - change function names to altera_v_*() as we can use
>    the driver with other V-series FPGAs. Also change
>    the driver file and Kconfig option names accordingly
>
>  - pad written firmware data if the file size is not a multiple of 4
>
>  - when config error checking is enabled, run checks after a 4KiB
>    chunk is written
>
>  - remove single built-in firmware string, add module parameter
>    to allow setting default firmware for multiple cards as a
>    Bus:Device.Function specific strings (optional)
>
>  - remove polling of non-existing data bits DATA_ENC, DATA_COMP.
>    Instead, add module parameter for bitstream specific clock to
>    data ratio setting
>
> drivers/fpga/Kconfig      |   7 +
> drivers/fpga/Makefile     |   1 +
> drivers/fpga/altera-cvp.c | 587 ++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 595 insertions(+)
> create mode 100644 drivers/fpga/altera-cvp.c
>
> diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
> index 161ba9d..be48c2e 100644
> --- a/drivers/fpga/Kconfig
> +++ b/drivers/fpga/Kconfig
> @@ -26,6 +26,13 @@ config FPGA_MGR_ICE40_SPI
> 	help
> 	  FPGA manager driver support for Lattice iCE40 FPGAs over SPI.
>
> +config FPGA_MGR_ALTERA_CVP
> +	tristate "Altera Arria/Cyclone/Stratix CvP FPGA Manager"
> +	depends on PCI
> +	help
> +	  FPGA manager driver support for Altera FPGAs using the
> +	  CvP interface over PCIe.
> +
> config FPGA_MGR_SOCFPGA
> 	tristate "Altera SOCFPGA FPGA Manager"
> 	depends on ARCH_SOCFPGA || COMPILE_TEST
> diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile
> index 2a4f021..2e5c8b6 100644
> --- a/drivers/fpga/Makefile
> +++ b/drivers/fpga/Makefile
> @@ -6,6 +6,7 @@
> obj-$(CONFIG_FPGA)			+= fpga-mgr.o
>
> # FPGA Manager Drivers
> +obj-$(CONFIG_FPGA_MGR_ALTERA_CVP)	+= altera-cvp.o
> obj-$(CONFIG_FPGA_MGR_ICE40_SPI)	+= ice40-spi.o
> obj-$(CONFIG_FPGA_MGR_SOCFPGA)		+= socfpga.o
> obj-$(CONFIG_FPGA_MGR_SOCFPGA_A10)	+= socfpga-a10.o
> diff --git a/drivers/fpga/altera-cvp.c b/drivers/fpga/altera-cvp.c
> new file mode 100644
> index 0000000..b146033
> --- /dev/null
> +++ b/drivers/fpga/altera-cvp.c
> @@ -0,0 +1,587 @@
> +/*
> + * FPGA Manager Driver for Altera Arria/Cyclone/Stratix CvP
> + *
> + * Copyright (C) 2017 DENX Software Engineering
> + *
> + * Anatolij Gustschin <agust@denx.de>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; version 2 of the License.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * Manage Altera FPGA firmware using PCIe CvP.
> + * Firmware must be in binary "rbf" format.
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/fpga/fpga-mgr.h>
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include <linux/sizes.h>
> +
> +#define CVP_BAR		0	/* BAR used for data transfer in memory mode */
> +#define CVP_DUMMY_WR	244	/* dummy writes to clear CvP state machine */
> +#define TIMEOUT_IN_US	2000
> +
> +/* Vendor Specific Extended Capability Offset */
> +#define VSEC_OFFSET			0x200
> +#define VSEC_PCIE_EXT_CAP_ID_VAL	0x000b
> +
> +/* offsets from VSEC_OFFSET */
> +#define VSEC_PCIE_EXT_CAP_ID		(VSEC_OFFSET + 0x00)	/* 16bit */
> +#define VSEC_VERSION			0x02		/* 8bit */
> +#define VSEC_NEXT_CAP_OFF		0x03		/* 8bit */
> +#define VSEC_ID				0x04		/* 16bit */
> +#define VSEC_REV			0x06		/* 8bit */
> +#define VSEC_LENGTH			0x07		/* 8bit */
> +#define VSEC_ALTERA_MARKER		0x08		/* 32bit */
> +
> +#define VSEC_CVP_STATUS			(VSEC_OFFSET + 0x1c)	/* 32bit */
> +#define VSEC_CVP_STATUS_DATA_ENC	BIT(16)	/* is treated as encrypted */
> +#define VSEC_CVP_STATUS_DATA_COMP	BIT(17)	/* is treated as compressed */
> +#define VSEC_CVP_STATUS_CFG_RDY		BIT(18)	/* CVP_CONFIG_READY */
> +#define VSEC_CVP_STATUS_CFG_ERR		BIT(19)	/* CVP_CONFIG_ERROR */
> +#define VSEC_CVP_STATUS_CVP_EN		BIT(20)	/* ctrl block is enabling CVP */
> +#define VSEC_CVP_STATUS_USERMODE	BIT(21)	/* USERMODE */
> +#define VSEC_CVP_STATUS_CFG_DONE	BIT(23)	/* CVP_CONFIG_DONE */
> +#define VSEC_CVP_STATUS_PLD_CLK_IN_USE	BIT(24)	/* PLD_CLK_IN_USE */
> +
> +#define VSEC_CVP_MODE_CTRL		(VSEC_OFFSET + 0x20)	/* 32bit */
> +#define VSEC_CVP_MODE_CTRL_CVP_MODE	BIT(0) /* CVP (1) or normal mode (0) */
> +#define VSEC_CVP_MODE_CTRL_HIP_CLK_SEL	BIT(1) /* PMA (1) or fabric clock (0) */
> +#define VSEC_CVP_MODE_CTRL_FULL_CFG	BIT(2) /* CVP_FULLCONFIG */
> +#define VSEC_CVP_MODE_CTRL_NUMCLKS	(0xff<<8) /* CVP_NUMCLKS */
> +
> +#define VSEC_CVP_DATA			(VSEC_OFFSET + 0x28)	/* 32bit */
> +#define VSEC_CVP_PROG_CTRL		(VSEC_OFFSET + 0x2c)	/* 32bit */
> +#define VSEC_CVP_PROG_CTRL_CONFIG	BIT(0)
> +#define VSEC_CVP_PROG_CTRL_START_XFER	BIT(1)
> +
> +#define VSEC_UNCOR_ERR_STATUS		(VSEC_OFFSET + 0x34)	/* 32bit */
> +#define VSEC_UNCOR_ERR_MASK		(VSEC_OFFSET + 0x38)	/* 32bit */
> +#define	VSEC_UNCOR_ERR_CVP_CFG_ERR	BIT(5)	/* CVP_CONFIG_ERROR_LATCHED */
> +
> +#define	DRV_NAME		"altera-cvp"
> +#define ALTERA_CVP_MGR_NAME	"Altera CvP FPGA Manager"
> +
> +static int chkcfg; /* use value 1 for debugging only */
> +module_param(chkcfg, int, 0664);
> +MODULE_PARM_DESC(chkcfg, "1 - check CvP status, 0 - skip checking (default 0)");
> +
> +static int numclks = 1; /* default 1 for uncompressed and unencrypted */
> +module_param(numclks, int, 0664);
> +MODULE_PARM_DESC(numclks, "Clock to data ratio 1, 4 or 8 (default 1)");
> +
> +struct altera_cvp_conf {
> +	struct fpga_manager	*mgr;
> +	struct pci_dev		*pci_dev;
> +	void __iomem		*map;
> +	char			mgr_name[64];
> +};
> +
> +static inline void altera_cvp_chk_numclks(void)
> +{
> +	switch (numclks) {
> +	case 1:
> +	case 4:
> +	case 8:
> +		break;
> +	default:
> +		numclks = 1;
> +		break;
> +	}
> +}
> +
> +static enum fpga_mgr_states altera_cvp_state(struct fpga_manager *mgr)
> +{
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	u32 status;
> +
> +	pci_read_config_dword(conf->pci_dev, VSEC_CVP_STATUS, &status);
> +
> +	if (status & VSEC_CVP_STATUS_CFG_DONE)
> +		return FPGA_MGR_STATE_OPERATING;
> +
> +	if (status & VSEC_CVP_STATUS_CVP_EN)
> +		return FPGA_MGR_STATE_POWER_UP;
> +
> +	return FPGA_MGR_STATE_UNKNOWN;
> +}
> +
> +static int altera_cvp_teardown(struct fpga_manager *mgr,
> +			       struct fpga_image_info *info)
> +{
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	struct pci_dev *pdev = conf->pci_dev;
> +	int status = 0;
> +	int delay_us, i;
> +	u32 val32;
> +
> +	/*
> +	 * STEP 12 - reset START_XFER bit
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_PROG_CTRL, &val32);
> +	val32 &= ~VSEC_CVP_PROG_CTRL_START_XFER;
> +	pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +	/*
> +	 * STEP 13 - reset CVP_CONFIG bit
> +	 */
> +	val32 &= ~VSEC_CVP_PROG_CTRL_CONFIG;
> +	pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +	/*
> +	 * STEP 14
> +	 * - set CVP_NUMCLKS to 0x01 and then issue CVP_DUMMY_WR dummy
> +	 *   writes to the HIP
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +	/* set number of CVP clock cycles for every CVP Data Register Write */
> +	val32 |= 0x01 << 8;	/* 1 clock */
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	/* dummy memory write to switch from CVP clock to internal clock */
> +	val32 = 0xdeadbeef;
> +	if (conf->map) {
> +		u32 *map = conf->map;
> +
> +		for (i = 0; i < CVP_DUMMY_WR; i++)
> +			*map++ = val32; /* use MemoryWrite */
> +	} else {
> +		for (i = 0; i < CVP_DUMMY_WR; i++)
> +			pci_write_config_dword(pdev, VSEC_CVP_DATA, val32);
> +	}
> +
> +	/*
> +	 * STEP 15 - poll CVP_CONFIG_READY bit
> +	 */
> +	delay_us = 0;
> +	while (1) {
> +		pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +		if ((val32 & VSEC_CVP_STATUS_CFG_RDY) == 0)
> +			break;
> +
> +		udelay(1);	/* wait 1us */
> +
> +		if (delay_us++ > TIMEOUT_IN_US) {
> +			dev_warn(&mgr->dev, "CVP_CONFIG_READY == 0 timeout\n");
> +			status = -ETIMEDOUT;
> +			break;
> +		}
> +	}
> +
> +	return status;
> +}
> +
> +static int altera_cvp_write_init(struct fpga_manager *mgr,
> +				 struct fpga_image_info *info,
> +				 const char *buf, size_t count)
> +{
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	struct pci_dev *pdev = conf->pci_dev;
> +	int delay_us, i;
> +	u32 val32;
> +	int ret;
> +
> +	if (info && info->flags & FPGA_MGR_PARTIAL_RECONFIG) {
> +		dev_err(&mgr->dev, "Partial reconfiguration not supported.\n");
> +		return -EINVAL;
> +	}
> +
> +	/*
> +	 * STEP 1 - read CVP status and check CVP_EN flag
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +	if (!(val32 & VSEC_CVP_STATUS_CVP_EN)) {
> +		dev_err(&mgr->dev, "CVP mode off: 0x%04x\n", val32);
> +		return -ENODEV;
> +	}
> +
> +	if (val32 & VSEC_CVP_STATUS_CFG_RDY) {
> +		dev_warn(&mgr->dev, "CvP already started, teardown first\n");
> +		ret = altera_cvp_teardown(mgr, info);
> +		if (ret < 0)
> +			return ret;
> +	}
> +
> +	/*
> +	 * STEP 2
> +	 * - set HIP_CLK_SEL and CVP_MODE (must be set in the order mentioned)
> +	 */
> +	/* switch from fabric to PMA clock */
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 |= VSEC_CVP_MODE_CTRL_HIP_CLK_SEL;
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	/* set CVP mode */
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 |= VSEC_CVP_MODE_CTRL_CVP_MODE;
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	/*
> +	 * STEP 3
> +	 * - set CVP_NUMCLKS to 1 and issue CVP_DUMMY_WR dummy writes to the HIP
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +	val32 |= 0x01 << 8;	/* 1 clock */
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	/* dummy memory write to switch from internal clock to CVP clock */
> +	val32 = 0xdeadbeef;
> +	if (conf->map) {
> +		u32 *map = conf->map;
> +
> +		for (i = 0; i < CVP_DUMMY_WR; i++)
> +			*map++ = val32; /* use MemoryWrite */
> +	} else {
> +		for (i = 0; i < CVP_DUMMY_WR; i++)
> +			pci_write_config_dword(pdev, VSEC_CVP_DATA, val32);
> +	}
> +
> +	/*
> +	 * STEP 4 - set CVP_CONFIG bit
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_PROG_CTRL, &val32);
> +	/* request control block to begin transfer using CVP */
> +	val32 |= VSEC_CVP_PROG_CTRL_CONFIG;
> +	pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +	/*
> +	 * STEP 5 - poll CVP_CONFIG READY
> +	 */
> +	delay_us = 0;
> +	while (1) {
> +		pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +		if ((val32 & VSEC_CVP_STATUS_CFG_RDY) ==
> +		    VSEC_CVP_STATUS_CFG_RDY)
> +			break;
> +
> +		udelay(1); /* wait 1us */
> +
> +		if (delay_us++ > TIMEOUT_IN_US) {
> +			dev_warn(&mgr->dev, "CVP_CONFIG_READY == 1 timeout\n");
> +			return -ETIMEDOUT;
> +		}
> +	}
> +
> +	/*
> +	 * STEP 6
> +	 * - set CVP_NUMCLKS to 1 and issue CVP_DUMMY_WR dummy writes to the HIP
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +	val32 |= 0x01 << 8; /* 1 clock */
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	/* dummy memory write to switch from internal clock to CVP clock */
> +	val32 = 0xdeadbeef;
> +	if (conf->map) {
> +		u32 *map = conf->map;
> +
> +		for (i = 0; i < CVP_DUMMY_WR; i++)
> +			*map++ = val32; /* use MemoryWrite */
> +	} else {
> +		for (i = 0; i < CVP_DUMMY_WR; i++)
> +			pci_write_config_dword(pdev, VSEC_CVP_DATA, val32);
> +	}
> +
> +	/*
> +	 * STEP 7 - set START_XFER
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_PROG_CTRL, &val32);
> +	val32 |= VSEC_CVP_PROG_CTRL_START_XFER;
> +	pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +	/*
> +	 * STEP 8 - start transfer (set CVP_NUMCLKS)
> +	 */
> +	altera_cvp_chk_numclks();
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +	val32 |= numclks << 8; /* bitstream specific clock count */
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	return 0;
> +}
> +
> +static inline int altera_cvp_chk_error(struct fpga_manager *mgr, size_t bytes)
> +{
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	u32 val32;
> +
> +	/*
> +	 * STEP 10 (optional) - check CVP_CONFIG_ERROR flag
> +	 */
> +	pci_read_config_dword(conf->pci_dev, VSEC_CVP_STATUS, &val32);
> +	if (val32 & VSEC_CVP_STATUS_CFG_ERR) {
> +		dev_err(&mgr->dev, "CVP_CONFIG_ERROR after %zi bytes!\n",
> +			bytes);
> +		return -EPROTO;
> +	}
> +	return 0;
> +}
> +
> +static int altera_cvp_write(struct fpga_manager *mgr, const char *buf,
> +			    size_t count)
> +{
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	struct pci_dev *pdev = conf->pci_dev;
> +	const u32 *data;
> +	size_t bytes = 0;
> +	int status = 0;
> +	u32 mask;
> +	u32 *map;
> +
> +	/*
> +	 * STEP 9
> +	 * - write 32-bit configuration data from RBF file to CVP data register
> +	 */
> +	data = (u32 *)buf;
> +	map = conf->map;
> +	bytes = count;
> +
> +	if (map) {
> +		while (bytes >= 4) {
> +			*map = *data++;
> +			bytes -= 4;
> +
> +			/*
> +			 * STEP 10 (optional) and STEP 11
> +			 * - check error flag
> +			 * - loop until data transfer completed
> +			 */
> +			if (chkcfg && !(bytes % SZ_4K)) {
> +				size_t done = count - bytes;
> +
> +				status = altera_cvp_chk_error(mgr, done);
> +				if (status < 0)
> +					return status;
> +			}
> +		}
> +	} else {
> +		while (bytes >= 4) {
> +			pci_write_config_dword(pdev, VSEC_CVP_DATA, *data++);
> +			bytes -= 4;
> +
> +			if (chkcfg && !(bytes % SZ_4K)) {
> +				size_t done = count - bytes;
> +
> +				status = altera_cvp_chk_error(mgr, done);
> +				if (status < 0)
> +					return status;
> +			}
> +		}
> +	}
> +
> +	switch (bytes) {
> +	case 3:
> +		mask = 0x00ffffff;
> +		break;
> +	case 2:
> +		mask = 0x0000ffff;
> +		break;
> +	case 1:
> +		mask = 0x000000ff;
> +		break;
> +	case 0:
> +		mask = 0x00000000;
> +		break;
> +	}
> +
> +	if (mask) {
> +		u32 tail = *data & mask;
> +
> +		if (map)
> +			*map = tail;
> +		else
> +			pci_write_config_dword(pdev, VSEC_CVP_DATA, tail);
> +
> +		if (chkcfg)
> +			status = altera_cvp_chk_error(mgr, count);
> +	}
> +
> +	return status;
> +}
> +
> +static int altera_cvp_write_complete(struct fpga_manager *mgr,
> +				     struct fpga_image_info *info)
> +{
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	struct pci_dev *pdev = conf->pci_dev;
> +	int status;
> +	int delay_us;
> +	u32 status_msk;
> +	u32 val32;
> +
> +	status = altera_cvp_teardown(mgr, info);
> +	if (status < 0)
> +		return status;
> +
> +	/*
> +	 * STEP 16 - check CVP_CONFIG_ERROR_LATCHED bit
> +	 */
> +	pci_read_config_dword(pdev, VSEC_UNCOR_ERR_STATUS, &val32);
> +	if (val32 & VSEC_UNCOR_ERR_CVP_CFG_ERR) {
> +		dev_err(&mgr->dev, "detected CVP_CONFIG_ERROR_LATCHED!\n");
> +		return -EFAULT;
> +	}
> +
> +	/*
> +	 * STEP 17 - reset CVP_MODE and HIP_CLK_SEL bit
> +	 */
> +	pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +	val32 &= ~VSEC_CVP_MODE_CTRL_HIP_CLK_SEL;
> +	val32 &= ~VSEC_CVP_MODE_CTRL_CVP_MODE;
> +	pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +	/*
> +	 * STEP 18 - poll PLD_CLK_IN_USE and USER_MODE bits
> +	 */
> +	delay_us = 0;
> +	status_msk = VSEC_CVP_STATUS_PLD_CLK_IN_USE | VSEC_CVP_STATUS_USERMODE;
> +	while (1) {
> +		pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +		if ((val32 & status_msk) == status_msk)
> +			break;
> +
> +		udelay(1); /* wait 1us */
> +
> +		if (delay_us++ > TIMEOUT_IN_US) {
> +			dev_warn(&mgr->dev, "PLD_CLK_IN_USE|USERMODE timeout\n");
> +			status = -ETIMEDOUT;
> +			break;
> +		}
> +	}
> +
> +	return status;
> +}
> +
> +static const struct fpga_manager_ops altera_cvp_ops = {
> +	.state		= altera_cvp_state,
> +	.write_init	= altera_cvp_write_init,
> +	.write		= altera_cvp_write,
> +	.write_complete	= altera_cvp_write_complete,
> +};
> +
> +static int altera_cvp_probe(struct pci_dev *pdev,
> +			    const struct pci_device_id *dev_id)
> +{
> +	struct altera_cvp_conf *conf;
> +	u16 cmd, val16;
> +	int ret;
> +
> +	pci_read_config_word(pdev, VSEC_PCIE_EXT_CAP_ID, &val16);
> +	if (val16 != VSEC_PCIE_EXT_CAP_ID_VAL) {
> +		dev_err(&pdev->dev, "Wrong EXT_CAP_ID value 0x%x\n", val16);
> +		ret = -ENODEV;
> +		goto err;
> +	}
> +
> +	/* Enable memory BAR access */
> +	pci_read_config_word(pdev, PCI_COMMAND, &cmd);
> +	if (!(cmd & PCI_COMMAND_MEMORY)) {
> +		cmd |= PCI_COMMAND_MEMORY;
> +		pci_write_config_word(pdev, PCI_COMMAND, cmd);
> +	}
> +
> +	conf = devm_kzalloc(&pdev->dev, sizeof(*conf), GFP_KERNEL);
> +	if (!conf) {
> +		ret = -ENOMEM;
> +		goto err;
> +	}
> +
> +	conf->pci_dev = pdev;
> +
> +	if (pci_request_region(pdev, CVP_BAR, "CVP") < 0) {
> +		dev_err(&pdev->dev, "Requesting CVP BAR region failed\n");
> +		ret = -ENODEV;
> +		goto err;
> +	}
> +
> +	conf->map = pci_iomap(pdev, CVP_BAR, 0);
> +	if (!conf->map)
> +		dev_warn(&pdev->dev, "Mapping CVP BAR failed\n");
> +
> +	snprintf(conf->mgr_name, sizeof(conf->mgr_name), "%s @%02x:%02x.%d",
> +		 ALTERA_CVP_MGR_NAME, pdev->bus->number,
> +		 PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn));
> +
> +	ret = fpga_mgr_register(&pdev->dev, conf->mgr_name,
> +				&altera_cvp_ops, conf);
> +	if (ret)
> +		goto err_unmap;
> +
> +	conf->mgr = fpga_mgr_get(&pdev->dev);
> +	if (IS_ERR(conf->mgr)) {
> +		dev_err(&pdev->dev, "Getting fpga mgr reference failed\n");
> +		ret = -ENODEV;
> +		goto err_unreg;
> +	}
> +	fpga_mgr_put(conf->mgr);
> +
> +	return 0;
> +
> +err_unreg:
> +	fpga_mgr_unregister(&pdev->dev);
> +err_unmap:
> +	pci_iounmap(pdev, conf->map);
> +	pci_release_region(pdev, CVP_BAR);
> +err:
> +	cmd &= ~PCI_COMMAND_MEMORY;
> +	pci_write_config_word(pdev, PCI_COMMAND, cmd);
> +	return ret;
> +}
> +
> +static void altera_cvp_remove(struct pci_dev *pdev)
> +{
> +	struct fpga_manager *mgr = pci_get_drvdata(pdev);
> +	struct altera_cvp_conf *conf = mgr->priv;
> +	u16 cmd;
> +
> +	fpga_mgr_unregister(&pdev->dev);
> +	pci_iounmap(pdev, conf->map);
> +	pci_release_region(pdev, CVP_BAR);
> +	pci_read_config_word(pdev, PCI_COMMAND, &cmd);
> +	cmd &= ~PCI_COMMAND_MEMORY;
> +	pci_write_config_word(pdev, PCI_COMMAND, cmd);
> +}
> +
> +#define PCI_VENDOR_ID_ALTERA	0x1172
> +
> +static struct pci_device_id altera_cvp_id_tbl[] = {
> +	{ PCI_VDEVICE(ALTERA, PCI_ANY_ID) },
> +	{ }
> +};
> +
> +MODULE_DEVICE_TABLE(pci, altera_cvp_id_tbl);
> +
> +static struct pci_driver altera_cvp_driver = {
> +	.name   = DRV_NAME,
> +	.id_table = altera_cvp_id_tbl,
> +	.probe  = altera_cvp_probe,
> +	.remove = altera_cvp_remove,
> +};
> +
> +static int __init altera_cvp_init(void)
> +{
> +	return pci_register_driver(&altera_cvp_driver);
> +}
> +
> +static void __exit altera_cvp_exit(void)
> +{
> +	pci_unregister_driver(&altera_cvp_driver);
> +}
> +
> +module_init(altera_cvp_init);
> +module_exit(altera_cvp_exit);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_AUTHOR("Anatolij Gustschin <agust@denx.de>");
> +MODULE_DESCRIPTION("Module to load Altera FPGA over CvP");
> -- 
> 2.7.4
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anatolij Gustschin April 20, 2017, 5:47 p.m. UTC | #2

Hi Matthew,

On Thu, 20 Apr 2017 10:29:34 -0700 (PDT)
matthew.gerlach@linux.intel.com matthew.gerlach@linux.intel.com wrote:
...
>Since you say the driver works with Arria-10, I thought I would give it 
>a try with the Altera Arria10 PCIe DevKit I am using.  I successfully
>compiled your patch as an out of tree module against a 3.10 kernel.  The 
>module successfully loaded and created instances for both boards in the 
>host.

thanks for testing!

>Now that I have the driver instances running, I'm not sure howto actually 
>perform CvP.  Do you use a debugfs interface or something else?  Do you 
>use the sof or an rbf file?

I used the debugfs interface for first testing, now I'm using custom
fpga config interface driver, but it depends on many other drivers
and won't work without additional special hardware.

I use an rbf file. The v2 patch [1] used a module parameter to specify
firmware file name for a given PCIe device, but it is bypassing the
framework and is not acceptable for mainline.

Thanks,
Anatolij

[1] https://patchwork.kernel.org/patch/9582731
--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Li, Yi April 21, 2017, 9:14 p.m. UTC | #3

On 4/20/2017 12:29 PM, matthew.gerlach@linux.intel.com wrote:
>
>
> On Thu, 20 Apr 2017, Anatolij Gustschin wrote:
>
>> Add FPGA manager driver for loading Arria/Cyclone/Stratix
>> FPGAs via CvP.
>>
>> Signed-off-by: Anatolij Gustschin <agust@denx.de>
>> ---
>
> Hi Anatolij,
>
> Since you say the driver works with Arria-10, I thought I would give 
> it a try with the Altera Arria10 PCIe DevKit I am using.  I successfully
> compiled your patch as an out of tree module against a 3.10 kernel.  
> The module successfully loaded and created instances for both boards 
> in the host.
> Now that I have the driver instances running, I'm not sure howto 
> actually perform CvP.  Do you use a debugfs interface or something 
> else?  Do you use the sof or an rbf file?
>
> Thanks,
> Matthew Gerlach
>
>> Changes in v3:
>>
>>  - removed V-series from description (since the driver works
>>    also with Arria-10). Also renamed functions, config option
>>    and driver file name. Changed module description in Kconfig
 From the User guild 
https://www.altera.com/documentation/dsu1441819344145.html#dsu1442261652730, 
it says Configuration via Protocol (CvP) is a configuration scheme 
supported inArria^® V,Cyclone^® V,Stratix^® V, andArria^® 10device 
families. But I have seen different CvP registers bitmask defines 
somewhere else, so I am not positive that there is only one CvP protocol 
for all Altera FPGAs. Maybe we could put those Vs + Arria 10 as 
supported devices on the Kconfig file on the safe side?

>> ...
>>
>> diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
>> index 161ba9d..be48c2e 100644
>> --- a/drivers/fpga/Kconfig
>> +++ b/drivers/fpga/Kconfig
>> @@ -26,6 +26,13 @@ config FPGA_MGR_ICE40_SPI
>>     help
>>       FPGA manager driver support for Lattice iCE40 FPGAs over SPI.
>>
>> +config FPGA_MGR_ALTERA_CVP
>> +    tristate "Altera Arria/Cyclone/Stratix CvP FPGA Manager"
>> +    depends on PCI
>> +    help
>> +      FPGA manager driver support for Altera FPGAs using the
>> +      CvP interface over PCIe.
>> +
>> config FPGA_MGR_SOCFPGA
>>     tristate "Altera SOCFPGA FPGA Manager"
>>     depends on ARCH_SOCFPGA || COMPILE_TEST
>> diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile
>> index 2a4f021..2e5c8b6 100644
>> --- a/drivers/fpga/Makefile
>> +++ b/drivers/fpga/Makefile
>> @@ -6,6 +6,7 @@
>> obj-$(CONFIG_FPGA)            += fpga-mgr.o
...
--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Li, Yi April 25, 2017, 4:49 p.m. UTC | #4

hi Anatolij


On 04/21/2017 04:14 PM, Li, Yi wrote:
>
> On 4/20/2017 12:29 PM, matthew.gerlach@linux.intel.com wrote:
>>
>>
>> On Thu, 20 Apr 2017, Anatolij Gustschin wrote:
>>
>>> Add FPGA manager driver for loading Arria/Cyclone/Stratix
>>> FPGAs via CvP.
>>>
>>> Signed-off-by: Anatolij Gustschin <agust@denx.de>
>>> ---
>>
>> Hi Anatolij,
>>
>> Since you say the driver works with Arria-10, I thought I would give 
>> it a try with the Altera Arria10 PCIe DevKit I am using.  I successfully
>> compiled your patch as an out of tree module against a 3.10 kernel.  
>> The module successfully loaded and created instances for both boards 
>> in the host.
>> Now that I have the driver instances running, I'm not sure howto 
>> actually perform CvP.  Do you use a debugfs interface or something 
>> else?  Do you use the sof or an rbf file?
>>
>> Thanks,
>> Matthew Gerlach
>>
>>> Changes in v3:
>>>
>>>  - removed V-series from description (since the driver works
>>>    also with Arria-10). Also renamed functions, config option
>>>    and driver file name. Changed module description in Kconfig
> From the User guild 
> https://www.altera.com/documentation/dsu1441819344145.html#dsu1442261652730, 
> it says Configuration via Protocol (CvP) is a configuration scheme 
> supported inArria^® V,Cyclone^® V,Stratix^® V, andArria^® 10device 
> families. But I have seen different CvP registers bitmask defines 
> somewhere else, so I am not positive that there is only one CvP 
> protocol for all Altera FPGAs. Maybe we could put those Vs + Arria 10 
> as supported devices on the Kconfig file on the safe side?

Reviewed the rest of the driver and it looks good to me.

>
>>> ...
>>>
>>> diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
>>> index 161ba9d..be48c2e 100644
>>> --- a/drivers/fpga/Kconfig
>>> +++ b/drivers/fpga/Kconfig
>>> @@ -26,6 +26,13 @@ config FPGA_MGR_ICE40_SPI
>>>     help
>>>       FPGA manager driver support for Lattice iCE40 FPGAs over SPI.
>>>
>>> +config FPGA_MGR_ALTERA_CVP
>>> +    tristate "Altera Arria/Cyclone/Stratix CvP FPGA Manager"
>>> +    depends on PCI
>>> +    help
>>> +      FPGA manager driver support for Altera FPGAs using the
>>> +      CvP interface over PCIe.
>>> +
>>> config FPGA_MGR_SOCFPGA
>>>     tristate "Altera SOCFPGA FPGA Manager"
>>>     depends on ARCH_SOCFPGA || COMPILE_TEST
>>> diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile
>>> index 2a4f021..2e5c8b6 100644
>>> --- a/drivers/fpga/Makefile
>>> +++ b/drivers/fpga/Makefile
>>> @@ -6,6 +6,7 @@
>>> obj-$(CONFIG_FPGA)            += fpga-mgr.o
> ...

--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Anatolij Gustschin April 26, 2017, 4:36 p.m. UTC | #5

Hi,

On Fri, 21 Apr 2017 16:14:31 -0500
Li, Yi yi1.li@linux.intel.com wrote:

> From the User guild 
>https://www.altera.com/documentation/dsu1441819344145.html#dsu1442261652730, 
>it says Configuration via Protocol (CvP) is a configuration scheme 
>supported inArria^® V,Cyclone^® V,Stratix^® V, andArria^® 10device 
>families. But I have seen different CvP registers bitmask defines 
>somewhere else, so I am not positive that there is only one CvP protocol 
>for all Altera FPGAs. Maybe we could put those Vs + Arria 10 as 
>supported devices on the Kconfig file on the safe side?

yes, I can extend the description in Kconfig. Will wait for other
comments and then wil send v4 patch. Thanks!

Anatolij
--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Alan Tull April 26, 2017, 5:10 p.m. UTC | #6

On Thu, Apr 20, 2017 at 9:58 AM, Anatolij Gustschin <agust@denx.de> wrote:

Hi Anatolij,

> Add FPGA manager driver for loading Arria/Cyclone/Stratix
> FPGAs via CvP.
>
> Signed-off-by: Anatolij Gustschin <agust@denx.de>
> ---
> Changes in v3:
>
>   - removed V-series from description (since the driver works
>     also with Arria-10). Also renamed functions, config option
>     and driver file name. Changed module description in Kconfig
>
>   - dropped 'firmware=' module option (does not allow to bypass
>     the fpga framework any more)
>
>   - fixed build warning with newer gcc
>
>   - changed to use hex numbers for PCI bus/device in registered
>     fpga manager name. Use more generic fpga manager name (no
>     V-series in the name any more)
>
>   - moved steps 16 to 18 from teardown func to write_complete()
>     as suggested by Yi
>
>   - rebased on linux-next and tested with Arria-10/Stratix-V
>     devices
>
> Changes in v2:
>
>   - rebase for v4.10, change subject ("Stratix V" to "V series")
>     and add GPL header
>
>   - use BIT() in register bit macro definitions
>
>   - change .state() to return status or FPGA_MGR_STATE_UNKNOWN
>
>   - use unique name for registered FPGA manager (append "@B:D.F")
>
>   - use PCI_ANY_ID for device ID, users can set custom ID using
>     new_id sysfs interface
>
>   - remove map_base/map_len init and use pci_iomap() instead
>
>   - simplify to use pci_read_config_word() instead of
>     pci_bus_read_config_word()
>
>   - run fpga_mgr_put() after usage, so the module can be removed
>     without unbinding the driver
>
>   - access CVP_STATUS as a 32-bit register, update CVP_STATUS
>     bits macros
>
>   - check CONFIG_READY bit in write_init and perform a teardown
>     if this bit is set. Move teardown code to separate function
>     as suggested
>
>   - change function names to altera_v_*() as we can use
>     the driver with other V-series FPGAs. Also change
>     the driver file and Kconfig option names accordingly
>
>   - pad written firmware data if the file size is not a multiple of 4
>
>   - when config error checking is enabled, run checks after a 4KiB
>     chunk is written
>
>   - remove single built-in firmware string, add module parameter
>     to allow setting default firmware for multiple cards as a
>     Bus:Device.Function specific strings (optional)
>
>   - remove polling of non-existing data bits DATA_ENC, DATA_COMP.
>     Instead, add module parameter for bitstream specific clock to
>     data ratio setting
>
>  drivers/fpga/Kconfig      |   7 +
>  drivers/fpga/Makefile     |   1 +
>  drivers/fpga/altera-cvp.c | 587 ++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 595 insertions(+)
>  create mode 100644 drivers/fpga/altera-cvp.c
>
> diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
> index 161ba9d..be48c2e 100644
> --- a/drivers/fpga/Kconfig
> +++ b/drivers/fpga/Kconfig
> @@ -26,6 +26,13 @@ config FPGA_MGR_ICE40_SPI
>         help
>           FPGA manager driver support for Lattice iCE40 FPGAs over SPI.
>
> +config FPGA_MGR_ALTERA_CVP
> +       tristate "Altera Arria/Cyclone/Stratix CvP FPGA Manager"
> +       depends on PCI
> +       help
> +         FPGA manager driver support for Altera FPGAs using the
> +         CvP interface over PCIe.
> +
>  config FPGA_MGR_SOCFPGA
>         tristate "Altera SOCFPGA FPGA Manager"
>         depends on ARCH_SOCFPGA || COMPILE_TEST
> diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile
> index 2a4f021..2e5c8b6 100644
> --- a/drivers/fpga/Makefile
> +++ b/drivers/fpga/Makefile
> @@ -6,6 +6,7 @@
>  obj-$(CONFIG_FPGA)                     += fpga-mgr.o
>
>  # FPGA Manager Drivers
> +obj-$(CONFIG_FPGA_MGR_ALTERA_CVP)      += altera-cvp.o
>  obj-$(CONFIG_FPGA_MGR_ICE40_SPI)       += ice40-spi.o
>  obj-$(CONFIG_FPGA_MGR_SOCFPGA)         += socfpga.o
>  obj-$(CONFIG_FPGA_MGR_SOCFPGA_A10)     += socfpga-a10.o
> diff --git a/drivers/fpga/altera-cvp.c b/drivers/fpga/altera-cvp.c
> new file mode 100644
> index 0000000..b146033
> --- /dev/null
> +++ b/drivers/fpga/altera-cvp.c
> @@ -0,0 +1,587 @@
> +/*
> + * FPGA Manager Driver for Altera Arria/Cyclone/Stratix CvP
> + *
> + * Copyright (C) 2017 DENX Software Engineering
> + *
> + * Anatolij Gustschin <agust@denx.de>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; version 2 of the License.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * Manage Altera FPGA firmware using PCIe CvP.
> + * Firmware must be in binary "rbf" format.
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/fpga/fpga-mgr.h>
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include <linux/sizes.h>
> +
> +#define CVP_BAR                0       /* BAR used for data transfer in memory mode */
> +#define CVP_DUMMY_WR   244     /* dummy writes to clear CvP state machine */
> +#define TIMEOUT_IN_US  2000
> +
> +/* Vendor Specific Extended Capability Offset */
> +#define VSEC_OFFSET                    0x200
> +#define VSEC_PCIE_EXT_CAP_ID_VAL       0x000b
> +
> +/* offsets from VSEC_OFFSET */
> +#define VSEC_PCIE_EXT_CAP_ID           (VSEC_OFFSET + 0x00)    /* 16bit */
> +#define VSEC_VERSION                   0x02            /* 8bit */
> +#define VSEC_NEXT_CAP_OFF              0x03            /* 8bit */
> +#define VSEC_ID                                0x04            /* 16bit */
> +#define VSEC_REV                       0x06            /* 8bit */
> +#define VSEC_LENGTH                    0x07            /* 8bit */
> +#define VSEC_ALTERA_MARKER             0x08            /* 32bit */
> +
> +#define VSEC_CVP_STATUS                        (VSEC_OFFSET + 0x1c)    /* 32bit */
> +#define VSEC_CVP_STATUS_DATA_ENC       BIT(16) /* is treated as encrypted */
> +#define VSEC_CVP_STATUS_DATA_COMP      BIT(17) /* is treated as compressed */
> +#define VSEC_CVP_STATUS_CFG_RDY                BIT(18) /* CVP_CONFIG_READY */
> +#define VSEC_CVP_STATUS_CFG_ERR                BIT(19) /* CVP_CONFIG_ERROR */
> +#define VSEC_CVP_STATUS_CVP_EN         BIT(20) /* ctrl block is enabling CVP */
> +#define VSEC_CVP_STATUS_USERMODE       BIT(21) /* USERMODE */
> +#define VSEC_CVP_STATUS_CFG_DONE       BIT(23) /* CVP_CONFIG_DONE */
> +#define VSEC_CVP_STATUS_PLD_CLK_IN_USE BIT(24) /* PLD_CLK_IN_USE */
> +
> +#define VSEC_CVP_MODE_CTRL             (VSEC_OFFSET + 0x20)    /* 32bit */
> +#define VSEC_CVP_MODE_CTRL_CVP_MODE    BIT(0) /* CVP (1) or normal mode (0) */
> +#define VSEC_CVP_MODE_CTRL_HIP_CLK_SEL BIT(1) /* PMA (1) or fabric clock (0) */
> +#define VSEC_CVP_MODE_CTRL_FULL_CFG    BIT(2) /* CVP_FULLCONFIG */
> +#define VSEC_CVP_MODE_CTRL_NUMCLKS     (0xff<<8) /* CVP_NUMCLKS */
> +
> +#define VSEC_CVP_DATA                  (VSEC_OFFSET + 0x28)    /* 32bit */
> +#define VSEC_CVP_PROG_CTRL             (VSEC_OFFSET + 0x2c)    /* 32bit */
> +#define VSEC_CVP_PROG_CTRL_CONFIG      BIT(0)
> +#define VSEC_CVP_PROG_CTRL_START_XFER  BIT(1)
> +
> +#define VSEC_UNCOR_ERR_STATUS          (VSEC_OFFSET + 0x34)    /* 32bit */
> +#define VSEC_UNCOR_ERR_MASK            (VSEC_OFFSET + 0x38)    /* 32bit */
> +#define        VSEC_UNCOR_ERR_CVP_CFG_ERR      BIT(5)  /* CVP_CONFIG_ERROR_LATCHED */
> +
> +#define        DRV_NAME                "altera-cvp"
> +#define ALTERA_CVP_MGR_NAME    "Altera CvP FPGA Manager"
> +
> +static int chkcfg; /* use value 1 for debugging only */
> +module_param(chkcfg, int, 0664);
> +MODULE_PARM_DESC(chkcfg, "1 - check CvP status, 0 - skip checking (default 0)");
> +
> +static int numclks = 1; /* default 1 for uncompressed and unencrypted */
> +module_param(numclks, int, 0664);
> +MODULE_PARM_DESC(numclks, "Clock to data ratio 1, 4 or 8 (default 1)");
> +
> +struct altera_cvp_conf {
> +       struct fpga_manager     *mgr;
> +       struct pci_dev          *pci_dev;
> +       void __iomem            *map;
> +       char                    mgr_name[64];
> +};
> +
> +static inline void altera_cvp_chk_numclks(void)
> +{
> +       switch (numclks) {
> +       case 1:
> +       case 4:
> +       case 8:
> +               break;
> +       default:
> +               numclks = 1;
> +               break;
> +       }
> +}
> +
> +static enum fpga_mgr_states altera_cvp_state(struct fpga_manager *mgr)
> +{
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       u32 status;
> +
> +       pci_read_config_dword(conf->pci_dev, VSEC_CVP_STATUS, &status);
> +
> +       if (status & VSEC_CVP_STATUS_CFG_DONE)
> +               return FPGA_MGR_STATE_OPERATING;
> +
> +       if (status & VSEC_CVP_STATUS_CVP_EN)
> +               return FPGA_MGR_STATE_POWER_UP;
> +
> +       return FPGA_MGR_STATE_UNKNOWN;
> +}
> +
> +static int altera_cvp_teardown(struct fpga_manager *mgr,
> +                              struct fpga_image_info *info)
> +{
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       struct pci_dev *pdev = conf->pci_dev;
> +       int status = 0;
> +       int delay_us, i;
> +       u32 val32;
> +
> +       /*
> +        * STEP 12 - reset START_XFER bit
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_PROG_CTRL, &val32);
> +       val32 &= ~VSEC_CVP_PROG_CTRL_START_XFER;
> +       pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +       /*
> +        * STEP 13 - reset CVP_CONFIG bit
> +        */
> +       val32 &= ~VSEC_CVP_PROG_CTRL_CONFIG;
> +       pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +       /*
> +        * STEP 14
> +        * - set CVP_NUMCLKS to 0x01 and then issue CVP_DUMMY_WR dummy
> +        *   writes to the HIP
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +       /* set number of CVP clock cycles for every CVP Data Register Write */
> +       val32 |= 0x01 << 8;     /* 1 clock */
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       /* dummy memory write to switch from CVP clock to internal clock */
> +       val32 = 0xdeadbeef;
> +       if (conf->map) {
> +               u32 *map = conf->map;
> +
> +               for (i = 0; i < CVP_DUMMY_WR; i++)
> +                       *map++ = val32; /* use MemoryWrite */

*map is writing to iomem, it should be using an accessor routine like
writel() [1].

Also, there are several 'if (conf->map)' here and below.  You could
add a function that handles either using writel or
pci_write_config_dword.  Probably save 30+ lines of code.

> +       } else {
> +               for (i = 0; i < CVP_DUMMY_WR; i++)
> +                       pci_write_config_dword(pdev, VSEC_CVP_DATA, val32);
> +       }
> +
> +       /*
> +        * STEP 15 - poll CVP_CONFIG_READY bit
> +        */
> +       delay_us = 0;
> +       while (1) {
> +               pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +               if ((val32 & VSEC_CVP_STATUS_CFG_RDY) == 0)
> +                       break;
> +
> +               udelay(1);      /* wait 1us */
> +
> +               if (delay_us++ > TIMEOUT_IN_US) {
> +                       dev_warn(&mgr->dev, "CVP_CONFIG_READY == 0 timeout\n");
> +                       status = -ETIMEDOUT;
> +                       break;
> +               }
> +       }
> +
> +       return status;
> +}
> +
> +static int altera_cvp_write_init(struct fpga_manager *mgr,
> +                                struct fpga_image_info *info,
> +                                const char *buf, size_t count)
> +{
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       struct pci_dev *pdev = conf->pci_dev;
> +       int delay_us, i;
> +       u32 val32;
> +       int ret;
> +
> +       if (info && info->flags & FPGA_MGR_PARTIAL_RECONFIG) {
> +               dev_err(&mgr->dev, "Partial reconfiguration not supported.\n");
> +               return -EINVAL;
> +       }
> +
> +       /*
> +        * STEP 1 - read CVP status and check CVP_EN flag
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +       if (!(val32 & VSEC_CVP_STATUS_CVP_EN)) {
> +               dev_err(&mgr->dev, "CVP mode off: 0x%04x\n", val32);
> +               return -ENODEV;
> +       }
> +
> +       if (val32 & VSEC_CVP_STATUS_CFG_RDY) {
> +               dev_warn(&mgr->dev, "CvP already started, teardown first\n");
> +               ret = altera_cvp_teardown(mgr, info);
> +               if (ret < 0)
> +                       return ret;
> +       }
> +
> +       /*
> +        * STEP 2
> +        * - set HIP_CLK_SEL and CVP_MODE (must be set in the order mentioned)
> +        */
> +       /* switch from fabric to PMA clock */
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 |= VSEC_CVP_MODE_CTRL_HIP_CLK_SEL;
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       /* set CVP mode */
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 |= VSEC_CVP_MODE_CTRL_CVP_MODE;
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       /*
> +        * STEP 3
> +        * - set CVP_NUMCLKS to 1 and issue CVP_DUMMY_WR dummy writes to the HIP
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +       val32 |= 0x01 << 8;     /* 1 clock */
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       /* dummy memory write to switch from internal clock to CVP clock */
> +       val32 = 0xdeadbeef;
> +       if (conf->map) {
> +               u32 *map = conf->map;
> +
> +               for (i = 0; i < CVP_DUMMY_WR; i++)
> +                       *map++ = val32; /* use MemoryWrite */
> +       } else {
> +               for (i = 0; i < CVP_DUMMY_WR; i++)
> +                       pci_write_config_dword(pdev, VSEC_CVP_DATA, val32);
> +       }
> +
> +       /*
> +        * STEP 4 - set CVP_CONFIG bit
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_PROG_CTRL, &val32);
> +       /* request control block to begin transfer using CVP */
> +       val32 |= VSEC_CVP_PROG_CTRL_CONFIG;
> +       pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +       /*
> +        * STEP 5 - poll CVP_CONFIG READY
> +        */
> +       delay_us = 0;
> +       while (1) {
> +               pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +               if ((val32 & VSEC_CVP_STATUS_CFG_RDY) ==
> +                   VSEC_CVP_STATUS_CFG_RDY)
> +                       break;
> +
> +               udelay(1); /* wait 1us */
> +
> +               if (delay_us++ > TIMEOUT_IN_US) {
> +                       dev_warn(&mgr->dev, "CVP_CONFIG_READY == 1 timeout\n");
> +                       return -ETIMEDOUT;
> +               }
> +       }
> +
> +       /*
> +        * STEP 6
> +        * - set CVP_NUMCLKS to 1 and issue CVP_DUMMY_WR dummy writes to the HIP
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +       val32 |= 0x01 << 8; /* 1 clock */
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       /* dummy memory write to switch from internal clock to CVP clock */
> +       val32 = 0xdeadbeef;
> +       if (conf->map) {
> +               u32 *map = conf->map;
> +
> +               for (i = 0; i < CVP_DUMMY_WR; i++)
> +                       *map++ = val32; /* use MemoryWrite */
> +       } else {
> +               for (i = 0; i < CVP_DUMMY_WR; i++)
> +                       pci_write_config_dword(pdev, VSEC_CVP_DATA, val32);
> +       }
> +
> +       /*
> +        * STEP 7 - set START_XFER
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_PROG_CTRL, &val32);
> +       val32 |= VSEC_CVP_PROG_CTRL_START_XFER;
> +       pci_write_config_dword(pdev, VSEC_CVP_PROG_CTRL, val32);
> +
> +       /*
> +        * STEP 8 - start transfer (set CVP_NUMCLKS)
> +        */
> +       altera_cvp_chk_numclks();
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 &= ~VSEC_CVP_MODE_CTRL_NUMCLKS;
> +       val32 |= numclks << 8; /* bitstream specific clock count */
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       return 0;
> +}
> +
> +static inline int altera_cvp_chk_error(struct fpga_manager *mgr, size_t bytes)
> +{
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       u32 val32;
> +
> +       /*
> +        * STEP 10 (optional) - check CVP_CONFIG_ERROR flag
> +        */
> +       pci_read_config_dword(conf->pci_dev, VSEC_CVP_STATUS, &val32);
> +       if (val32 & VSEC_CVP_STATUS_CFG_ERR) {
> +               dev_err(&mgr->dev, "CVP_CONFIG_ERROR after %zi bytes!\n",
> +                       bytes);
> +               return -EPROTO;
> +       }
> +       return 0;
> +}
> +
> +static int altera_cvp_write(struct fpga_manager *mgr, const char *buf,
> +                           size_t count)
> +{
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       struct pci_dev *pdev = conf->pci_dev;
> +       const u32 *data;
> +       size_t bytes = 0;
> +       int status = 0;
> +       u32 mask;
> +       u32 *map;
> +
> +       /*
> +        * STEP 9
> +        * - write 32-bit configuration data from RBF file to CVP data register
> +        */
> +       data = (u32 *)buf;
> +       map = conf->map;
> +       bytes = count;
> +
> +       if (map) {
> +               while (bytes >= 4) {
> +                       *map = *data++;
> +                       bytes -= 4;
> +
> +                       /*
> +                        * STEP 10 (optional) and STEP 11
> +                        * - check error flag
> +                        * - loop until data transfer completed
> +                        */
> +                       if (chkcfg && !(bytes % SZ_4K)) {
> +                               size_t done = count - bytes;
> +
> +                               status = altera_cvp_chk_error(mgr, done);
> +                               if (status < 0)
> +                                       return status;
> +                       }
> +               }
> +       } else {
> +               while (bytes >= 4) {
> +                       pci_write_config_dword(pdev, VSEC_CVP_DATA, *data++);
> +                       bytes -= 4;
> +
> +                       if (chkcfg && !(bytes % SZ_4K)) {
> +                               size_t done = count - bytes;
> +
> +                               status = altera_cvp_chk_error(mgr, done);
> +                               if (status < 0)
> +                                       return status;
> +                       }
> +               }
> +       }
> +
> +       switch (bytes) {
> +       case 3:
> +               mask = 0x00ffffff;
> +               break;
> +       case 2:
> +               mask = 0x0000ffff;
> +               break;
> +       case 1:
> +               mask = 0x000000ff;
> +               break;
> +       case 0:
> +               mask = 0x00000000;
> +               break;
> +       }
> +
> +       if (mask) {
> +               u32 tail = *data & mask;
> +
> +               if (map)
> +                       *map = tail;
> +               else
> +                       pci_write_config_dword(pdev, VSEC_CVP_DATA, tail);
> +
> +               if (chkcfg)
> +                       status = altera_cvp_chk_error(mgr, count);
> +       }
> +
> +       return status;
> +}
> +
> +static int altera_cvp_write_complete(struct fpga_manager *mgr,
> +                                    struct fpga_image_info *info)
> +{
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       struct pci_dev *pdev = conf->pci_dev;
> +       int status;
> +       int delay_us;
> +       u32 status_msk;
> +       u32 val32;
> +
> +       status = altera_cvp_teardown(mgr, info);
> +       if (status < 0)
> +               return status;
> +
> +       /*
> +        * STEP 16 - check CVP_CONFIG_ERROR_LATCHED bit
> +        */
> +       pci_read_config_dword(pdev, VSEC_UNCOR_ERR_STATUS, &val32);
> +       if (val32 & VSEC_UNCOR_ERR_CVP_CFG_ERR) {
> +               dev_err(&mgr->dev, "detected CVP_CONFIG_ERROR_LATCHED!\n");
> +               return -EFAULT;
> +       }
> +
> +       /*
> +        * STEP 17 - reset CVP_MODE and HIP_CLK_SEL bit
> +        */
> +       pci_read_config_dword(pdev, VSEC_CVP_MODE_CTRL, &val32);
> +       val32 &= ~VSEC_CVP_MODE_CTRL_HIP_CLK_SEL;
> +       val32 &= ~VSEC_CVP_MODE_CTRL_CVP_MODE;
> +       pci_write_config_dword(pdev, VSEC_CVP_MODE_CTRL, val32);
> +
> +       /*
> +        * STEP 18 - poll PLD_CLK_IN_USE and USER_MODE bits
> +        */
> +       delay_us = 0;
> +       status_msk = VSEC_CVP_STATUS_PLD_CLK_IN_USE | VSEC_CVP_STATUS_USERMODE;
> +       while (1) {
> +               pci_read_config_dword(pdev, VSEC_CVP_STATUS, &val32);
> +               if ((val32 & status_msk) == status_msk)
> +                       break;
> +
> +               udelay(1); /* wait 1us */
> +
> +               if (delay_us++ > TIMEOUT_IN_US) {
> +                       dev_warn(&mgr->dev, "PLD_CLK_IN_USE|USERMODE timeout\n");
> +                       status = -ETIMEDOUT;
> +                       break;
> +               }
> +       }
> +
> +       return status;
> +}
> +
> +static const struct fpga_manager_ops altera_cvp_ops = {
> +       .state          = altera_cvp_state,
> +       .write_init     = altera_cvp_write_init,
> +       .write          = altera_cvp_write,
> +       .write_complete = altera_cvp_write_complete,
> +};
> +
> +static int altera_cvp_probe(struct pci_dev *pdev,
> +                           const struct pci_device_id *dev_id)
> +{
> +       struct altera_cvp_conf *conf;
> +       u16 cmd, val16;
> +       int ret;
> +
> +       pci_read_config_word(pdev, VSEC_PCIE_EXT_CAP_ID, &val16);
> +       if (val16 != VSEC_PCIE_EXT_CAP_ID_VAL) {
> +               dev_err(&pdev->dev, "Wrong EXT_CAP_ID value 0x%x\n", val16);
> +               ret = -ENODEV;
> +               goto err;
> +       }
> +
> +       /* Enable memory BAR access */
> +       pci_read_config_word(pdev, PCI_COMMAND, &cmd);
> +       if (!(cmd & PCI_COMMAND_MEMORY)) {
> +               cmd |= PCI_COMMAND_MEMORY;
> +               pci_write_config_word(pdev, PCI_COMMAND, cmd);
> +       }
> +
> +       conf = devm_kzalloc(&pdev->dev, sizeof(*conf), GFP_KERNEL);
> +       if (!conf) {
> +               ret = -ENOMEM;
> +               goto err;
> +       }
> +
> +       conf->pci_dev = pdev;
> +
> +       if (pci_request_region(pdev, CVP_BAR, "CVP") < 0) {
> +               dev_err(&pdev->dev, "Requesting CVP BAR region failed\n");
> +               ret = -ENODEV;
> +               goto err;
> +       }
> +
> +       conf->map = pci_iomap(pdev, CVP_BAR, 0);
> +       if (!conf->map)
> +               dev_warn(&pdev->dev, "Mapping CVP BAR failed\n");
> +
> +       snprintf(conf->mgr_name, sizeof(conf->mgr_name), "%s @%02x:%02x.%d",
> +                ALTERA_CVP_MGR_NAME, pdev->bus->number,
> +                PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn));
> +
> +       ret = fpga_mgr_register(&pdev->dev, conf->mgr_name,
> +                               &altera_cvp_ops, conf);
> +       if (ret)
> +               goto err_unmap;
> +
> +       conf->mgr = fpga_mgr_get(&pdev->dev);
> +       if (IS_ERR(conf->mgr)) {
> +               dev_err(&pdev->dev, "Getting fpga mgr reference failed\n");
> +               ret = -ENODEV;
> +               goto err_unreg;
> +       }
> +       fpga_mgr_put(conf->mgr);
> +
> +       return 0;
> +
> +err_unreg:
> +       fpga_mgr_unregister(&pdev->dev);
> +err_unmap:
> +       pci_iounmap(pdev, conf->map);
> +       pci_release_region(pdev, CVP_BAR);
> +err:
> +       cmd &= ~PCI_COMMAND_MEMORY;
> +       pci_write_config_word(pdev, PCI_COMMAND, cmd);
> +       return ret;
> +}
> +
> +static void altera_cvp_remove(struct pci_dev *pdev)
> +{
> +       struct fpga_manager *mgr = pci_get_drvdata(pdev);
> +       struct altera_cvp_conf *conf = mgr->priv;
> +       u16 cmd;
> +
> +       fpga_mgr_unregister(&pdev->dev);
> +       pci_iounmap(pdev, conf->map);
> +       pci_release_region(pdev, CVP_BAR);
> +       pci_read_config_word(pdev, PCI_COMMAND, &cmd);
> +       cmd &= ~PCI_COMMAND_MEMORY;
> +       pci_write_config_word(pdev, PCI_COMMAND, cmd);
> +}
> +
> +#define PCI_VENDOR_ID_ALTERA   0x1172
> +
> +static struct pci_device_id altera_cvp_id_tbl[] = {
> +       { PCI_VDEVICE(ALTERA, PCI_ANY_ID) },
> +       { }
> +};
> +
> +MODULE_DEVICE_TABLE(pci, altera_cvp_id_tbl);
> +
> +static struct pci_driver altera_cvp_driver = {
> +       .name   = DRV_NAME,
> +       .id_table = altera_cvp_id_tbl,
> +       .probe  = altera_cvp_probe,
> +       .remove = altera_cvp_remove,
> +};
> +
> +static int __init altera_cvp_init(void)
> +{
> +       return pci_register_driver(&altera_cvp_driver);
> +}
> +
> +static void __exit altera_cvp_exit(void)
> +{
> +       pci_unregister_driver(&altera_cvp_driver);
> +}
> +
> +module_init(altera_cvp_init);
> +module_exit(altera_cvp_exit);
> +
> +MODULE_LICENSE("GPL v2");
> +MODULE_AUTHOR("Anatolij Gustschin <agust@denx.de>");
> +MODULE_DESCRIPTION("Module to load Altera FPGA over CvP");
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[1] linux/Documentation/memory-barriers.txt
--
To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[v3] fpga manager: Add Altera CvP driver

Commit Message

Comments

Patch