| Message ID | 2ba262668e86e58acb086c64fc759ba02b39a525.1663007783.git.quic_gokukris@quicinc.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | Memory allocation change in scm/mdt_loader | expand |
On Mon, Sep 12, 2022 at 11:41:32AM -0700, Gokul krishna Krishnakumar wrote: > By moving the memory allocation to mdt loader we can simplify the scm > call, by just packing arguments provided to it from the clients for > making secuer world calls. We can also simplify the memory allocation > for the qcom metadata, by just doing one memory allocation in the > mdt loader. > > Signed-off-by: Gokul krishna Krishnakumar <quic_gokukris@quicinc.com> > --- > drivers/remoteproc/qcom_q6v5_mss.c | 2 +- > drivers/soc/qcom/mdt_loader.c | 41 ++++++++++++++++++++++++++++--------- > include/linux/soc/qcom/mdt_loader.h | 5 +++-- > 3 files changed, 35 insertions(+), 13 deletions(-) > > diff --git a/drivers/remoteproc/qcom_q6v5_mss.c b/drivers/remoteproc/qcom_q6v5_mss.c > index fddb63c..1919bfc 100644 > --- a/drivers/remoteproc/qcom_q6v5_mss.c > +++ b/drivers/remoteproc/qcom_q6v5_mss.c > @@ -947,7 +947,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw, > int ret; > int i; > > - metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev); > + metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev, NULL); At the end of this function we invoke kfree(metadata), which would be bad if that comes from dma_alloc_coherent(). > if (IS_ERR(metadata)) > return PTR_ERR(metadata); > > diff --git a/drivers/soc/qcom/mdt_loader.c b/drivers/soc/qcom/mdt_loader.c [..] > @@ -160,9 +164,18 @@ void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, > ehdr_size = phdrs[0].p_filesz; > hash_size = phdrs[hash_segment].p_filesz; > > - data = kmalloc(ehdr_size + hash_size, GFP_KERNEL); > - if (!data) > - return ERR_PTR(-ENOMEM); > + /* > + * During the scm call memory protection will be enabled for the meta > + * data blob, so make sure it's physically contiguous, 4K aligned and > + * non-cachable to avoid XPU violations. > + */ > + scm_dev = qcom_get_scm_device(); As LKP points out, I don't seem to have this function. > + data = dma_alloc_coherent(scm_dev, ehdr_size + hash_size, mdata_phys, > + GFP_KERNEL); I am not thrilled about the idea of doing dma_alloc_coherent() in this file and dma_free_coherent() in the scm driver. Similarly, I consider these functions to operate in the context of the caller, so operating on the scm device's struct device isn't so nice. After trying various models I came to the conclusion that it was better to try to keep the MDT loader to just load MDT files, and move the SCM/PAS interaction out of that. Unfortunately we have a number of client drivers that would then need to (essentially) duplicate the content of qcom_mdt_pas_init() - so I left that in there. I still believe that keeping the MDT loader focused on loading MDTs is a good idea, but I'm open to any suggestions for improvements in the interaction between these different components. Regards, Bjorn
>At the end of this function we invoke kfree(metadata), which would be bad if that comes from dma_alloc_coherent(). + if (mdata_phys) { + data = dma_alloc_coherent(dev, ehdr_size + hash_size, mdata_phys, + GFP_KERNEL); + } else { + data = kmalloc(ehdr_size + hash_size, GFP_KERNEL); Adding dma_alloc_coherent without affecting the mss driver. > As LKP points out, I don't seem to have this function. Removing the qcom_get_scm_device() and calling dma_alloc_coherent from device context. + data = dma_alloc_coherent(dev, ehdr_size + hash_size, mdata_phys, + GFP_KERNEL); >I am not thrilled about the idea of doing dma_alloc_coherent() in this file and dma_free_coherent() in the scm driver. Similarly, I consider these functions to operate in the context of the caller, so operating on the scm device's struct device isn't so nice. >After trying various models I came to the conclusion that it was better to try to keep the MDT loader to just load MDT files, and move the SCM/PAS interaction out of that. Unfortunately we have a number of client drivers that would then need to (essentially) duplicate the content of qcom_mdt_pas_init() - so I left >that in there. >I still believe that keeping the MDT loader focused on loading MDTs is a good idea, but I'm open to any suggestions for improvements in the interaction between these different components. With this patch we moving all the dma_alloc_coherent() and dma_free_coherent() to the MDT loader. So now the MDT loader has the functionality of loading and allocating memory and the SCM driver packs the arguments and makes a call to the secure world. -----Original Message----- From: Bjorn Andersson <andersson@kernel.org> Sent: Tuesday, September 13, 2022 4:11 PM To: Gokul krishna Krishnakumar (QUIC) <quic_gokukris@quicinc.com> Cc: Andy Gross <agross@kernel.org>; Konrad Dybcio <konrad.dybcio@somainline.org>; Philipp Zabel <p.zabel@pengutronix.de>; linux-arm-msm@vger.kernel.org; linux-kernel@vger.kernel.org; Trilok Soni (QUIC) <quic_tsoni@quicinc.com>; Satya Durga Srinivasu Prabhala (QUIC) <quic_satyap@quicinc.com>; Rajendra Nayak (QUIC) <quic_rjendra@quicinc.com>; Elliot Berman (QUIC) <quic_eberman@quicinc.com>; Guru Das Srinagesh (QUIC) <quic_gurus@quicinc.com> Subject: Re: [PATCH v1 2/2] soc: qcom: mdt_loader: Move the memory allocation into mdt loader WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros. On Mon, Sep 12, 2022 at 11:41:32AM -0700, Gokul krishna Krishnakumar wrote: > By moving the memory allocation to mdt loader we can simplify the scm > call, by just packing arguments provided to it from the clients for > making secuer world calls. We can also simplify the memory allocation > for the qcom metadata, by just doing one memory allocation in the mdt > loader. > > Signed-off-by: Gokul krishna Krishnakumar <quic_gokukris@quicinc.com> > --- > drivers/remoteproc/qcom_q6v5_mss.c | 2 +- > drivers/soc/qcom/mdt_loader.c | 41 ++++++++++++++++++++++++++++--------- > include/linux/soc/qcom/mdt_loader.h | 5 +++-- > 3 files changed, 35 insertions(+), 13 deletions(-) > > diff --git a/drivers/remoteproc/qcom_q6v5_mss.c > b/drivers/remoteproc/qcom_q6v5_mss.c > index fddb63c..1919bfc 100644 > --- a/drivers/remoteproc/qcom_q6v5_mss.c > +++ b/drivers/remoteproc/qcom_q6v5_mss.c > @@ -947,7 +947,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw, > int ret; > int i; > > - metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev); > + metadata = qcom_mdt_read_metadata(fw, &size, fw_name, > + qproc->dev, NULL); At the end of this function we invoke kfree(metadata), which would be bad if that comes from dma_alloc_coherent(). > if (IS_ERR(metadata)) > return PTR_ERR(metadata); > > diff --git a/drivers/soc/qcom/mdt_loader.c > b/drivers/soc/qcom/mdt_loader.c [..] > @@ -160,9 +164,18 @@ void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, > ehdr_size = phdrs[0].p_filesz; > hash_size = phdrs[hash_segment].p_filesz; > > - data = kmalloc(ehdr_size + hash_size, GFP_KERNEL); > - if (!data) > - return ERR_PTR(-ENOMEM); > + /* > + * During the scm call memory protection will be enabled for the meta > + * data blob, so make sure it's physically contiguous, 4K aligned and > + * non-cachable to avoid XPU violations. > + */ > + scm_dev = qcom_get_scm_device(); As LKP points out, I don't seem to have this function. > + data = dma_alloc_coherent(scm_dev, ehdr_size + hash_size, mdata_phys, > + GFP_KERNEL); I am not thrilled about the idea of doing dma_alloc_coherent() in this file and dma_free_coherent() in the scm driver. Similarly, I consider these functions to operate in the context of the caller, so operating on the scm device's struct device isn't so nice. After trying various models I came to the conclusion that it was better to try to keep the MDT loader to just load MDT files, and move the SCM/PAS interaction out of that. Unfortunately we have a number of client drivers that would then need to (essentially) duplicate the content of qcom_mdt_pas_init() - so I left that in there. I still believe that keeping the MDT loader focused on loading MDTs is a good idea, but I'm open to any suggestions for improvements in the interaction between these different components. Regards, Bjorn
Hi Bjorn, With this patch we have moved the dma_alloc_coherent/dma_free_coherent is called from the mdt loader and is operating in the context of the caller, the scm device's struct device is not used in this patch. For the clients which do not pass the metadata physical argument to the qcom_mdt_read_metadata() - the memory is allocated using kmalloc- so the clients like qcom_q6v5_mss.c, where kfree is called will not be broken with this change. Thanks, Gokul On 9/21/2022 12:39 PM, Gokul krishna Krishnakumar (QUIC) wrote: >> At the end of this function we invoke kfree(metadata), which would be bad if that comes from dma_alloc_coherent(). > + if (mdata_phys) { > + data = dma_alloc_coherent(dev, ehdr_size + hash_size, mdata_phys, > + GFP_KERNEL); > + } else { > + data = kmalloc(ehdr_size + hash_size, GFP_KERNEL); > Adding dma_alloc_coherent without affecting the mss driver. > > >> As LKP points out, I don't seem to have this function. > Removing the qcom_get_scm_device() and calling dma_alloc_coherent from device context. > + data = dma_alloc_coherent(dev, ehdr_size + hash_size, mdata_phys, > + GFP_KERNEL); > >> I am not thrilled about the idea of doing dma_alloc_coherent() in this file and dma_free_coherent() in the scm driver. Similarly, I consider these functions to operate in the context of the caller, so operating on the scm device's struct device isn't so nice. >> After trying various models I came to the conclusion that it was better to try to keep the MDT loader to just load MDT files, and move the SCM/PAS interaction out of that. Unfortunately we have a number of client drivers that would then need to (essentially) duplicate the content of qcom_mdt_pas_init() - so I left >that in there. >> I still believe that keeping the MDT loader focused on loading MDTs is a good idea, but I'm open to any suggestions for improvements in the interaction between these different components. > > With this patch we moving all the dma_alloc_coherent() and dma_free_coherent() to the MDT loader. > So now the MDT loader has the functionality of loading and allocating memory > and the SCM driver packs the arguments and makes a call to the secure world. > > -----Original Message----- > From: Bjorn Andersson <andersson@kernel.org> > Sent: Tuesday, September 13, 2022 4:11 PM > To: Gokul krishna Krishnakumar (QUIC) <quic_gokukris@quicinc.com> > Cc: Andy Gross <agross@kernel.org>; Konrad Dybcio <konrad.dybcio@somainline.org>; Philipp Zabel <p.zabel@pengutronix.de>; linux-arm-msm@vger.kernel.org; linux-kernel@vger.kernel.org; Trilok Soni (QUIC) <quic_tsoni@quicinc.com>; Satya Durga Srinivasu Prabhala (QUIC) <quic_satyap@quicinc.com>; Rajendra Nayak (QUIC) <quic_rjendra@quicinc.com>; Elliot Berman (QUIC) <quic_eberman@quicinc.com>; Guru Das Srinagesh (QUIC) <quic_gurus@quicinc.com> > Subject: Re: [PATCH v1 2/2] soc: qcom: mdt_loader: Move the memory allocation into mdt loader > > WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros. > > On Mon, Sep 12, 2022 at 11:41:32AM -0700, Gokul krishna Krishnakumar wrote: >> By moving the memory allocation to mdt loader we can simplify the scm >> call, by just packing arguments provided to it from the clients for >> making secuer world calls. We can also simplify the memory allocation >> for the qcom metadata, by just doing one memory allocation in the mdt >> loader. >> >> Signed-off-by: Gokul krishna Krishnakumar <quic_gokukris@quicinc.com> >> --- >> drivers/remoteproc/qcom_q6v5_mss.c | 2 +- >> drivers/soc/qcom/mdt_loader.c | 41 ++++++++++++++++++++++++++++--------- >> include/linux/soc/qcom/mdt_loader.h | 5 +++-- >> 3 files changed, 35 insertions(+), 13 deletions(-) >> >> diff --git a/drivers/remoteproc/qcom_q6v5_mss.c >> b/drivers/remoteproc/qcom_q6v5_mss.c >> index fddb63c..1919bfc 100644 >> --- a/drivers/remoteproc/qcom_q6v5_mss.c >> +++ b/drivers/remoteproc/qcom_q6v5_mss.c >> @@ -947,7 +947,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw, >> int ret; >> int i; >> >> - metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev); >> + metadata = qcom_mdt_read_metadata(fw, &size, fw_name, >> + qproc->dev, NULL); > > At the end of this function we invoke kfree(metadata), which would be bad if that comes from dma_alloc_coherent(). > >> if (IS_ERR(metadata)) >> return PTR_ERR(metadata); >> >> diff --git a/drivers/soc/qcom/mdt_loader.c >> b/drivers/soc/qcom/mdt_loader.c > [..] >> @@ -160,9 +164,18 @@ void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, >> ehdr_size = phdrs[0].p_filesz; >> hash_size = phdrs[hash_segment].p_filesz; >> >> - data = kmalloc(ehdr_size + hash_size, GFP_KERNEL); >> - if (!data) >> - return ERR_PTR(-ENOMEM); >> + /* >> + * During the scm call memory protection will be enabled for the meta >> + * data blob, so make sure it's physically contiguous, 4K aligned and >> + * non-cachable to avoid XPU violations. >> + */ >> + scm_dev = qcom_get_scm_device(); > > As LKP points out, I don't seem to have this function. > >> + data = dma_alloc_coherent(scm_dev, ehdr_size + hash_size, mdata_phys, >> + GFP_KERNEL); > > I am not thrilled about the idea of doing dma_alloc_coherent() in this file and dma_free_coherent() in the scm driver. Similarly, I consider these functions to operate in the context of the caller, so operating on the scm device's struct device isn't so nice. > > > After trying various models I came to the conclusion that it was better to try to keep the MDT loader to just load MDT files, and move the SCM/PAS interaction out of that. Unfortunately we have a number of client drivers that would then need to (essentially) duplicate the content of qcom_mdt_pas_init() - so I left that in there. > > I still believe that keeping the MDT loader focused on loading MDTs is a good idea, but I'm open to any suggestions for improvements in the interaction between these different components. > > Regards, > Bjorn
diff --git a/drivers/remoteproc/qcom_q6v5_mss.c b/drivers/remoteproc/qcom_q6v5_mss.c index fddb63c..1919bfc 100644 --- a/drivers/remoteproc/qcom_q6v5_mss.c +++ b/drivers/remoteproc/qcom_q6v5_mss.c @@ -947,7 +947,7 @@ static int q6v5_mpss_init_image(struct q6v5 *qproc, const struct firmware *fw, int ret; int i; - metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev); + metadata = qcom_mdt_read_metadata(fw, &size, fw_name, qproc->dev, NULL); if (IS_ERR(metadata)) return PTR_ERR(metadata); diff --git a/drivers/soc/qcom/mdt_loader.c b/drivers/soc/qcom/mdt_loader.c index 8d06125..e730413 100644 --- a/drivers/soc/qcom/mdt_loader.c +++ b/drivers/soc/qcom/mdt_loader.c @@ -16,6 +16,7 @@ #include <linux/sizes.h> #include <linux/slab.h> #include <linux/soc/qcom/mdt_loader.h> +#include <linux/dma-mapping.h> static bool mdt_phdr_valid(const struct elf32_phdr *phdr) { @@ -110,6 +111,7 @@ EXPORT_SYMBOL_GPL(qcom_mdt_get_size); * @data_len: length of the read metadata blob * @fw_name: name of the firmware, for construction of segment file names * @dev: device handle to associate resources with + * @mdata_phys: phys address for the assigned metadata buffer * * The mechanism that performs the authentication of the loading firmware * expects an ELF header directly followed by the segment of hashes, with no @@ -124,11 +126,13 @@ EXPORT_SYMBOL_GPL(qcom_mdt_get_size); * Return: pointer to data, or ERR_PTR() */ void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, - const char *fw_name, struct device *dev) + const char *fw_name, struct device *dev, + dma_addr_t *mdata_phys) { const struct elf32_phdr *phdrs; const struct elf32_hdr *ehdr; unsigned int hash_segment = 0; + struct device *scm_dev = NULL; size_t hash_offset; size_t hash_size; size_t ehdr_size; @@ -160,9 +164,18 @@ void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, ehdr_size = phdrs[0].p_filesz; hash_size = phdrs[hash_segment].p_filesz; - data = kmalloc(ehdr_size + hash_size, GFP_KERNEL); - if (!data) - return ERR_PTR(-ENOMEM); + /* + * During the scm call memory protection will be enabled for the meta + * data blob, so make sure it's physically contiguous, 4K aligned and + * non-cachable to avoid XPU violations. + */ + scm_dev = qcom_get_scm_device(); + data = dma_alloc_coherent(scm_dev, ehdr_size + hash_size, mdata_phys, + GFP_KERNEL); + if (!data) { + dev_err(dev, "Allocation of metadata buffer failed.\n"); + return NULL; + } /* Copy ELF header */ memcpy(data, fw->data, ehdr_size); @@ -179,7 +192,7 @@ void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, /* Hash is in its own segment, beyond the loaded file */ ret = mdt_load_split_segment(data + ehdr_size, phdrs, hash_segment, fw_name, dev); if (ret) { - kfree(data); + dma_free_coherent(scm_dev, ehdr_size + hash_size, data, mdata_phys); return ERR_PTR(ret); } } @@ -209,10 +222,11 @@ int qcom_mdt_pas_init(struct device *dev, const struct firmware *fw, const struct elf32_phdr *phdr; const struct elf32_hdr *ehdr; phys_addr_t min_addr = PHYS_ADDR_MAX; + struct device *scm_dev = NULL; phys_addr_t max_addr = 0; dma_addr_t mdata_phys; size_t metadata_len; - void *metadata; + void *mdata_buf; int ret; int i; @@ -232,15 +246,22 @@ int qcom_mdt_pas_init(struct device *dev, const struct firmware *fw, max_addr = ALIGN(phdr->p_paddr + phdr->p_memsz, SZ_4K); } - metadata = qcom_mdt_read_metadata(fw, &metadata_len, fw_name, dev); - if (IS_ERR(metadata)) { - ret = PTR_ERR(metadata); + mdata_buf = qcom_mdt_read_metadata(fw, &metadata_len, fw_name, dev, &mdata_phys); + if (IS_ERR(mdata_buf)) { + ret = PTR_ERR(mdata_buf); dev_err(dev, "error %d reading firmware %s metadata\n", ret, fw_name); goto out; } ret = qcom_scm_pas_init_image(pas_id, mdata_phys); - kfree(metadata); + if (ret || !ctx) { + dma_free_coherent(scm_dev, metadata_len, mdata_buf, mdata_phys); + } else if (ctx) { + ctx->ptr = mdata_buf; + ctx->phys = mdata_phys; + ctx->size = metadata_len; + } + if (ret) { /* Invalid firmware metadata */ dev_err(dev, "error %d initializing firmware %s\n", ret, fw_name); diff --git a/include/linux/soc/qcom/mdt_loader.h b/include/linux/soc/qcom/mdt_loader.h index 9e8e604..d438442 100644 --- a/include/linux/soc/qcom/mdt_loader.h +++ b/include/linux/soc/qcom/mdt_loader.h @@ -28,7 +28,8 @@ int qcom_mdt_load_no_init(struct device *dev, const struct firmware *fw, phys_addr_t mem_phys, size_t mem_size, phys_addr_t *reloc_base); void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, - const char *fw_name, struct device *dev); + const char *fw_name, struct device *dev, + dma_addr_t *mdata_phys); #else /* !IS_ENABLED(CONFIG_QCOM_MDT_LOADER) */ @@ -64,7 +65,7 @@ static inline int qcom_mdt_load_no_init(struct device *dev, static inline void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len, const char *fw_name, - struct device *dev) + struct device *dev, dma_addr_t *mdata_phys) { return ERR_PTR(-ENODEV); }
By moving the memory allocation to mdt loader we can simplify the scm call, by just packing arguments provided to it from the clients for making secuer world calls. We can also simplify the memory allocation for the qcom metadata, by just doing one memory allocation in the mdt loader. Signed-off-by: Gokul krishna Krishnakumar <quic_gokukris@quicinc.com> --- drivers/remoteproc/qcom_q6v5_mss.c | 2 +- drivers/soc/qcom/mdt_loader.c | 41 ++++++++++++++++++++++++++++--------- include/linux/soc/qcom/mdt_loader.h | 5 +++-- 3 files changed, 35 insertions(+), 13 deletions(-)