| Message ID | 0648312f1c78e519142ff1829a984e7e965bf3c7.1557957269.git.lars.kurth@citrix.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Add TRACKING.IMPORTS to xen.git to more easily manage imported files that need to be kept in sync with an upstream | expand |
>>> On 16.05.19 at 00:18, <lars.kurth@citrix.com> wrote: > --- /dev/null > +++ b/TRACKING.IMPORTS > @@ -0,0 +1,40 @@ > +# This file contains information about source files that have been > +# copied from other sources and need to be tracked > +# > +# The file may contain lines starting with ... > +# -------------------------------------------- > +# version: of file format > +# repo: repository definition > +# auto|manual: a mapping to track files > +# > +# Repository Definitions are of the following format > +# -------------------------------------------------- > +# repo name-of-source-repo git|svn https-url-of-source-repo > +# > +# For example: > +# repo linux-torvalds git https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > +# > +# Mappings to track files are of the following format > +# --------------------------------------------------- > +# manual|auto xen-file name-of-original-repo original-file commit-id > +# > +# auto: > +# The xen-file needs to track the the original-file exactly > +# In other words, we can automatically update the file using a script Do we have _any_ example of this? I can't even imagine one, due to e.g. our includes all starting with xen/ whereas Linux'es (just to take as example) all start with linux/. Perhaps "auto" needs to include sed expressions that need to be applied before actually applying the original change to our tree? > +# manual: > +# A developer needs to make a decision whether a > +# specific change is applied or ignored and update the last commit id > +# accordingly > +# > +# name-of-original-repo: > +# A reference to a source repository defined by *repo* keyword > +# > +# commit id: > +# Last commit id of source file that was deemed to be ok > +# and either imported into the tree or rejected > +# > +# For example: > +# manual xen/drivers/passthrough/arm/smmu.c linux-torvalds linux/drivers/iommu/arm-smmu.c b77cf11f094136 > + > +version 1 Perhaps it wouldn't hurt to include the colons in the actual entries as well? I also don't think examples are needed once we get the first real entries. Hence I'd move them to the commit message or a post-commit message remark. Jan
On 16/05/2019, 04:47, "Jan Beulich" <JBeulich@suse.com> wrote: >>> On 16.05.19 at 00:18, <lars.kurth@citrix.com> wrote: > +# Mappings to track files are of the following format > +# --------------------------------------------------- > +# manual|auto xen-file name-of-original-repo original-file commit-id > +# > +# auto: > +# The xen-file needs to track the the original-file exactly > +# In other words, we can automatically update the file using a script Do we have _any_ example of this? I can't even imagine one, due to e.g. our includes all starting with xen/ whereas Linux'es (just to take as example) all start with linux/. Perhaps "auto" needs to include sed expressions that need to be applied before actually applying the original change to our tree? I am not sure I fully understand your concern. This was intended for the case where say we would exactly track xen/.../foo.bar with linux/.../foo.bar In other words, auto only applies to the content of a file: the filename isn't relevant, because all the information that would be needed to do this is in the file. But if there is no need for it, we can drop it. And if needed, we can add in future. @Julien, @Stefano, @Jan: are any of the files you listed fall into the "should be tracked exactly" category? > +# manual: > +# A developer needs to make a decision whether a > +# specific change is applied or ignored and update the last commit id > +# accordingly > +# > +# name-of-original-repo: > +# A reference to a source repository defined by *repo* keyword > +# > +# commit id: > +# Last commit id of source file that was deemed to be ok > +# and either imported into the tree or rejected > +# > +# For example: > +# manual xen/drivers/passthrough/arm/smmu.c linux-torvalds linux/drivers/iommu/arm-smmu.c b77cf11f094136 > + > +version 1 Perhaps it wouldn't hurt to include the colons in the actual entries as well? I am not sure what you mean, which colons? Are you saying, the format should be version: 1 repo: ... I think the confusion comes because I used colons after statements in the comments. I think that "version: 1" is slightly more human-readable, so I would be OK with that I also don't think examples are needed once we get the first real entries. Hence I'd move them to the commit message or a post-commit message remark. Agreed. Lars
>>> On 16.05.19 at 17:54, <lars.kurth@citrix.com> wrote: > > On 16/05/2019, 04:47, "Jan Beulich" <JBeulich@suse.com> wrote: > > >>> On 16.05.19 at 00:18, <lars.kurth@citrix.com> wrote: > > +# Mappings to track files are of the following format > > +# --------------------------------------------------- > > +# manual|auto xen-file name-of-original-repo original-file commit-id > > +# > > +# auto: > > +# The xen-file needs to track the the original-file exactly > > +# In other words, we can automatically update the file using a script > > Do we have _any_ example of this? I can't even imagine one, due > to e.g. our includes all starting with xen/ whereas Linux'es (just to > take as example) all start with linux/. Perhaps "auto" needs to > include sed expressions that need to be applied before actually > applying the original change to our tree? > > I am not sure I fully understand your concern. > This was intended for the case where say we would exactly track > xen/.../foo.bar with linux/.../foo.bar > In other words, auto only applies to the content of a file: the filename > isn't relevant, because all the information that would be needed to do this > is in the file. When talking about file names in my reply, I referred to C language #include directives inside the file in question, as a (pretty important) example. There was no talk about the cloned/copied file's name itself. Hence the suggestion to accompany auto: with a set of sed expressions, which could then e.g. transform #include <linux/...> into #include <xen/...>. > @Julien, @Stefano, @Jan: are any of the files you listed fall into the > "should be tracked exactly" category? As I've said before - I can't even imagine such a file to exist. > > +# manual: > > +# A developer needs to make a decision whether a > > +# specific change is applied or ignored and update the last commit id > > +# accordingly > > +# > > +# name-of-original-repo: > > +# A reference to a source repository defined by *repo* keyword > > +# > > +# commit id: > > +# Last commit id of source file that was deemed to be ok > > +# and either imported into the tree or rejected > > +# > > +# For example: > > +# manual xen/drivers/passthrough/arm/smmu.c linux-torvalds > linux/drivers/iommu/arm-smmu.c b77cf11f094136 > > + > > +version 1 > > Perhaps it wouldn't hurt to include the colons in the actual entries as > well? > > I am not sure what you mean, which colons? Are you saying, the format should be > version: 1 > repo: ... Yes. This would make it even more prominent that these are tags of some sort. But this was only a thought of mine, it's in no way meant to be a requirement I have. > I think the confusion comes because I used colons after statements in the > comments. Right, that's how I got there. > I think that "version: 1" is slightly more human-readable, so I would be OK > with that A well defined non-blank separator also allows machine processing to notice more easily if there's a malformed line. Plus (if need be) it would permit tags with blanks in their names. Jan
On 17/05/2019, 01:34, "Jan Beulich" <JBeulich@suse.com> wrote: >>> On 16.05.19 at 17:54, <lars.kurth@citrix.com> wrote: > > On 16/05/2019, 04:47, "Jan Beulich" <JBeulich@suse.com> wrote: > > >>> On 16.05.19 at 00:18, <lars.kurth@citrix.com> wrote: > > +# Mappings to track files are of the following format > > +# --------------------------------------------------- > > +# manual|auto xen-file name-of-original-repo original-file commit-id > > +# > > +# auto: > > +# The xen-file needs to track the the original-file exactly > > +# In other words, we can automatically update the file using a script > > Do we have _any_ example of this? I can't even imagine one, due > to e.g. our includes all starting with xen/ whereas Linux'es (just to > take as example) all start with linux/. Perhaps "auto" needs to > include sed expressions that need to be applied before actually > applying the original change to our tree? > > I am not sure I fully understand your concern. > This was intended for the case where say we would exactly track > xen/.../foo.bar with linux/.../foo.bar > In other words, auto only applies to the content of a file: the filename > isn't relevant, because all the information that would be needed to do this > is in the file. When talking about file names in my reply, I referred to C language #include directives inside the file in question, as a (pretty important) example. There was no talk about the cloned/copied file's name itself. Hence the suggestion to accompany auto: with a set of sed expressions, which could then e.g. transform #include <linux/...> into #include <xen/...>. That makes perfect sense now. In that case, I tend to agree that "auto" is probably not needed. Would be quite happy to drop it. > @Julien, @Stefano, @Jan: are any of the files you listed fall into the > "should be tracked exactly" category? As I've said before - I can't even imagine such a file to exist. > > +# manual: > > +# A developer needs to make a decision whether a > > +# specific change is applied or ignored and update the last commit id > > +# accordingly > > +# > > +# name-of-original-repo: > > +# A reference to a source repository defined by *repo* keyword > > +# > > +# commit id: > > +# Last commit id of source file that was deemed to be ok > > +# and either imported into the tree or rejected > > +# > > +# For example: > > +# manual xen/drivers/passthrough/arm/smmu.c linux-torvalds > linux/drivers/iommu/arm-smmu.c b77cf11f094136 > > + > > +version 1 > > Perhaps it wouldn't hurt to include the colons in the actual entries as > well? > > I am not sure what you mean, which colons? Are you saying, the format should be > version: 1 > repo: ... Yes. This would make it even more prominent that these are tags of some sort. But this was only a thought of mine, it's in no way meant to be a requirement I have. > I think the confusion comes because I used colons after statements in the > comments. Right, that's how I got there. > I think that "version: 1" is slightly more human-readable, so I would be OK > with that A well defined non-blank separator also allows machine processing to notice more easily if there's a malformed line. Plus (if need be) it would permit tags with blanks in their names. I can do that. No problem. Any other comments from anyone, before sending version 2? Lars
This is going in the right direction IMO.
Lars Kurth writes ("Re: [PATCH] Add TRACKING.IMPORTS to xen.git to more easily manage imported files that need to be kept in sync with an upstream"):
> That makes perfect sense now. In that case, I tend to agree that "auto" is probably not needed. Would be quite happy to drop it.
It will considerably complicate things to add a way to define
seddery. Let us leave that to a future extension.
That suggests that `manual' should become `file:'.
As for delimiters
> On 17/05/2019, 01:34, "Jan Beulich" <JBeulich@suse.com> wrote:
> > I am not sure what you mean, which colons? Are you saying, the format should be
> > version: 1
> > repo: ...
>
> Yes. This would make it even more prominent that these are tags of
> some sort. But this was only a thought of mine, it's in no way meant
> to be a requirement I have.
It will make writing a parser easier if each entry is a single line
with the fields in a defined order and if we can say that a `repo:'
must precede every `file:' that mentions it.
Ian.
On 20/05/2019, 10:37, "Ian Jackson" <ian.jackson@citrix.com> wrote:
This is going in the right direction IMO.
Lars Kurth writes ("Re: [PATCH] Add TRACKING.IMPORTS to xen.git to more easily manage imported files that need to be kept in sync with an upstream"):
> That makes perfect sense now. In that case, I tend to agree that "auto" is probably not needed. Would be quite happy to drop it.
It will considerably complicate things to add a way to define
seddery. Let us leave that to a future extension.
That suggests that `manual' should become `file:'.
As for delimiters
> On 17/05/2019, 01:34, "Jan Beulich" <JBeulich@suse.com> wrote:
> > I am not sure what you mean, which colons? Are you saying, the format should be
> > version: 1
> > repo: ...
>
> Yes. This would make it even more prominent that these are tags of
> some sort. But this was only a thought of mine, it's in no way meant
> to be a requirement I have.
It will make writing a parser easier if each entry is a single line
with the fields in a defined order and if we can say that a `repo:'
must precede every `file:' that mentions it.
That sounds perfectly sensible
Version 2 on its way, sometimes next week
Lars
diff --git a/TRACKING.IMPORTS b/TRACKING.IMPORTS new file mode 100644 index 0000000000..39829e078c --- /dev/null +++ b/TRACKING.IMPORTS @@ -0,0 +1,40 @@ +# This file contains information about source files that have been +# copied from other sources and need to be tracked +# +# The file may contain lines starting with ... +# -------------------------------------------- +# version: of file format +# repo: repository definition +# auto|manual: a mapping to track files +# +# Repository Definitions are of the following format +# -------------------------------------------------- +# repo name-of-source-repo git|svn https-url-of-source-repo +# +# For example: +# repo linux-torvalds git https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git +# +# Mappings to track files are of the following format +# --------------------------------------------------- +# manual|auto xen-file name-of-original-repo original-file commit-id +# +# auto: +# The xen-file needs to track the the original-file exactly +# In other words, we can automatically update the file using a script +# +# manual: +# A developer needs to make a decision whether a +# specific change is applied or ignored and update the last commit id +# accordingly +# +# name-of-original-repo: +# A reference to a source repository defined by *repo* keyword +# +# commit id: +# Last commit id of source file that was deemed to be ok +# and either imported into the tree or rejected +# +# For example: +# manual xen/drivers/passthrough/arm/smmu.c linux-torvalds linux/drivers/iommu/arm-smmu.c b77cf11f094136 + +version 1
Following the recent discussion, we had on IRC and the action I had in the March community call, this file provides a file format that enables writing an automated test to check whether files are out of sync. Unlike in the IRC discussion, which suggested a single line for all information, I broke out the repository into a separate statement for - Better readability (aka shorter lines) - Better maintainability if a repo URL changes The list of files that need to be included are Once the file format is agree, I will write a test or script. I also need some more correct test data, aka entries in the file from committers looking after the following files [Jan] xen/arch/x86/cpu/mwait-idle.c [Stefano, Julien - this has to be finalized] xen/drivers/passthrough/arm/smmu.c xen/arch/arm/vgic/* xen/include/asm-arm/div64.h xen/drivers/char/meson-uart.c xen/arch/arm/arm32/lib/* xen/arch/arm/arm64/lib/* xen/arch/arm/arm64/cache.S xen/arch/arm/arm64/bpi.S xen/include/asm-arm/system.h xen/arch/arm/arm64/insn.c [Others?] xen/common/rbtree.c Note that in some cases Linux has diverged and some Linux files have disappeared. Julien also raised the point, that in some cases only a subset of code from Linux Xen files was applied or that only some functions get moved across to Xen. I believe that is entirely OK. The workflow would be in most cases that: - We use a Linux (source) commit as a benchmark and record the commit ID - If there is a change in Linux the test will fail - The committer looks at the diff and either - Decides to ignore it and bumps the commit ID in this file - Decides the change is needed, integrates it into Xen and then bumps the commit ID in this file Signed-off-by: Lars Kurth <lars.kurth@citrix.com> CC: committers@xenproject.org --- TRACKING.IMPORTS | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 TRACKING.IMPORTS