diff mbox series

[v3] Add TRACKING.IMPORTS to xen.git to more easily manage imported files that need to be kept in sync with an upstream

Message ID 6250372b05e95412cfd0f8e6f558f2376c4f3d94.1558719481.git.lars.kurth@citrix.com (mailing list archive)
State New, archived
Headers show
Series [v3] Add TRACKING.IMPORTS to xen.git to more easily manage imported files that need to be kept in sync with an upstream | expand

Commit Message

Lars Kurth May 24, 2019, 5:44 p.m. UTC 
Following the recent discussion, we had on IRC and the action I had in 
the March community call, this file provides a file format that 
enables writing an automated test to check whether files are out of sync. 

An example, what file content may look like is embedded below
repo: linux-torvalds git https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
file: xen/drivers/passthrough/arm/smmu.c linux-torvalds linux/drivers/iommu/arm-smmu.c b77cf11f094136

Once the file format is agree, I will write a test or script.

I also need some more correct test data, aka entries in the file from
committers looking after the following files
[Jan]
xen/arch/x86/cpu/mwait-idle.c 
[Stefano, Julien - this has to be finalized]
xen/drivers/passthrough/arm/smmu.c
xen/arch/arm/vgic/*
xen/include/asm-arm/div64.h
xen/drivers/char/meson-uart.c
xen/arch/arm/arm32/lib/*
xen/arch/arm/arm64/lib/*
xen/arch/arm/arm64/cache.S
xen/arch/arm/arm64/bpi.S
xen/include/asm-arm/system.h
xen/arch/arm/arm64/insn.c
[Others?]
xen/common/rbtree.c

Note that in some cases Linux has diverged and some Linux files have disappeared. 
Julien also raised the point, that in some cases only a subset of code from Linux Xen files was applied or that only some functions get moved across to Xen.

I believe that is entirely OK. The workflow would be in most cases that:
- We use a Linux (source) commit as a benchmark and record the commit ID
- If there is a change in Linux the test will fail
- The committer looks at the diff and either
  - Decides to ignore it and bumps the commit ID in this file
  - Decides the change is needed, integrates it into Xen and then 
    bumps the commit ID in this file

Changes since v1
* Require a colon after repo:, file:, ... keywords
* Replace manual:|auto: with file: as there auto: use-case was invalid
* Added more verbose description of format

Changes since v2
* Changed some formatting
* Removed examples
* Removed references to https

Signed-off-by: Lars Kurth <lars.kurth@citrix.com>
CC: committers@xenproject.org
---
 TRACKING.FILES | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 TRACKING.FILES

Comments

Jan Beulich May 27, 2019, 9:40 a.m. UTC | #1 
>>> On 24.05.19 at 19:44, <lars.kurth@citrix.com> wrote:
> Following the recent discussion, we had on IRC and the action I had in 
> the March community call, this file provides a file format that 
> enables writing an automated test to check whether files are out of sync. 
> 
> An example, what file content may look like is embedded below
> repo: linux-torvalds git 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 
> file: xen/drivers/passthrough/arm/smmu.c linux-torvalds 
> linux/drivers/iommu/arm-smmu.c b77cf11f094136
> 
> Once the file format is agree, I will write a test or script.
> 
> I also need some more correct test data, aka entries in the file from
> committers looking after the following files
> [Jan]
> xen/arch/x86/cpu/mwait-idle.c 
> [Stefano, Julien - this has to be finalized]
> xen/drivers/passthrough/arm/smmu.c
> xen/arch/arm/vgic/*
> xen/include/asm-arm/div64.h
> xen/drivers/char/meson-uart.c
> xen/arch/arm/arm32/lib/*
> xen/arch/arm/arm64/lib/*
> xen/arch/arm/arm64/cache.S
> xen/arch/arm/arm64/bpi.S
> xen/include/asm-arm/system.h
> xen/arch/arm/arm64/insn.c
> [Others?]
> xen/common/rbtree.c
> 
> Note that in some cases Linux has diverged and some Linux files have disappeared. 
> Julien also raised the point, that in some cases only a subset of code from 
> Linux Xen files was applied or that only some functions get moved across to 
> Xen.
> 
> I believe that is entirely OK. The workflow would be in most cases that:
> - We use a Linux (source) commit as a benchmark and record the commit ID
> - If there is a change in Linux the test will fail
> - The committer looks at the diff and either
>   - Decides to ignore it and bumps the commit ID in this file
>   - Decides the change is needed, integrates it into Xen and then 
>     bumps the commit ID in this file

With this ...

> Changes since v1
> * Require a colon after repo:, file:, ... keywords
> * Replace manual:|auto: with file: as there auto: use-case was invalid
> * Added more verbose description of format
> 
> Changes since v2
> * Changed some formatting
> * Removed examples
> * Removed references to https

... move past the first --- separator ...

> Signed-off-by: Lars Kurth <lars.kurth@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
Lars Kurth June 20, 2019, 2:18 p.m. UTC | #2 
On 27/05/2019, 10:41, "Jan Beulich" <JBeulich@suse.com> wrote:

    >>> On 24.05.19 at 19:44, <lars.kurth@citrix.com> wrote:
    > Following the recent discussion, we had on IRC and the action I had in 
    > the March community call, this file provides a file format that 
    > enables writing an automated test to check whether files are out of sync. 
    > 
    > An example, what file content may look like is embedded below
    > repo: linux-torvalds git 
    > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 
    > file: xen/drivers/passthrough/arm/smmu.c linux-torvalds 
    > linux/drivers/iommu/arm-smmu.c b77cf11f094136
    > 
    > Once the file format is agree, I will write a test or script.
    > 
    > I also need some more correct test data, aka entries in the file from
    > committers looking after the following files
    > [Jan]
    > xen/arch/x86/cpu/mwait-idle.c 
    > [Stefano, Julien - this has to be finalized]
    > xen/drivers/passthrough/arm/smmu.c
    > xen/arch/arm/vgic/*
    > xen/include/asm-arm/div64.h
    > xen/drivers/char/meson-uart.c
    > xen/arch/arm/arm32/lib/*
    > xen/arch/arm/arm64/lib/*
    > xen/arch/arm/arm64/cache.S
    > xen/arch/arm/arm64/bpi.S
    > xen/include/asm-arm/system.h
    > xen/arch/arm/arm64/insn.c
    > [Others?]
    > xen/common/rbtree.c
    > 
    > Note that in some cases Linux has diverged and some Linux files have disappeared. 
    > Julien also raised the point, that in some cases only a subset of code from 
    > Linux Xen files was applied or that only some functions get moved across to 
    > Xen.
    > 
    > I believe that is entirely OK. The workflow would be in most cases that:
    > - We use a Linux (source) commit as a benchmark and record the commit ID
    > - If there is a change in Linux the test will fail
    > - The committer looks at the diff and either
    >   - Decides to ignore it and bumps the commit ID in this file
    >   - Decides the change is needed, integrates it into Xen and then 
    >     bumps the commit ID in this file
    
    With this ...
    
    > Changes since v1
    > * Require a colon after repo:, file:, ... keywords
    > * Replace manual:|auto: with file: as there auto: use-case was invalid
    > * Added more verbose description of format
    > 
    > Changes since v2
    > * Changed some formatting
    > * Removed examples
    > * Removed references to https
    
    ... move past the first --- separator ...
    
    > Signed-off-by: Lars Kurth <lars.kurth@citrix.com>
    
    Acked-by: Jan Beulich <jbeulich@suse.com>
    
Are any extra ACKs needed?  
Also, I would be willing to write a script which helps checking the file, but it does require some test data (aka Jan, Julien and Stefano) to populate the file
Regards
Lars
Jan Beulich June 21, 2019, 6:14 a.m. UTC | #3 
>>> On 20.06.19 at 16:18, <lars.kurth@citrix.com> wrote:
> On 27/05/2019, 10:41, "Jan Beulich" <JBeulich@suse.com> wrote:
> 
>     >>> On 24.05.19 at 19:44, <lars.kurth@citrix.com> wrote:
>     > Following the recent discussion, we had on IRC and the action I had in 
>     > the March community call, this file provides a file format that 
>     > enables writing an automated test to check whether files are out of sync. 
>     > 
>     > An example, what file content may look like is embedded below
>     > repo: linux-torvalds git 
>     > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 
>     > file: xen/drivers/passthrough/arm/smmu.c linux-torvalds 
>     > linux/drivers/iommu/arm-smmu.c b77cf11f094136
>     > 
>     > Once the file format is agree, I will write a test or script.
>     > 
>     > I also need some more correct test data, aka entries in the file from
>     > committers looking after the following files
>     > [Jan]
>     > xen/arch/x86/cpu/mwait-idle.c 
>     > [Stefano, Julien - this has to be finalized]
>     > xen/drivers/passthrough/arm/smmu.c
>     > xen/arch/arm/vgic/*
>     > xen/include/asm-arm/div64.h
>     > xen/drivers/char/meson-uart.c
>     > xen/arch/arm/arm32/lib/*
>     > xen/arch/arm/arm64/lib/*
>     > xen/arch/arm/arm64/cache.S
>     > xen/arch/arm/arm64/bpi.S
>     > xen/include/asm-arm/system.h
>     > xen/arch/arm/arm64/insn.c
>     > [Others?]
>     > xen/common/rbtree.c
>     > 
>     > Note that in some cases Linux has diverged and some Linux files have disappeared. 
>     > Julien also raised the point, that in some cases only a subset of code from 
>     > Linux Xen files was applied or that only some functions get moved across to 
>     > Xen.
>     > 
>     > I believe that is entirely OK. The workflow would be in most cases that:
>     > - We use a Linux (source) commit as a benchmark and record the commit ID
>     > - If there is a change in Linux the test will fail
>     > - The committer looks at the diff and either
>     >   - Decides to ignore it and bumps the commit ID in this file
>     >   - Decides the change is needed, integrates it into Xen and then 
>     >     bumps the commit ID in this file
>     
>     With this ...
>     
>     > Changes since v1
>     > * Require a colon after repo:, file:, ... keywords
>     > * Replace manual:|auto: with file: as there auto: use-case was invalid
>     > * Added more verbose description of format
>     > 
>     > Changes since v2
>     > * Changed some formatting
>     > * Removed examples
>     > * Removed references to https
>     
>     ... move past the first --- separator ...
>     
>     > Signed-off-by: Lars Kurth <lars.kurth@citrix.com>
>     
>     Acked-by: Jan Beulich <jbeulich@suse.com>
>     
> Are any extra ACKs needed?  

I don't think there's a strict requirement for more, but I think it
would be nice to at least get whoever brought up the tracking
idea to acknowledge that this approach is likely going to fit.
Unless of course it was yourself to bring this up first ...

Jan
Lars Kurth June 21, 2019, 12:07 p.m. UTC | #4 
On 21/06/2019, 07:14, "Jan Beulich" <JBeulich@suse.com> wrote:

    >>> On 20.06.19 at 16:18, <lars.kurth@citrix.com> wrote:
    > On 27/05/2019, 10:41, "Jan Beulich" <JBeulich@suse.com> wrote:
    > 
    >     >>> On 24.05.19 at 19:44, <lars.kurth@citrix.com> wrote:
    >     > Following the recent discussion, we had on IRC and the action I had in 
    >     > the March community call, this file provides a file format that 
    >     > enables writing an automated test to check whether files are out of sync. 
    >     > 
    >     > An example, what file content may look like is embedded below
    >     > repo: linux-torvalds git 
    >     > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 
    >     > file: xen/drivers/passthrough/arm/smmu.c linux-torvalds 
    >     > linux/drivers/iommu/arm-smmu.c b77cf11f094136
    >     > 
    >     > Once the file format is agree, I will write a test or script.
    >     > 
    >     > I also need some more correct test data, aka entries in the file from
    >     > committers looking after the following files
    >     > [Jan]
    >     > xen/arch/x86/cpu/mwait-idle.c 
    >     > [Stefano, Julien - this has to be finalized]
    >     > xen/drivers/passthrough/arm/smmu.c
    >     > xen/arch/arm/vgic/*
    >     > xen/include/asm-arm/div64.h
    >     > xen/drivers/char/meson-uart.c
    >     > xen/arch/arm/arm32/lib/*
    >     > xen/arch/arm/arm64/lib/*
    >     > xen/arch/arm/arm64/cache.S
    >     > xen/arch/arm/arm64/bpi.S
    >     > xen/include/asm-arm/system.h
    >     > xen/arch/arm/arm64/insn.c
    >     > [Others?]
    >     > xen/common/rbtree.c
    >     > 
    >     > Note that in some cases Linux has diverged and some Linux files have disappeared. 
    >     > Julien also raised the point, that in some cases only a subset of code from 
    >     > Linux Xen files was applied or that only some functions get moved across to 
    >     > Xen.
    >     > 
    >     > I believe that is entirely OK. The workflow would be in most cases that:
    >     > - We use a Linux (source) commit as a benchmark and record the commit ID
    >     > - If there is a change in Linux the test will fail
    >     > - The committer looks at the diff and either
    >     >   - Decides to ignore it and bumps the commit ID in this file
    >     >   - Decides the change is needed, integrates it into Xen and then 
    >     >     bumps the commit ID in this file
    >     
    >     With this ...
    >     
    >     > Changes since v1
    >     > * Require a colon after repo:, file:, ... keywords
    >     > * Replace manual:|auto: with file: as there auto: use-case was invalid
    >     > * Added more verbose description of format
    >     > 
    >     > Changes since v2
    >     > * Changed some formatting
    >     > * Removed examples
    >     > * Removed references to https
    >     
    >     ... move past the first --- separator ...
    >     
    >     > Signed-off-by: Lars Kurth <lars.kurth@citrix.com>
    >     
    >     Acked-by: Jan Beulich <jbeulich@suse.com>
    >     
    > Are any extra ACKs needed?  
    
    I don't think there's a strict requirement for more, but I think it
    would be nice to at least get whoever brought up the tracking
    idea to acknowledge that this approach is likely going to fit.
    Unless of course it was yourself to bring this up first ...
    
It was brought up in a community call in February and you, Julien, George, Paul Durrant and Stefano all felt this was a good idea. 
But there was no specific requestor
The trigger of the discussion was a post-mortem of 4.12 (and the ARGO patches) .

Regards
Lars
diff mbox series

Patch

diff --git a/TRACKING.FILES b/TRACKING.FILES
new file mode 100644
index 0000000000..60c47bdefb
--- /dev/null
+++ b/TRACKING.FILES
@@ -0,0 +1,44 @@ 
+# This file contains information about source files that have been
+# copied from other sources and need to be tracked
+#
+# The file may contain lines starting with ...
+# --------------------------------------------
+# version: of file format
+# repo: repository definition
+# file: a mapping to track files
+#
+# Note that repo: entries must come *before* file: entries
+#
+# Repository Definitions are of the following format
+# --------------------------------------------------
+# repo: name-of-source-repo git|svn url-of-source-repo
+#
+# name-of-source-repo
+#   Name of source repository. The name will be used as reference in file:
+#   statements
+#
+# git|svn
+#   Type of source repository
+#
+# url-of-source-repo
+#   URL of source repository
+#
+# Mappings to track files are of the following format
+# ---------------------------------------------------
+# file: xen-file name-of-original-repo original-file commit-id
+#
+# xen-file
+#   Xen file that needs to be tracked
+#
+# name-of-original-repo
+#   A reference to a source repository defined by *repo* keyword
+#
+# original-file
+#   File in original-repo from which we regularly want to merge changes
+#   into xen-file
+#
+# commit id
+#   Last commit id of original-file that was deemed to be ok
+#   and either imported into the tree or rejected
+
+version: 1