Message ID | 20220901205745.323326-10-longman@redhat.com (mailing list archive) |
---|---|
State | New |
Headers | show |
Series | cgroup/cpuset: cpu partition code fixes & enhancements | expand |
Hi Waiman, I love your patch! Perhaps something to improve: [auto build test WARNING on linus/master] [also build test WARNING on v6.0-rc3 next-20220901] [cannot apply to tj-cgroup/for-next] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Waiman-Long/cgroup-cpuset-cpu-partition-code-fixes-enhancements/20220902-050019 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 42e66b1cc3a070671001f8a1e933a80818a192bf reproduce: # https://github.com/intel-lab-lkp/linux/commit/dce03e1a3eb6fce8d7c849c8daeff91ec9a47fc8 git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch --no-tags linux-review Waiman-Long/cgroup-cpuset-cpu-partition-code-fixes-enhancements/20220902-050019 git checkout dce03e1a3eb6fce8d7c849c8daeff91ec9a47fc8 make menuconfig # enable CONFIG_COMPILE_TEST, CONFIG_WARN_MISSING_DOCUMENTS, CONFIG_WARN_ABI_ERRORS make htmldocs If you fix the issue, kindly add following tag where applicable Reported-by: kernel test robot <lkp@intel.com> All warnings (new ones prefixed by >>): >> Documentation/admin-guide/cgroup-v2.rst:2191: WARNING: Malformed table. vim +2191 Documentation/admin-guide/cgroup-v2.rst 2091 2092 cpuset.cpus 2093 A read-write multiple values file which exists on non-root 2094 cpuset-enabled cgroups. 2095 2096 It lists the requested CPUs to be used by tasks within this 2097 cgroup. The actual list of CPUs to be granted, however, is 2098 subjected to constraints imposed by its parent and can differ 2099 from the requested CPUs. 2100 2101 The CPU numbers are comma-separated numbers or ranges. 2102 For example:: 2103 2104 # cat cpuset.cpus 2105 0-4,6,8-10 2106 2107 An empty value indicates that the cgroup is using the same 2108 setting as the nearest cgroup ancestor with a non-empty 2109 "cpuset.cpus" or all the available CPUs if none is found. 2110 2111 The value of "cpuset.cpus" stays constant until the next update 2112 and won't be affected by any CPU hotplug events. 2113 2114 cpuset.cpus.effective 2115 A read-only multiple values file which exists on all 2116 cpuset-enabled cgroups. 2117 2118 It lists the onlined CPUs that are actually granted to this 2119 cgroup by its parent. These CPUs are allowed to be used by 2120 tasks within the current cgroup. 2121 2122 If "cpuset.cpus" is empty, the "cpuset.cpus.effective" file shows 2123 all the CPUs from the parent cgroup that can be available to 2124 be used by this cgroup. Otherwise, it should be a subset of 2125 "cpuset.cpus" unless none of the CPUs listed in "cpuset.cpus" 2126 can be granted. In this case, it will be treated just like an 2127 empty "cpuset.cpus". 2128 2129 Its value will be affected by CPU hotplug events. 2130 2131 cpuset.mems 2132 A read-write multiple values file which exists on non-root 2133 cpuset-enabled cgroups. 2134 2135 It lists the requested memory nodes to be used by tasks within 2136 this cgroup. The actual list of memory nodes granted, however, 2137 is subjected to constraints imposed by its parent and can differ 2138 from the requested memory nodes. 2139 2140 The memory node numbers are comma-separated numbers or ranges. 2141 For example:: 2142 2143 # cat cpuset.mems 2144 0-1,3 2145 2146 An empty value indicates that the cgroup is using the same 2147 setting as the nearest cgroup ancestor with a non-empty 2148 "cpuset.mems" or all the available memory nodes if none 2149 is found. 2150 2151 The value of "cpuset.mems" stays constant until the next update 2152 and won't be affected by any memory nodes hotplug events. 2153 2154 Setting a non-empty value to "cpuset.mems" causes memory of 2155 tasks within the cgroup to be migrated to the designated nodes if 2156 they are currently using memory outside of the designated nodes. 2157 2158 There is a cost for this memory migration. The migration 2159 may not be complete and some memory pages may be left behind. 2160 So it is recommended that "cpuset.mems" should be set properly 2161 before spawning new tasks into the cpuset. Even if there is 2162 a need to change "cpuset.mems" with active tasks, it shouldn't 2163 be done frequently. 2164 2165 cpuset.mems.effective 2166 A read-only multiple values file which exists on all 2167 cpuset-enabled cgroups. 2168 2169 It lists the onlined memory nodes that are actually granted to 2170 this cgroup by its parent. These memory nodes are allowed to 2171 be used by tasks within the current cgroup. 2172 2173 If "cpuset.mems" is empty, it shows all the memory nodes from the 2174 parent cgroup that will be available to be used by this cgroup. 2175 Otherwise, it should be a subset of "cpuset.mems" unless none of 2176 the memory nodes listed in "cpuset.mems" can be granted. In this 2177 case, it will be treated just like an empty "cpuset.mems". 2178 2179 Its value will be affected by memory nodes hotplug events. 2180 2181 cpuset.cpus.partition 2182 A read-write single value file which exists on non-root 2183 cpuset-enabled cgroups. This flag is owned by the parent cgroup 2184 and is not delegatable. 2185 2186 It accepts only the following input values when written to. 2187 2188 ======== ================================ 2189 "member" Non-root member of a partition 2190 "root" Partition root > 2191 "isolated" Partition root without load balancing 2192 ======== ================================ 2193 2194 The root cgroup is always a partition root and its state 2195 cannot be changed. All other non-root cgroups start out as 2196 "member". 2197 2198 When set to "root", the current cgroup is the root of a new 2199 partition or scheduling domain that comprises itself and all 2200 its descendants except those that are separate partition roots 2201 themselves and their descendants. 2202 2203 When set to "isolated", the CPUs in that partition root will 2204 be in an isolated state without any load balancing from the 2205 scheduler. Tasks placed in such a partition with multiple 2206 CPUs should be carefully distributed and bound to each of the 2207 individual CPUs for optimal performance. 2208 2209 The value shown in "cpuset.cpus.effective" of a partition root 2210 is the CPUs that the partition root can dedicate to a potential 2211 new child partition root. The new child subtracts available 2212 CPUs from its parent "cpuset.cpus.effective". 2213 2214 A partition root ("root" or "isolated") can be in one of the 2215 two possible states - valid or invalid. An invalid partition 2216 root is in a degraded state where some state information may 2217 be retained, but behaves more like a "member". 2218 2219 All possible state transitions among "member", "root" and 2220 "isolated" are allowed. 2221 2222 On read, the "cpuset.cpus.partition" file can show the following 2223 values. 2224 2225 ====================== ============================== 2226 "member" Non-root member of a partition 2227 "root" Partition root 2228 "isolated" Partition root without load balancing 2229 "root invalid (<reason>)" Invalid partition root 2230 "isolated invalid (<reason>)" Invalid isolated partition root 2231 ====================== ============================== 2232 2233 In the case of an invalid partition root, a descriptive string on 2234 why the partition is invalid is included within parentheses. 2235 2236 For a partition root to become valid, the following conditions 2237 must be met. 2238 2239 1) The "cpuset.cpus" is exclusive with its siblings , i.e. they 2240 are not shared by any of its siblings (exclusivity rule). 2241 2) The parent cgroup is a valid partition root. 2242 3) The "cpuset.cpus" is not empty and must contain at least 2243 one of the CPUs from parent's "cpuset.cpus", i.e. they overlap. 2244 4) The "cpuset.cpus.effective" cannot be empty unless there is 2245 no task associated with this partition. 2246 2247 External events like hotplug or changes to "cpuset.cpus" can 2248 cause a valid partition root to become invalid and vice versa. 2249 Note that a task cannot be moved to a cgroup with empty 2250 "cpuset.cpus.effective". 2251 2252 For a valid partition root with the sibling cpu exclusivity 2253 rule enabled, changes made to "cpuset.cpus" that violate the 2254 exclusivity rule will invalidate the partition as well as its 2255 sibiling partitions with conflicting cpuset.cpus values. So 2256 care must be taking in changing "cpuset.cpus". 2257 2258 A valid non-root parent partition may distribute out all its CPUs 2259 to its child partitions when there is no task associated with it. 2260 2261 Care must be taken to change a valid partition root to 2262 "member" as all its child partitions, if present, will become 2263 invalid causing disruption to tasks running in those child 2264 partitions. These inactivated partitions could be recovered if 2265 their parent is switched back to a partition root with a proper 2266 set of "cpuset.cpus". 2267 2268 Poll and inotify events are triggered whenever the state of 2269 "cpuset.cpus.partition" changes. That includes changes caused 2270 by write to "cpuset.cpus.partition", cpu hotplug or other 2271 changes that modify the validity status of the partition. 2272 This will allow user space agents to monitor unexpected changes 2273 to "cpuset.cpus.partition" without the need to do continuous 2274 polling. 2275 2276
On Thu, Sep 01, 2022 at 04:57:44PM -0400, Waiman Long wrote: > It accepts only the following input values when written to. > > ======== ================================ > - "root" a partition root > - "member" a non-root member of a partition > + "member" Non-root member of a partition > + "root" Partition root > + "isolated" Partition root without load balancing > ======== ================================ > ><snipped> > + On read, the "cpuset.cpus.partition" file can show the following > + values. > + > + ====================== ============================== > + "member" Non-root member of a partition > + "root" Partition root > + "isolated" Partition root without load balancing > + "root invalid (<reason>)" Invalid partition root > + "isolated invalid (<reason>)" Invalid isolated partition root > + ====================== ============================== > + These tables above produced htmldocs warnings: Documentation/admin-guide/cgroup-v2.rst:2191: WARNING: Malformed table. Text in column margin in table line 4. ======== ================================ "member" Non-root member of a partition "root" Partition root "isolated" Partition root without load balancing ======== ================================ Documentation/admin-guide/cgroup-v2.rst:2229: WARNING: Malformed table. Text in column margin in table line 5. ====================== ============================== "member" Non-root member of a partition "root" Partition root "isolated" Partition root without load balancing "root invalid (<reason>)" Invalid partition root "isolated invalid (<reason>)" Invalid isolated partition root ====================== ============================== I have applied the fixup: ---- >8 ---- diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 76b3ea9fd5c560..77b6faecf066cb 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2185,11 +2185,11 @@ Cpuset Interface Files It accepts only the following input values when written to. - ======== ================================ + ========== ================================ "member" Non-root member of a partition "root" Partition root "isolated" Partition root without load balancing - ======== ================================ + ========== ================================ The root cgroup is always a partition root and its state cannot be changed. All other non-root cgroups start out as @@ -2222,13 +2222,13 @@ Cpuset Interface Files On read, the "cpuset.cpus.partition" file can show the following values. - ====================== ============================== + ============================= ===================================== "member" Non-root member of a partition "root" Partition root "isolated" Partition root without load balancing "root invalid (<reason>)" Invalid partition root "isolated invalid (<reason>)" Invalid isolated partition root - ====================== ============================== + ============================= ===================================== In the case of an invalid partition root, a descriptive string on why the partition is invalid is included within parentheses. Thanks.
On 9/4/22 04:20, Bagas Sanjaya wrote: > On Thu, Sep 01, 2022 at 04:57:44PM -0400, Waiman Long wrote: >> It accepts only the following input values when written to. >> >> ======== ================================ >> - "root" a partition root >> - "member" a non-root member of a partition >> + "member" Non-root member of a partition >> + "root" Partition root >> + "isolated" Partition root without load balancing >> ======== ================================ >> >> <snipped> >> + On read, the "cpuset.cpus.partition" file can show the following >> + values. >> + >> + ====================== ============================== >> + "member" Non-root member of a partition >> + "root" Partition root >> + "isolated" Partition root without load balancing >> + "root invalid (<reason>)" Invalid partition root >> + "isolated invalid (<reason>)" Invalid isolated partition root >> + ====================== ============================== >> + > These tables above produced htmldocs warnings: > > Documentation/admin-guide/cgroup-v2.rst:2191: WARNING: Malformed table. > Text in column margin in table line 4. > > ======== ================================ > "member" Non-root member of a partition > "root" Partition root > "isolated" Partition root without load balancing > ======== ================================ > Documentation/admin-guide/cgroup-v2.rst:2229: WARNING: Malformed table. > Text in column margin in table line 5. > > ====================== ============================== > "member" Non-root member of a partition > "root" Partition root > "isolated" Partition root without load balancing > "root invalid (<reason>)" Invalid partition root > "isolated invalid (<reason>)" Invalid isolated partition root > ====================== ============================== > > I have applied the fixup: > > ---- >8 ---- > > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst > index 76b3ea9fd5c560..77b6faecf066cb 100644 > --- a/Documentation/admin-guide/cgroup-v2.rst > +++ b/Documentation/admin-guide/cgroup-v2.rst > @@ -2185,11 +2185,11 @@ Cpuset Interface Files > > It accepts only the following input values when written to. > > - ======== ================================ > + ========== ================================ > "member" Non-root member of a partition > "root" Partition root > "isolated" Partition root without load balancing > - ======== ================================ > + ========== ================================ > > The root cgroup is always a partition root and its state > cannot be changed. All other non-root cgroups start out as > @@ -2222,13 +2222,13 @@ Cpuset Interface Files > On read, the "cpuset.cpus.partition" file can show the following > values. > > - ====================== ============================== > + ============================= ===================================== > "member" Non-root member of a partition > "root" Partition root > "isolated" Partition root without load balancing > "root invalid (<reason>)" Invalid partition root > "isolated invalid (<reason>)" Invalid isolated partition root > - ====================== ============================== > + ============================= ===================================== > > In the case of an invalid partition root, a descriptive string on > why the partition is invalid is included within parentheses. > > Thanks. Thanks for the fixes. Will apply that. Cheers, Longman
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index be4a77baf784..76b3ea9fd5c5 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2186,74 +2186,92 @@ Cpuset Interface Files It accepts only the following input values when written to. ======== ================================ - "root" a partition root - "member" a non-root member of a partition + "member" Non-root member of a partition + "root" Partition root + "isolated" Partition root without load balancing ======== ================================ - When set to be a partition root, the current cgroup is the - root of a new partition or scheduling domain that comprises - itself and all its descendants except those that are separate - partition roots themselves and their descendants. The root - cgroup is always a partition root. - - There are constraints on where a partition root can be set. - It can only be set in a cgroup if all the following conditions - are true. - - 1) The "cpuset.cpus" is not empty and the list of CPUs are - exclusive, i.e. they are not shared by any of its siblings. - 2) The parent cgroup is a partition root. - 3) The "cpuset.cpus" is also a proper subset of the parent's - "cpuset.cpus.effective". - 4) There is no child cgroups with cpuset enabled. This is for - eliminating corner cases that have to be handled if such a - condition is allowed. - - Setting it to partition root will take the CPUs away from the - effective CPUs of the parent cgroup. Once it is set, this - file cannot be reverted back to "member" if there are any child - cgroups with cpuset enabled. - - A parent partition cannot distribute all its CPUs to its - child partitions. There must be at least one cpu left in the - parent partition. - - Once becoming a partition root, changes to "cpuset.cpus" is - generally allowed as long as the first condition above is true, - the change will not take away all the CPUs from the parent - partition and the new "cpuset.cpus" value is a superset of its - children's "cpuset.cpus" values. - - Sometimes, external factors like changes to ancestors' - "cpuset.cpus" or cpu hotplug can cause the state of the partition - root to change. On read, the "cpuset.sched.partition" file - can show the following values. - - ============== ============================== - "member" Non-root member of a partition - "root" Partition root - "root invalid" Invalid partition root - ============== ============================== - - It is a partition root if the first 2 partition root conditions - above are true and at least one CPU from "cpuset.cpus" is - granted by the parent cgroup. - - A partition root can become invalid if none of CPUs requested - in "cpuset.cpus" can be granted by the parent cgroup or the - parent cgroup is no longer a partition root itself. In this - case, it is not a real partition even though the restriction - of the first partition root condition above will still apply. - The cpu affinity of all the tasks in the cgroup will then be - associated with CPUs in the nearest ancestor partition. - - An invalid partition root can be transitioned back to a - real partition root if at least one of the requested CPUs - can now be granted by its parent. In this case, the cpu - affinity of all the tasks in the formerly invalid partition - will be associated to the CPUs of the newly formed partition. - Changing the partition state of an invalid partition root to - "member" is always allowed even if child cpusets are present. + The root cgroup is always a partition root and its state + cannot be changed. All other non-root cgroups start out as + "member". + + When set to "root", the current cgroup is the root of a new + partition or scheduling domain that comprises itself and all + its descendants except those that are separate partition roots + themselves and their descendants. + + When set to "isolated", the CPUs in that partition root will + be in an isolated state without any load balancing from the + scheduler. Tasks placed in such a partition with multiple + CPUs should be carefully distributed and bound to each of the + individual CPUs for optimal performance. + + The value shown in "cpuset.cpus.effective" of a partition root + is the CPUs that the partition root can dedicate to a potential + new child partition root. The new child subtracts available + CPUs from its parent "cpuset.cpus.effective". + + A partition root ("root" or "isolated") can be in one of the + two possible states - valid or invalid. An invalid partition + root is in a degraded state where some state information may + be retained, but behaves more like a "member". + + All possible state transitions among "member", "root" and + "isolated" are allowed. + + On read, the "cpuset.cpus.partition" file can show the following + values. + + ====================== ============================== + "member" Non-root member of a partition + "root" Partition root + "isolated" Partition root without load balancing + "root invalid (<reason>)" Invalid partition root + "isolated invalid (<reason>)" Invalid isolated partition root + ====================== ============================== + + In the case of an invalid partition root, a descriptive string on + why the partition is invalid is included within parentheses. + + For a partition root to become valid, the following conditions + must be met. + + 1) The "cpuset.cpus" is exclusive with its siblings , i.e. they + are not shared by any of its siblings (exclusivity rule). + 2) The parent cgroup is a valid partition root. + 3) The "cpuset.cpus" is not empty and must contain at least + one of the CPUs from parent's "cpuset.cpus", i.e. they overlap. + 4) The "cpuset.cpus.effective" cannot be empty unless there is + no task associated with this partition. + + External events like hotplug or changes to "cpuset.cpus" can + cause a valid partition root to become invalid and vice versa. + Note that a task cannot be moved to a cgroup with empty + "cpuset.cpus.effective". + + For a valid partition root with the sibling cpu exclusivity + rule enabled, changes made to "cpuset.cpus" that violate the + exclusivity rule will invalidate the partition as well as its + sibiling partitions with conflicting cpuset.cpus values. So + care must be taking in changing "cpuset.cpus". + + A valid non-root parent partition may distribute out all its CPUs + to its child partitions when there is no task associated with it. + + Care must be taken to change a valid partition root to + "member" as all its child partitions, if present, will become + invalid causing disruption to tasks running in those child + partitions. These inactivated partitions could be recovered if + their parent is switched back to a partition root with a proper + set of "cpuset.cpus". + + Poll and inotify events are triggered whenever the state of + "cpuset.cpus.partition" changes. That includes changes caused + by write to "cpuset.cpus.partition", cpu hotplug or other + changes that modify the validity status of the partition. + This will allow user space agents to monitor unexpected changes + to "cpuset.cpus.partition" without the need to do continuous + polling. Device controller
Update Documentation/admin-guide/cgroup-v2.rst on the newly introduced "isolated" cpuset partition type as well as other changes made in other cpuset patches. Signed-off-by: Waiman Long <longman@redhat.com> --- Documentation/admin-guide/cgroup-v2.rst | 150 +++++++++++++----------- 1 file changed, 84 insertions(+), 66 deletions(-)