mbox series

[v7,00/11] kconfig: Add support for conflict resolution

Message ID 20250208163959.3973163-1-ole0811sch@gmail.com (mailing list archive)
Headers show
Series kconfig: Add support for conflict resolution | expand

Message

Ole Schuerks Feb. 8, 2025, 4:39 p.m. UTC
Hi,

This patch series (for next-20250207) provides support to enable users
to express their desired target configuration and display possible
resolutions to their conflicts. This support is provided within xconfig.

Conflict resolution is provided by translating kconfig's configuration
option tree to a propositional formula and allowing our resolution
algorithm and PicoSAT to calculate the possible fixes for an expressed
target kernel configuration. PicoSAT is the smallest and most robust C SAT
solver we could find which is also GPL compatible. PicoSAT is used as a
dynamically loaded library. For Debian and Fedora, the apt and dnf packages
named "picosat" provide the library. For other distros, we provide
instructions on how PicoSAT needs to be compiled from the sources in the
documentation.

New UI extensions are made to xconfig with panes and buttons to allow users
to express new desired target options, calculate fixes, and apply any of
found solutions.

You can see a YouTube video demonstrating this work [2]. This effort is
part of the kernelnewbies Kconfig-SAT project [3], the approach and effort
is also explained in detail in our paper [4]. The results from the
evaluation have significantly improved since then: Around 80 % of the
conflicts could be resolved, and 99.9 % of the generated fixes resolved the
conflict. It is also our attempt at contributing back to the kernel
community, whose configurator researchers studied a lot.

Patches applicable to next-20250207.

[0] https://gsd.uwaterloo.ca/sites/default/files/vamos12-survey.pdf
[1] https://www.linux-magazine.com/Issues/2021/244/Kconfig-Deep-Dive
[2] https://www.youtube.com/watch?v=vn2JgK_PTbc
[3] https://kernelnewbies.org/KernelProjects/kconfig-sat
[4] http://www.cse.chalmers.se/~bergert/paper/2021-icseseip-configfix.pdf

Thanks from the team! (and thanks to Luis Chamberlain for guiding us here)

Changelog v7:
* improve error handling when loading picosat symbols with dlsym(): use
  dlerror() instead of checking for NULL
* move list_at_index from scripts/include/list.h to
  scripts/kconfig/cf_utils.h
* change interface of list_for_each_entry_from and rename list_size to
  list_count_nodes to more closely match scripts/include/list.h
* change misleading name "rangefix" of fix generation algorithm to
  "fixgen"

Changelog v6:
* remove script for manually installing PicoSAT
* adapt documentation and help text in the GUI accordingly
* convert Qt signal/slot connects to new style

Changelog v5:
* use lists from scripts/include/list.h
* use PicoSAT as a dynamically loaded library
* Fix GUI bug that made the displayed tables editable
* Allow cycling through the desired values of a symbol in the conflict view
  in the GUI by clicking on the cell
* Fix usage of "NO" instead of "N" etc. in some places in the GUI
* Improve function naming
* Add documentation
* Simlify xcalloc to xmalloc in some places
* Fix allocation bug in fexpr_add_to_satmap() and init_data()
* Remove functions pexpr_eliminate_dups() and print_expr()

Ole Schuerks (11):
  kconfig: Add PicoSAT interface
  kconfig: Add definitions
  kbuild: Add list_count_nodes and list_for_each_entry_from
  kconfig: Add files for building constraints
  kconfig: Add files for handling expressions
  kconfig: Add files for fix generation
  kconfig: Add files with utility functions
  kconfig: Add tools
  kconfig: Add xconfig-modifications
  kconfig: Add loader.gif
  kconfig: Add documentation for the conflict resolver

 Documentation/kbuild/kconfig.rst    |   56 +
 scripts/include/list.h              |   26 +
 scripts/kconfig/.gitignore          |    1 +
 scripts/kconfig/Makefile            |   11 +-
 scripts/kconfig/cf_constraints.c    | 1777 ++++++++++++++++++++++++
 scripts/kconfig/cf_constraints.h    |   24 +
 scripts/kconfig/cf_defs.h           |  391 ++++++
 scripts/kconfig/cf_expr.c           | 2003 +++++++++++++++++++++++++++
 scripts/kconfig/cf_expr.h           |  181 +++
 scripts/kconfig/cf_fixgen.c         | 1136 +++++++++++++++
 scripts/kconfig/cf_fixgen.h         |   21 +
 scripts/kconfig/cf_utils.c          |  980 +++++++++++++
 scripts/kconfig/cf_utils.h          |  136 ++
 scripts/kconfig/cfoutconfig.c       |  149 ++
 scripts/kconfig/configfix.c         |  352 +++++
 scripts/kconfig/configfix.h         |   31 +
 scripts/kconfig/expr.h              |   17 +
 scripts/kconfig/loader.gif          |  Bin 0 -> 4177 bytes
 scripts/kconfig/picosat_functions.c |   79 ++
 scripts/kconfig/picosat_functions.h |   35 +
 scripts/kconfig/qconf.cc            |  676 ++++++++-
 scripts/kconfig/qconf.h             |  111 ++
 22 files changed, 8189 insertions(+), 4 deletions(-)
 create mode 100644 scripts/kconfig/cf_constraints.c
 create mode 100644 scripts/kconfig/cf_constraints.h
 create mode 100644 scripts/kconfig/cf_defs.h
 create mode 100644 scripts/kconfig/cf_expr.c
 create mode 100644 scripts/kconfig/cf_expr.h
 create mode 100644 scripts/kconfig/cf_fixgen.c
 create mode 100644 scripts/kconfig/cf_fixgen.h
 create mode 100644 scripts/kconfig/cf_utils.c
 create mode 100644 scripts/kconfig/cf_utils.h
 create mode 100644 scripts/kconfig/cfoutconfig.c
 create mode 100644 scripts/kconfig/configfix.c
 create mode 100644 scripts/kconfig/configfix.h
 create mode 100644 scripts/kconfig/loader.gif
 create mode 100644 scripts/kconfig/picosat_functions.c
 create mode 100644 scripts/kconfig/picosat_functions.h

Comments

Masahiro Yamada Feb. 10, 2025, 5 a.m. UTC | #1
On Sun, Feb 9, 2025 at 1:40 AM Ole Schuerks <ole0811sch@gmail.com> wrote:
>
> Hi,
>
> This patch series (for next-20250207) provides support to enable users
> to express their desired target configuration and display possible
> resolutions to their conflicts. This support is provided within xconfig.
>
> Conflict resolution is provided by translating kconfig's configuration
> option tree to a propositional formula and allowing our resolution
> algorithm and PicoSAT to calculate the possible fixes for an expressed
> target kernel configuration. PicoSAT is the smallest and most robust C SAT
> solver we could find which is also GPL compatible. PicoSAT is used as a
> dynamically loaded library. For Debian and Fedora, the apt and dnf packages
> named "picosat" provide the library. For other distros, we provide
> instructions on how PicoSAT needs to be compiled from the sources in the
> documentation.
>
> New UI extensions are made to xconfig with panes and buttons to allow users
> to express new desired target options, calculate fixes, and apply any of
> found solutions.
>
> You can see a YouTube video demonstrating this work [2]. This effort is
> part of the kernelnewbies Kconfig-SAT project [3], the approach and effort
> is also explained in detail in our paper [4]. The results from the
> evaluation have significantly improved since then: Around 80 % of the
> conflicts could be resolved, and 99.9 % of the generated fixes resolved the
> conflict. It is also our attempt at contributing back to the kernel
> community, whose configurator researchers studied a lot.
>
> Patches applicable to next-20250207.
>
> [0] https://gsd.uwaterloo.ca/sites/default/files/vamos12-survey.pdf
> [1] https://www.linux-magazine.com/Issues/2021/244/Kconfig-Deep-Dive
> [2] https://www.youtube.com/watch?v=vn2JgK_PTbc
> [3] https://kernelnewbies.org/KernelProjects/kconfig-sat
> [4] http://www.cse.chalmers.se/~bergert/paper/2021-icseseip-configfix.pdf
>
> Thanks from the team! (and thanks to Luis Chamberlain for guiding us here)
>
> Changelog v7:
> * improve error handling when loading picosat symbols with dlsym(): use
>   dlerror() instead of checking for NULL
> * move list_at_index from scripts/include/list.h to
>   scripts/kconfig/cf_utils.h
> * change interface of list_for_each_entry_from and rename list_size to
>   list_count_nodes to more closely match scripts/include/list.h
> * change misleading name "rangefix" of fix generation algorithm to
>   "fixgen"



Thanks for this, but I have no plans to merge the SAT solver.

The reason is that my future plan is to move toolchain selection
to the Kconfig stage instead of specifying it statically from the command line.
This approach was suggested by Linus [1], and to achieve that,
the shell evaluation must be dynamically re-evaluated [2].

The SAT solver would likely conflict with this plan. At least due to the
significant amount of additional code, which would be an obstacle.


[1] https://lore.kernel.org/lkml/CAHk-=whdrvCkSWh=BRrwZwNo3=yLBXXM88NGx8VEpP1VTgmkyQ@mail.gmail.com/

[2] https://lore.kernel.org/lkml/CAK7LNATe7Ah-ow9wYGrtL9i4z-VD=MCo=sJjbC_S0ofEoH6CFQ@mail.gmail.com/
Luis Chamberlain Feb. 10, 2025, 3:43 p.m. UTC | #2
On Mon, Feb 10, 2025 at 02:00:52PM +0900, Masahiro Yamada wrote:
> Thanks for this, but I have no plans to merge the SAT solver.
> 
> The reason is that my future plan is to move toolchain selection
> to the Kconfig stage instead of specifying it statically from the command line.

That makes sense.

> This approach was suggested by Linus [1], and to achieve that,
> the shell evaluation must be dynamically re-evaluated [2].

Sure.

> The SAT solver would likely conflict with this plan. At least due to the
> significant amount of additional code, which would be an obstacle.

I can't see how the toolchain selection, if set on Kconfig can't be
leveraged later to enable / disable the SAT solver, however I can
see the amount of code shuffling incurred to be an extra hurdle to
address and a preference to leave that for later.

In other words, I susepct it is still possible to evaluate to
add support for the SAT solver post toolchain kconfig integration.

Thoughts?

 Luis
Masahiro Yamada Feb. 11, 2025, 12:46 a.m. UTC | #3
On Tue, Feb 11, 2025 at 12:43 AM Luis Chamberlain <mcgrof@kernel.org> wrote:
>
> On Mon, Feb 10, 2025 at 02:00:52PM +0900, Masahiro Yamada wrote:
> > Thanks for this, but I have no plans to merge the SAT solver.
> >
> > The reason is that my future plan is to move toolchain selection
> > to the Kconfig stage instead of specifying it statically from the command line.
>
> That makes sense.
>
> > This approach was suggested by Linus [1], and to achieve that,
> > the shell evaluation must be dynamically re-evaluated [2].
>
> Sure.
>
> > The SAT solver would likely conflict with this plan. At least due to the
> > significant amount of additional code, which would be an obstacle.
>
> I can't see how the toolchain selection, if set on Kconfig can't be
> leveraged later to enable / disable the SAT solver, however I can
> see the amount of code shuffling incurred to be an extra hurdle to
> address and a preference to leave that for later.
>
> In other words, I susepct it is still possible to evaluate to
> add support for the SAT solver post toolchain kconfig integration.
>
> Thoughts?


It depends on how the dynamic shell evaluation is implemented.
This is not limited to bool/tristate, but SAT solver only works for
those two types.
Jude Gyimah Feb. 21, 2025, 3:16 a.m. UTC | #4
Quick follow-up. On our end, our SAT-solver implementations can be 
easily adapted to accommodate your future toolchain selection 
refactorings. Also, could you share with us the timelines for your 
refactorings so we can plan and deliver the adjusted SAT-solver patches. 
Best Regards, Jude Gyimah

On 2/11/25 01:46, Masahiro Yamada wrote:
> On Tue, Feb 11, 2025 at 12:43 AM Luis Chamberlain <mcgrof@kernel.org> wrote:
>> On Mon, Feb 10, 2025 at 02:00:52PM +0900, Masahiro Yamada wrote:
>>> Thanks for this, but I have no plans to merge the SAT solver.
>>>
>>> The reason is that my future plan is to move toolchain selection
>>> to the Kconfig stage instead of specifying it statically from the command line.
>> That makes sense.
>>
>>> This approach was suggested by Linus [1], and to achieve that,
>>> the shell evaluation must be dynamically re-evaluated [2].
>> Sure.
>>
>>> The SAT solver would likely conflict with this plan. At least due to the
>>> significant amount of additional code, which would be an obstacle.
>> I can't see how the toolchain selection, if set on Kconfig can't be
>> leveraged later to enable / disable the SAT solver, however I can
>> see the amount of code shuffling incurred to be an extra hurdle to
>> address and a preference to leave that for later.
>>
>> In other words, I susepct it is still possible to evaluate to
>> add support for the SAT solver post toolchain kconfig integration.
>>
>> Thoughts?
>
> It depends on how the dynamic shell evaluation is implemented.
> This is not limited to bool/tristate, but SAT solver only works for
> those two types.
>
>
>
>