Page MenuHomePhabricator

[Driver] Allow search of included response files as configuration files
Needs ReviewPublic

Authored by sepavloff on Sep 5 2022, 9:44 AM.

Details

Summary

If a configuration file contains directive @fname, the file 'fname'
in it is normally searched for relative to the including configuration
file. In some cases it is not convenient. Configuration parameters may
be logically partitioned into caterories (like 'platform variant' and
'processor variant') and it may be convenient to have different
configuration files for these categories maintained separately. Similar
motivation was described in
https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606.
Simple file inclusion by @file does not help, because in this case the
path to file is specified relative to the directory of the used
configuration file, but it actually is not dependent on the latter. Using
absolute paths is not convenient and it still do not allow user to
easily override the settings.

This change implements option --search-config-dirs, which modifies the
search algorithm, - files included by @file are searched for as
configuration files, in the same directories, and can be overridden by a
user. With this facility a configuration file can include other files
like in:

@platform.cfg
@processor.cfg
...other options...

The effect is as if two configuration files are loaded, both are
searched for in well-known places and can be overridden if user
provides a file with the same name in user configuration directory.

Diff Detail

Event Timeline

sepavloff created this revision.Sep 5 2022, 9:44 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 5 2022, 9:44 AM
Herald added a subscriber: hiraditya. · View Herald Transcript
sepavloff requested review of this revision.Sep 5 2022, 9:44 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptSep 5 2022, 9:44 AM
MaskRay added a comment.EditedSep 5 2022, 9:23 PM

Normally a file included into configuration file by directive @file is
searched for relative the including configuration file. In some cases it
is not convenient.

relative => relative to.

(A configuration file may specify (value of an option) --option=file and (a "child" config file) @cfg. I assume this means @cfg.)

Configuration parameters may be logically partitioned
into caterories (like 'platform variant' and 'processor variant') and it
is convenient to have different configuration files for these
categories maintained separately. Similar motivation was described in
https://discourse.llvm.org/t/rfc-adding-a-default-file-location-to-config-file-support/63606.
Simple file inclusion by @file does not help, because in this case the
path to file is specified relative to the directory of the used
configuration file, but it actually is not dependent on the latter. Using
absolute paths is not convenient and it still do not allow user to
easily override the settings.

There are traditional response files and --config files. I was initially confused.
The patch does not change response files.

For a config file, --search-config-dirs is introduced to change how @cfg is resolved
(relative to --config-system-dir or --config-user-dir=).
I think mentioning the option names in the summary and having an elaborated example in clang/docs/UsersManual.rst will help.

This change implements option --search-config-dirs, which modifies the search algorithm, - files included by @file are searched for as configuration files, in the same directories, and can be overridden by a user.

Is the previous search algorithm sensible? If we think not, we can change the default, and introduce a driver or CC1 option to stay the old behavior.
My reasoning is that I cannot imagine many cases which can be broken by changing the search algorithm.

I think I'm still confused why we need a new option for changing the search algorithm.

Say, we have --config-system-dir=system --config-user-dir=user and three files system/a.cfg, system/b.cfg, user/b.cfg.
system/a.cfg contains @b.
The old algorithm resolves to system/b.cfg (relative to the dir of system/a.cfg: system/) while the new algorithm resolves to user/b.cfg?

clang/include/clang/Driver/Options.td
909

I am confused by the help text.

MaskRay added inline comments.Sep 5 2022, 9:56 PM
llvm/include/llvm/Support/CommandLine.h
2129

Unused

sepavloff updated this revision to Diff 458216.Sep 6 2022, 10:50 AM

Change help message

sepavloff edited the summary of this revision. (Show Details)Sep 6 2022, 10:51 AM

As mentioned, if the new search algorithm makes sense, perhaps use the new algorithm by default and provide a temporary driver option for keeping the old behavior?

aaron.ballman added inline comments.Sep 7 2022, 11:23 AM
clang/docs/UsersManual.rst
953–955
llvm/lib/Support/CommandLine.cpp
1188–1194
1382–1385

Rebased, addressed reviewers' notes.

sepavloff marked 4 inline comments as done.Sep 12 2022, 11:12 AM

For a config file, --search-config-dirs is introduced to change how @cfg is resolved
(relative to --config-system-dir or --config-user-dir=).
I think mentioning the option names in the summary and having an elaborated example in clang/docs/UsersManual.rst will help.

Added simple example in clang/docs/UsersManual.rst.

This change implements option --search-config-dirs, which modifies the search algorithm, - files included by @file are searched for as configuration files, in the same directories, and can be overridden by a user.

Is the previous search algorithm sensible? If we think not, we can change the default, and introduce a driver or CC1 option to stay the old behavior.
My reasoning is that I cannot imagine many cases which can be broken by changing the search algorithm.

There is nothing wrong with the previous algorithm, it is just for a different use case. For example, some users prepare set of config files, which contain set of options for various tasks, and put them to directories, which are easy to access (like ~/p1/dbg.cfg). There files may share bulky information like include paths by using @file directives. Another directory may contain set of configuration files, say, for different platform. Enforcing the new algorithm would break existing work environments and require additional efforts to restore functionality.

One of the use case for the new algorithm is described in the patch description. As the option --search-config-dirs may be specified inside config file, from user perspective nothing changes, they just specify config file with the option --config, as previously. Compatibility is not broken.

I think I'm still confused why we need a new option for changing the search algorithm.

Say, we have --config-system-dir=system --config-user-dir=user and three files system/a.cfg, system/b.cfg, user/b.cfg.
system/a.cfg contains @b.
The old algorithm resolves to system/b.cfg (relative to the dir of system/a.cfg: system/) while the new algorithm resolves to user/b.cfg?

You are right. It simplify preparing working environment by combining its parts (include paths, libraries, preprocessor definitions etc) taken from well-knows place, still allowing a user to modify it without need to copy all environment into a new directory and then changing it.

An alternative implementation is in D136354. Instead of using new command-line option, it uses --config to include file, which is searched for as a configuration file.