This is an archive of the discontinued LLVM Phabricator instance.

[SYCL] Driver option to enable SYCL mode and select SYCL version
ClosedPublic

Authored by bader on Jan 16 2020, 10:12 AM.

Details

Summary

User can select the version of SYCL the compiler will
use via the flag -sycl-std, similar to -cl-std.

The flag defines the LangOpts.SYCLVersion option to the
version of SYCL. The default value is undefined.
If driver is building SYCL code, flag is set to the default SYCL
version (1.2.1)

The preprocessor uses this variable to define CL_SYCL_LANGUAGE_VERSION macro,
which should be defined according to SYCL 1.2.1 standard.

Only valid value at this point for the flag is 1.2.1.

Co-Authored-By: David Wood <Q0KPU0H1YOEPHRY1R2SN5B5RL@david.davidtw.co>
Signed-off-by: Ruyman Reyes <ruyman@codeplay.com>
Signed-off-by: Alexey Bader <alexey.bader@intel.com>

Diff Detail

Event Timeline

bader created this revision.Jan 16 2020, 10:12 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 16 2020, 10:12 AM
Ruyk added a subscriber: Ruyk.Jan 21 2020, 8:36 AM

Maybe we should use the year of issue (2015 instead of 1.2.1) for the -sycl-std version? That would be more stable for the upcoming SYCL versions, and match somehow the C++ versioning.

bader marked 2 inline comments as done.Jan 22 2020, 4:54 AM

Maybe we should use the year of issue (2015 instead of 1.2.1) for the -sycl-std version? That would be more stable for the upcoming SYCL versions, and match somehow the C++ versioning.

Sounds good to me. I'll update the patch. I also have a couple of design related questions below.

clang/include/clang/Basic/LangOptions.def
207

All other language options controlling standard versions are added as "LANGOPT" i.e. int. Why SYCLVersion is different?
@Ruyk, do you think we should convert other options (e.g. OpenCL) to enums as well?

clang/include/clang/Driver/Options.td
3426

What do you think we integrate sycl versions to existing clang options controlling language version: -std.
As far as I can see it's used for all the C/C+ extensions like OpenMP/OpenCL/CUDA/HIP/ObjC.

If I understand correctly clang supports -cl-std only because it's required by OpenCL standard. Similar option (i.e. -sycl-std) is not required by the SYCL specification, so using -std is more aligned with existing clang design.

See clang/include/clang/Basic/LangStandard.h and clang/include/clang/Basic/LangStandards.def.

bader updated this revision to Diff 241757.Jan 31 2020, 9:40 AM

Applied suggestion from Ruyman.

Unit tests: pass. 62372 tests passed, 0 failed and 839 were skipped.

clang-tidy: fail. clang-tidy found 0 errors and 1 warnings. 0 of them are added as review comments below (why?).

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

bader updated this revision to Diff 241872.Feb 1 2020, 2:39 AM

Fix clang-format and clang-tidy issues reported by merge_guards_bot

Unit tests: pass. 62372 tests passed, 0 failed and 839 were skipped.

clang-tidy: pass.

clang-format: pass.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

ABataev added inline comments.Feb 3 2020, 7:17 AM
clang/include/clang/Driver/Options.td
3423–3425

Better to split this into 2 parts: the first for -fsycl and the second for -sycl-std=.

bader marked an inline comment as done.Feb 3 2020, 9:02 AM
bader added inline comments.
clang/include/clang/Driver/Options.td
3423–3425

Okay. Just one question:

Do you know if it's okay to have two commits per review request (like the first version of the "patch" in this review - it was split as you suggest)? Or I should create two separate review requests?

ABataev added inline comments.Feb 3 2020, 9:11 AM
clang/include/clang/Driver/Options.td
3423–3425

Separate patches must have separate review requests

Ruyk added inline comments.Feb 4 2020, 4:36 AM
clang/include/clang/Basic/LangOptions.def
207

Thats a good point. I don't see strong reasons for the enum, basically I read the comment in https://code.woboq.org/llvm/clang/include/clang/Basic/LangOptions.def.html#22

// ENUM_LANGOPT: for options that have enumeration, rather than unsigned, type.

And since there is a known set of SYCL specifications, made more sense to enumerate it.
It also simplifies writing variants to 1.2.1 (e.g. 1.2.1-oneapi) in the code since then you can add another entry to the enum.

But no strong feelings, so feel free to change it.

clang/include/clang/Driver/Options.td
3426

In the case of SYCL, you may want to compile your code with C++17 and SYCL 2015, in which case you need both -std=c++17 and -sycl=sycl-2015 .
SYCL specification mandates a minimum C++ version but users can write code on newer versions as long as the code in the kernel scope is still valid.

clang/lib/Driver/ToolChains/Clang.cpp
4052

This should probably change to 2015 if we are allowing that, just for consistency

bader updated this revision to Diff 242845.Feb 6 2020, 2:38 AM

Applied suggestions from Alexey and Ruyman and rebased on ToT.

Unit tests: unknown.

clang-tidy: pass.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

ABataev added inline comments.Feb 6 2020, 6:32 AM
clang/include/clang/Basic/LangOptions.h
122 ↗(On Diff #242845)

s/undefined/Undefined/g

122 ↗(On Diff #242845)

Do you use SYCL_1_2_1 anywhere in the code? I don't see it is used and you can drop this enum.

clang/lib/Frontend/CompilerInvocation.cpp
2543–2544

It does not match the list of values in Options.td file

2553

Can OPT_fsycl flag be ever passed to the frontend? Also, seems to me the driver has a special case already for device mode, so this code must be the dead code, actually.

bader updated this revision to Diff 242909.Feb 6 2020, 8:04 AM
bader marked 2 inline comments as done.

Applied Alexey's comments.

ABataev added inline comments.Feb 6 2020, 8:08 AM
clang/lib/Driver/ToolChains/Clang.cpp
4047–4048

Should this option also be controlled by -fsycl?

bader marked 7 inline comments as done.Feb 6 2020, 8:28 AM
bader added inline comments.
clang/lib/Driver/ToolChains/Clang.cpp
4047–4048

Make sense to me.
@Ruyk, are you okay with that?

@ABataev, do you have any suggestion on what we should we do if -sycl-std option is used w/o -fsycl? Ignore or report warning/error?

ABataev added inline comments.Feb 6 2020, 8:49 AM
clang/lib/Driver/ToolChains/Clang.cpp
4047–4048

Mark this option as NoArgumentUnused (so the compiler does not report unused option) and ignore it.

bader marked an inline comment as done.Feb 6 2020, 8:57 AM
bader added inline comments.
clang/lib/Driver/ToolChains/Clang.cpp
4047–4048

Ok. Is it okay if we add one more flag - CoreOption? We'd like enable these options fir clang-cl as well.

ABataev added inline comments.Feb 6 2020, 9:01 AM
clang/lib/Driver/ToolChains/Clang.cpp
4047–4048

Hard to say, need to look at the patch.

bader updated this revision to Diff 243029.Feb 6 2020, 3:11 PM

Ignore -sycl-std if it used in non-SYCL mode.

ABataev added inline comments.Feb 7 2020, 6:36 AM
clang/include/clang/Driver/Options.td
3423–3425

These flags should not be ignored, NoArgumentUnused should be applied to this flags.

clang/lib/Driver/ToolChains/Clang.cpp
5347–5349

This code is not required, you already forwarded sycl-std to the frontend earlier

clang/lib/Frontend/CompilerInvocation.cpp
2540

I think processing of sycl-std in the frontend also must be controlled by some high-level option, like -fsycl or something like this. Without this -fsycl-like option this std option also must be ignored.

bader updated this revision to Diff 243173.Feb 7 2020, 8:01 AM
bader marked 2 inline comments as done.

Applied code review comments.

bader marked an inline comment as done.Feb 7 2020, 8:01 AM
bader added inline comments.
clang/include/clang/Driver/Options.td
3423–3425

Do you mean "NoArgumentUnused should not be applied"?

ABataev added inline comments.Feb 7 2020, 8:10 AM
clang/include/clang/Driver/Options.td
3423–3425

Yes, missed not

ABataev added inline comments.Feb 7 2020, 8:16 AM
clang/lib/Frontend/CompilerInvocation.cpp
2538

-fno-sycl should not be passed to frontend. Just use hasFlag(OPT_fsycl).

2539

This option also must be controlled by -fsycl:

Opts.SYCLIsDevice =  Opts.SYCL && Args.hasArg(options::OPT_fsycl_is_device);
bader marked an inline comment as done.Feb 7 2020, 8:24 AM
bader added inline comments.
clang/lib/Frontend/CompilerInvocation.cpp
2539

Does it really has to? This logic is already present in the driver and it makes front-end tests verbose %clang_cc1 -fsycl -fsycl-is-device.
Can -fsycl-is-device imply -fsycl?
Looking how CUDA/OpenMP options are handled, not all of them are processed using this pattern.

ABataev added inline comments.Feb 7 2020, 10:05 AM
clang/lib/Frontend/CompilerInvocation.cpp
2539

In general, this is how we handle it in OpenMP. Cuda works differently, because it has its own kind of files (.cu) and Cuda is triggered by the language switch (-x cu). Seems to me, you're using something close to OpenMP model, no? Or do you want to define your own language kind just like Cuda?

bader updated this revision to Diff 243856.Feb 11 2020, 7:05 AM
bader marked 3 inline comments as done.

Applied review commits from Alexey.

bader marked 3 inline comments as done.Feb 11 2020, 7:09 AM
bader added inline comments.
clang/lib/Frontend/CompilerInvocation.cpp
2539

I applied you suggest, although I don't fully understand the need of using two options instead of two. I would prefer having following code:

Opts.SYCLIsDevice = Args.hasArg(options::OPT_fsycl_is_device);
Opts.SYCL = Args.hasArg(options::OPT_fsycl) || Opts.SYCLIsDevice; // -fsycl-is-device enable SYCL mode as well
ABataev added inline comments.Feb 11 2020, 7:22 AM
clang/lib/Frontend/CompilerInvocation.cpp
2539

I'm not quite familiar with SYCL model, maybe this the right approach. You'd better try to provide more details. Are there any differences between just SYCL and SYCL-device compilation modes? How do you see the compilation sequence in general? At first you're going to compile the host version of the code, then the device? OR something different?

bader marked 2 inline comments as done.Feb 11 2020, 8:27 AM
bader added inline comments.
clang/lib/Frontend/CompilerInvocation.cpp
2539

I think SYCL model is quite similar to OpenMP model. One significant difference might be that to implement standard SYCL functionality we don't need any modifications for the host compiler. AFAIK, OpenMP compiler has to support OpenMP pragmas.
We have a few attributes for Intel FPGA devices, which we can't pre-process with __SYCL_DEVICE_ONLY__ macro and we have added "SYCL-host" mode to suppress compiler warnings about attributes ignored on the host. I think there might be other options how this can be achieved w/o adding new compilation mode and use regular C++ front-end as SYCL host compiler.
I think there is a difference between SYCL and SYCL-device modes, but it's mostly changes the compilation workflow in the driver, but not in the front-end. In SYCL-device mode, driver invokes only one front-end instance to generate offload code. In SYCL mode, driver invokes multiple front-end instances: one in SYCL-device mode and one in regular C++ mode (to be accurate we use SYCL-host mode, but as I mentioned above I don't think it really needed).
I hope it makes it clear now. Let me know if you have any other questions.

Do I understand it correctly that OpenMP option enables OpenMP mode, which is equivalent of "SYCL-host" mode and enabling both OpenMP and OpenMPIsDevice is required for enabling OpenMP mode, which is similar to SYCL-device mode?
If so, can we assume that OpenMPIsDevice implies that OpenMP option is also set (implicitly)?

ABataev added inline comments.Feb 11 2020, 8:35 AM
clang/lib/Frontend/CompilerInvocation.cpp
2539

Do I understand it correctly that OpenMP option enables OpenMP mode, which is equivalent of "SYCL-host" mode and enabling both OpenMP and OpenMPIsDevice is required for enabling OpenMP mode, which is similar to SYCL-device mode?

Well, for driver you need to pass -fopenmp -fopenmp-target=<list_of_targets> to enable the compilation with offloading support. In the frontend the host part is compiled with -fopenmp only (+ aux-triple, probably), for devices - -fopenmp -fopenmp-is-device. Without -fopenmp -fopenmp-is-device is just ignored.

bader marked 2 inline comments as done.Feb 11 2020, 8:44 AM
bader added inline comments.
clang/lib/Frontend/CompilerInvocation.cpp
2539

What is the reason to require the driver to pass both options for the devices? It sounds like -fopenmp-is-device should be enough to differentiate from the host part (-fopenmp only). Right?

ABataev added inline comments.Feb 11 2020, 8:55 AM
clang/lib/Frontend/CompilerInvocation.cpp
2539

We treat a little bit differently. -fopenmp turns support for OpenMP, while -fopenmp-is-device turns processing for device compilation.

bader marked 2 inline comments as done.Feb 11 2020, 11:02 AM
bader added inline comments.
clang/lib/Frontend/CompilerInvocation.cpp
2539

I'm okay to use OpenMP way to treat front-end options as they impact only clang developers (more verbose LIT test + a little bit more logic in the code), but I'd like to keep driver options handling simpler for clang users.
We can discuss this later, when -fsycl-device-only driver option is added.

Do you have any other comments to this change?

BTW, thanks a lot for take your time to review SYCL patches - it's highly appreciated!

Ruyk added a subscriber: Naghasan.Feb 17 2020, 12:43 PM

@Naghasan please take a look

Naghasan added inline comments.Feb 18 2020, 1:41 AM
clang/include/clang/Basic/LangOptions.def
207

int allows the use of relational operators which should ease version managements.

As SYCLVersionList is a strongly typed enum so this may not be the best, and as the SYCL version are now meant to be a year int should do just fine.

clang/include/clang/Driver/Options.td
3426

+1 on this, ComputeCpp used to mix-up both and this proved to be complex to manage. It also integrates better with build systems.

clang/lib/Frontend/CompilerInvocation.cpp
2539
We treat a little bit differently. -fopenmp turns support for OpenMP, while -fopenmp-is-device turns processing for device compilation.

Seems to me that SYCL is kind of an hybrid here.

@bader do you plan on raising kernel specific diagnostics even on the host side ? In this case reusing this model will fully make sense.

bader marked 3 inline comments as done.Feb 18 2020, 1:59 AM
bader added inline comments.
clang/include/clang/Basic/LangOptions.def
207

Okay. I'll change the type from enum to int.

clang/include/clang/Driver/Options.td
3426

Okay. I'll leave it as a separate option.

clang/lib/Frontend/CompilerInvocation.cpp
2539

do you plan on raising kernel specific diagnostics even on the host side ? In this case reusing this model will fully make sense.

Not sure I understand the question. Why would we need raising kernel specific diagnostics on the host side? Is it against the SYCL design principal?

ABataev added inline comments.Feb 18 2020, 7:02 AM
clang/lib/Frontend/CompilerInvocation.cpp
2540

Just Opt.SYCL should be enough

clang/lib/Frontend/InitPreprocessor.cpp
465

Must be guarded with LangOpts.SYCL:

if (LangOpts.SYCL) ...
bader updated this revision to Diff 245213.Feb 18 2020, 11:00 AM

Address comments from Victor and Alexey.

keryell added inline comments.
clang/include/clang/Driver/Options.td
3428

I suggest replacing all the 2015 by 2017.
While this is true SYCL 1.2 was published in 2015, SYCL 1.2.1 was published in 2017. Only 1.2.1 matters here since 1.2 was never fully implemented by any conformant implementation. https://en.wikipedia.org/wiki/SYCL

clang/lib/Driver/ToolChains/Clang.cpp
4051

Replace 2015 by 2017 in both lines above.

clang/lib/Frontend/CompilerInvocation.cpp
2545

Replace 2015 by 2017.

clang/lib/Frontend/InitPreprocessor.cpp
466

Replace 2015 by 2017.

clang/test/Driver/sycl.c
5

Replace all the 2015 by 2017 here and below.

bader updated this revision to Diff 245373.Feb 19 2020, 4:05 AM
bader marked 12 inline comments as done.

Rebase to ToT and address comments from Ronan.

bader marked an inline comment as done.Feb 19 2020, 4:07 AM
bader added inline comments.
clang/include/clang/Driver/Options.td
3428

Done.

bader marked 3 inline comments as done.Feb 19 2020, 4:09 AM
bader updated this revision to Diff 246072.Feb 22 2020, 2:58 AM

Rebase.

Any other comments?

ABataev accepted this revision.Feb 22 2020, 3:33 AM

If no other comments, LGTM

This revision is now accepted and ready to land.Feb 22 2020, 3:33 AM
thakis added a subscriber: thakis.Feb 27 2020, 8:30 AM

This landed here: https://github.com/llvm/llvm-project/commit/bd97704eaaaab5a95ecb048ce343c1a4be5d94e5

It broke tests on mac: http://45.33.8.238/mac/9011/step_7.txt

Please take a look, and if it takes a while please revert while you investigate.

bader added a comment.Feb 27 2020, 8:36 AM

This landed here: https://github.com/llvm/llvm-project/commit/bd97704eaaaab5a95ecb048ce343c1a4be5d94e5

It broke tests on mac: http://45.33.8.238/mac/9011/step_7.txt

Please take a look, and if it takes a while please revert while you investigate.

@thakis, thank for letting me know. I've reverted it at 740ed617f7d4d16e7883636c5eff994f8be7eba4. Sorry for inconvenience.

This revision was automatically updated to reflect the committed changes.
bader added a comment.Mar 1 2020, 7:34 AM

This landed here: https://github.com/llvm/llvm-project/commit/bd97704eaaaab5a95ecb048ce343c1a4be5d94e5

It broke tests on mac: http://45.33.8.238/mac/9011/step_7.txt

Please take a look, and if it takes a while please revert while you investigate.

@thakis, thank for letting me know. I've reverted it at 740ed617f7d4d16e7883636c5eff994f8be7eba4. Sorry for inconvenience.

@thakis, could you help to find the logs for the "build" step?
I don't have access to a Mac system and I can't reproduce the problem on my Linux system, but just looking at the log it looks like that changes to clang/include/clang/Driver/Options.td were not applied.

<stdin>:7:42: note: possible intended match here
clang: warning: argument unused during compilation: '-fsycl' [-Wunused-command-line-argument]

I'd like to make sure that clang is built with the patch applied.

I also noticed that on your system uses gn instead of cmake. Can it be that there is a missing dependency in GN files, which lead to skipping clang re-build?