This is an archive of the discontinued LLVM Phabricator instance.

Set Windows console mode to enable support for ansi escape codes
ClosedPublic

Authored by xbolva00 on Sep 4 2018, 2:17 AM.

Diff Detail

Repository
rLLDB LLDB

Event Timeline

xbolva00 created this revision.Sep 4 2018, 2:17 AM
teemperor requested changes to this revision.Sep 4 2018, 3:45 AM

Thanks! Didn't know we can activate this setting for the user.

Some questions though: Should we restore this setting to the original value after LLDB shuts down? And what happens if this code runs on a Windows version before ENABLE_VIRTUAL_TERMINAL_PROCESSING was introduced? I see there some fallback code, but it's not clear to me what it does and what will happen on older platforms.

source/Core/Debugger.cpp
56

Can you document why this has to be 0x0004? I assume older Windows versions won't have ENABLE_VIRTUAL_TERMINAL_PROCESSING, so that should also be documented that this code is covering this case.

815

A comment here would be nice. E.g. `// Enabling use of ANSI color codes because LLDB is using them to highlight text.

This revision now requires changes to proceed.Sep 4 2018, 3:45 AM

Thanks! Didn't know we can activate this setting for the user.

Some questions though: Should we restore this setting to the original value after LLDB shuts down?

Good question. I tried run lldb, quit it and then I ran echo some ansi code in the same terminal. Ansi code wasn't resolved (same behaviour as before LLDB launch), so LLDB didn't modify something globally in the terminal.
I think we don't need to restore original value, but... suggestions welcome! :)

And what happens if this code runs on a Windows version before ENABLE_VIRTUAL_TERMINAL_PROCESSING was introduced? I see there some fallback code, but it's not clear to me what it does and what will happen on older platforms.

On older systems there is no ENABLE_VIRTUAL_TERMINAL_PROCESSING so we have
SetConsoleMode(0x004). On older systems Windows does not recognize 0x004 and function will return false.

source/Core/Debugger.cpp
56
815

Alright, will do.

xbolva00 added inline comments.Sep 4 2018, 6:15 AM
source/Core/Debugger.cpp
819

Should I rewrite it to more clear version like

bool ansi_supported = SetConsoleMode(hConsole, consoleMode);
SetUseColor(ansi_supported);

?

xbolva00 updated this revision to Diff 163820.Sep 4 2018, 7:52 AM
  • Addressed review comments
xbolva00 marked 5 inline comments as done.Sep 4 2018, 7:54 AM
xbolva00 planned changes to this revision.Sep 4 2018, 11:35 AM

I will reuse the accepted code which arrives with https://reviews.llvm.org/D51611 here also.

xbolva00 updated this revision to Diff 163882.Sep 4 2018, 11:47 AM
  • Updated

@teemperor please take a look again :)

LGTM from my side, but @zturner should take a look too.

zturner added inline comments.Sep 5 2018, 9:08 AM
source/Core/Debugger.cpp
53

This should be #include "lldb/Host/windows/windows.h". Otherwise you're getting the system windows.h file. If that's what you wanted, then it should be <windows.h> instead of "windows.h", but that's never what you want, we should always be including LLDB's wrapper around windows.h instead.

819

Shouldn't we change this to call llvm::sys::Process::UseANSIEscapeCodes(true);?

xbolva00 updated this revision to Diff 164053.Sep 5 2018, 9:23 AM
  • use llvm::sys::Process::UseANSIEscapeCodes
xbolva00 marked 2 inline comments as done.Sep 5 2018, 9:23 AM
zturner accepted this revision.Sep 5 2018, 9:37 AM

lgtm after the one suggested change to the pre-processor conditional.

source/Core/Debugger.cpp
814

Can you also add defined(_WIN32) to this condition?

815

As a future cleanup we should probably add a function to LLVM called AreAnsiEscapeCodesSupported(). Then you could just write:

if (llvm::sys::Process::AreAnsiEscapeCodesSupported())
  llvm::sys::Process::UseAnsiEscapeCodes(true);

and get rid of the preprocessor comment. Don't need to do it in this patch though.

xbolva00 updated this revision to Diff 164057.Sep 5 2018, 9:37 AM
xbolva00 updated this revision to Diff 164058.
  • Check for _WIN32
This revision was not accepted when it landed; it landed in state Needs Review.Sep 5 2018, 3:10 PM
This revision was automatically updated to reflect the committed changes.

This change causes one of the LLDB tests to fail on Windows. More specifically:

********************
FAIL: lldb :: Settings/TestDisableColor.test (42646 of 43956)
******************** TEST 'lldb :: Settings/TestDisableColor.test' FAILED ********************
          Script:
          --
          : 'RUN: at line 1';   E:\_work\36\b\LLVMBuild\Release\bin\lldb.EXE -S E:/_work/36/s/llvm/tools/lldb/lit/lit-lldb-init -x -b -s E:\_work\36\s\llvm\tools\lldb\lit\Settings\TestDisableColor.test | E:\_work\36\b\LLVMBuild\Release\bin\FileCheck.EXE E:\_work\36\s\llvm\tools\lldb\lit\Settings\TestDisableColor.test
          --
          Exit Code: 1
          
          Command Output (stdout):
          --
          $ ":" "RUN: at line 1"
          $ "E:\_work\36\b\LLVMBuild\Release\bin\lldb.EXE" "-S" "E:/_work/36/s/llvm/tools/lldb/lit/lit-lldb-init" "-x" "-b" "-s" "E:\_work\36\s\llvm\tools\lldb\lit\Settings\TestDisableColor.test"
          $ "E:\_work\36\b\LLVMBuild\Release\bin\FileCheck.EXE" "E:\_work\36\s\llvm\tools\lldb\lit\Settings\TestDisableColor.test"
          # command stderr:
 ##[error]llvm\tools\lldb\lit\Settings\TestDisableColor.test(7,10): Error GDC2C77F0: CHECK: expected string not found in input
      1>E:\_work\36\s\llvm\tools\lldb\lit\Settings\TestDisableColor.test(7,10): error GDC2C77F0: CHECK: expected string not found in input [e:\_work\36\b\LLVMBuild\check-all.vcxproj]
          
          # CHECK: use-color (boolean) = false
          
                   ^
          
          <stdin>:1:1: note: scanning from here
          
          (lldb) command source -s 0 'E:/_work/36/s/llvm/tools/lldb/lit/lit-lldb-init'
          
          ^
          
          <stdin>:9:1: note: possible intended match here
          
          use-color (boolean) = true
          
          ^

You can probably revert it. BTW, is the bot public? I would like to
understand whether I broke something after I commit.

Davide

Bot sure how to fix tests since this is somehow hard to problematic.

You can probably revert it. BTW, is the bot public? I would like to
understand whether I broke something after I commit.

Davide

It's not public (yet) because there's one more failure in the LLVM unit tests that needs to be fixed before it can be green and I don't want to make it public and just add noise because of a known failure.

davide added a comment.Sep 6 2018, 2:13 PM

Cool. The argument about reverting still stands though (you might
consider following up with logs et simila to make debugging easier).

Bot sure how to fix tests since this is somehow hard to problematic.

I think the fix for this is simply to not make the call to SetUseColor here. Is there a reason for the call? It looks like the check for "dumb" terminal and "GetIsTerminalWithColors" might have to come after setting the ANSI escape codes if setting the codes impacts those two cases.

Bot sure how to fix tests since this is somehow hard to problematic.

I think the fix for this is simply to not make the call to SetUseColor here. Is there a reason for the call? It looks like the check for "dumb" terminal and "GetIsTerminalWithColors" might have to come after setting the ANSI escape codes if setting the codes impacts those two cases.

We may disable "use color" setting thanks to GetIsTerminalWithColors but then we realize that we can "enable it" via SetConsoleMode for newer Win 10, so we need to call it I think. The reverse order does not help us either.. I think.

Bot sure how to fix tests since this is somehow hard to problematic.

I think the fix for this is simply to not make the call to SetUseColor here. Is there a reason for the call? It looks like the check for "dumb" terminal and "GetIsTerminalWithColors" might have to come after setting the ANSI escape codes if setting the codes impacts those two cases.

We may disable "use color" setting thanks to GetIsTerminalWithColors but then we realize that we can "enable it" via SetConsoleMode for newer Win 10, so we need to call it I think. The reverse order does not help us either.. I think.

Sounds like GetIsTerminalWithColors might have to be updated to include the new logic as well. Would you rather commit a fix or revert the change and then commit a new one?

Bot sure how to fix tests since this is somehow hard to problematic.

I think the fix for this is simply to not make the call to SetUseColor here. Is there a reason for the call? It looks like the check for "dumb" terminal and "GetIsTerminalWithColors" might have to come after setting the ANSI escape codes if setting the codes impacts those two cases.

We may disable "use color" setting thanks to GetIsTerminalWithColors but then we realize that we can "enable it" via SetConsoleMode for newer Win 10, so we need to call it I think. The reverse order does not help us either.. I think.

Sounds like GetIsTerminalWithColors might have to be updated to include the new logic as well. Would you rather commit a fix or revert the change and then commit a new one?

I can try to fix tomorrow with a new patch :)