Escaping with cmd on Windows

[starfishpatkhoo]

Is there something wonky with escaping with cmd shell on Windows? Like escaping ", or \ ? Feels kind of off to me?

Version: 1.56

C:\> echo My test is "awesome"
My test is "awesome"

C:\>  echo My test is \"awesome\"
My test is \"awesome\"

test:
  echo My test is "awesome"
  echo 'My test is "awesome"'
  echo My test is ""awesome""
  echo My test is '"'awesome'"'
  echo My test is '""'awesome'""'
  echo My test is \"awesome\"

C:\> just test
[11:43:21] echo My test is "awesome"
My test is \"awesome\"
[11:43:21] echo 'My test is "awesome"'
'My test is \"awesome\"'
[11:43:22] echo My test is ""awesome""
My test is \"\"awesome\"\"
[11:43:22] echo My test is '"'awesome'"'
My test is '\"'awesome'\"'
[11:43:22] echo My test is '""'awesome'""'
My test is '\"\"'awesome'\"\"'
[11:43:22] echo My test is \"awesome\"
My test is \\\"awesome\\\"

[casey]

Thanks for the report!

The issue is that when Rust passes arguments to a command on Windows, they are escaped so that they are parsed correctly by programs that use CommandLineToArgvW. cmd.exe does not use CommandLineToArgvW, so this escaping is incorrect when you're using cmd.exe.

I put #3636 which fixes it by detecting if the command is cmd.exe, and passing arguments using std::os::windows::process::CommandExt, which avoids escaping. However, I'm somewhat hesitant to merge it. I really don't like features which try to guess what to do.

I'm considering that a better alternative might be a setting, which would be more explicit and possibly less surprising, but would mean that using cmd.exe would be broken if you don't use the setting.

[starfishpatkhoo]

The issue is that when Rust passes arguments to a command on Windows, they are escaped so that they are parsed correctly by programs that use CommandLineToArgvW. cmd.exe does not use CommandLineToArgvW, so this escaping is incorrect when you're using cmd.exe.

I put #3636 which fixes it by detecting if the command is cmd.exe, and passing arguments using std::os::windows::process::CommandExt, which avoids escaping. However, I'm somewhat hesitant to merge it. I really don't like features which try to guess what to do.

Yeah, this is a tough one... I admit, cmd.exe is not the best, and putting stuff through cmd.exe /c is probably worse. For all we know, the escaping from rust itself may well be working as expected, just the cmd.exe /c is the one that is screwing it up when executing the /c...

I'm considering that a better alternative might be a setting, which would be more explicit and possibly less surprising, but would mean that using cmd.exe would be broken if you don't use the setting.

One possibility is to use (yet another sigil) that tells just (no matter which shell is being used, on any platform), do not perform any escaping on the entire line. So the author takes full responsibility for that line all the way. And then, add an escape() function (like quote() or encode_uri_component()) that performs the escaping of a given string.

At the very least, this strategy will be usable on any platform, not just for Windows. So if someone had some really complex .. I don't know, something for bash or tcsh.. If they were willing to take the responsibility for escaping as required, they could do it.

I just find this situation very ... Something to think about, note in docs, and maybe investigate further.. 😢

[casey]

May I ask why you're using cmd.exe? Is there some reason you can't use PowerShell, or another shell? Part of whether or not the fix is worth has got to be based on whether or not there are situations where you really can't avoid cmd.exe, and how common they are.

[starfishpatkhoo]

May I ask why you're using cmd.exe? Is there some reason you can't use PowerShell, or another shell? Part of whether or not the fix is worth has got to be based on whether or not there are situations where you really can't avoid cmd.exe, and how common they are.

TL;DR: From my perspective supporting systems, ultimately, it is always about the lowest common denominator.

Availability:

cmd.exe is available everywhere Windows is, and is the default. And the syntax (though idiotic) is ancient and well known. powershell.exe is available (almost) everywhere Windows is, but is pretty sucky. pwsh.exe is much improved, but not assured to be everywhere, and it has to be installed. FYI, I still write sometimes code targeting powershell.exe and not pwsh.exe for this very reason. Sigh.

If I have to install something, I might as well I install brush.exe and microsoft coreutils. And deployment for both of them are just a matter of copying a file into the %PATH%. No pwsh7_setup.msi to install with admin privs.

My dream is one script to rule them all. To be fair, I'm still testing brush.exe but it looks very promising so far. And given the pace of advancements in oxidisation efforts surrounding tooling everywhere, maybe it really will be true for 95+% cases in 2027. ^_^

Logic complexity:

I only use cmd.exe to launch other commands and not for any kind of complex script logic. And this bug is not about logic, just trying to launch a command that happens to require " in the params. But mind you, so far, it works for everything I need. I have only run into this issue once in actual use. And even then, I resolved it elsewhere/other-ways.

FYI: If I really have to do something needing much more complex logic (more than what I can do with a justfile alone) then I won't even bother with powershell (or cmd) and I will go straight to bash and brush.exe + microsoft coreutils. I probably will continue to maintain .sh scripts (and optionally, use just to execute them). This mostly works everywhere - even Github actions and inside docker containers.

echo %RANT% > NUL

I think this issue is valid, and is a concern, but is of a low(er) priority. So it does merit documentation, and kept here in Github if someone else chances upon this. But maybe just keep a look out if there is a better, more well-rounded solution for this in the future, and don't worry too much about it for now... If I chance upon more use cases that really cannot be avoided, I'll bring it up.. ^_^

[casey]

Thanks for the background! It's very helpful for me because I only use Windows for gaming.

Why not use powershell.exe? It's the older version of PowerShell, so not as good as the newer pwsh.exe, but as I understand it, it's installed by default on Windows 7 (2009) and later, and Windows Server 2008 R2 (2009) and later. So you have to go be using a pretty ancient version of Windows for it not to be available.

The reason I'm hemming and hawing about merging it is that the fix is just kind of gross 😅

[starfishpatkhoo]

Thanks for the background! It's very helpful for me because I only use Windows for gaming.

Why not use powershell.exe? It's the older version of PowerShell, so not as good as the newer pwsh.exe, but as I understand it, it's installed by default on Windows 7 (2009) and later, and Windows Server 2008 R2 (2009) and later. So you have to go be using a pretty ancient version of Windows for it not to be available.

Well, the syntax is different:

PS> echo 'This is "awesome"'

C:\> echo This is "awesome"

And ultimately, I have a lot of batch, and a lot less powershell to maintain. And if I have to migrate, then I want to migrate to bash, which is far more universal than even powershell.

But, this is me... ^_^

The reason I'm hemming and hawing about merging it is that the fix is just kind of gross 😅

Well, I think you should only do it if it (escaping control) can be used across any shell on any platform for any string, not solely for cmd as a shell .. It could be gross, but if it could be used by, and is useful for people who are doing things in bash or zsh, then the gross-ness becomes worth while... ^_^

[casey]

A setting might be better, but I'm really not sure it would be generally useful. I think basically the only shell which doesn't use CommandLineToArgvW is cmd.exe.

[casey]

I went ahead and merged #3636, which checks if the shell is cmd.exe, and passes arguments without escaping them if it is. Even though cmd.exe might not be the right choice, people still use it, and Windows users will probably expect that it works. Hopefully this won't cause weird unforeseen problems down the line 😅