SHA3 Hashes (on Windows) - Where Art Thou?

No sooner had posted on doing file and string hashes in PowerShell, when I (again) got asked by Jim - "What about SHA3? Shouldn't we be using Quantum Safe algorithms if we have them?"

Looking around, support for SHA3 is pretty sparse no matter what the OS. For Windows there's a decent solution in bouncycastle (https://www.bouncycastle.org/), but the install is likely more than folks want to tackle, especially if it gets rolled into PowerShell at some future date. Similarly, the SCCM ConfigurationManager module does implement them in some fashion, but that's kind of a dead-end for most of us too.

In a pinch, hashify.net has a public API that supports just about any hashing algorithm you'd care to mention:

curl --location --request GET "api.hashify.net/hash/sha3-512/hex?value=CQCQCQ"
{"Digest":"bcc7a070db5dd926bfbef21c6c5e8081402a79e45f96c4cd7fedc405e1a7fcb6b047cff266235f19f0d1219d2f0fd9299b93cd28d69517d7aefec8cf0c9ffdcc","DigestEnc":"hex","Type":"SHA3-512","Key":""}

The problem with that is - if the information you are hashing (presumably to verify against either now or later) is important or sensitive enough to warrant using one of the fancy SHA3 algorithms, it's likely not data that you want sent to a public website in the clear.

I eventually decided to use the functionality in OpenSSL, with the rationale that anyone who needs this function will likely have OpenSSL already installed locally, at most we'd be asking them to upgrade - you'll need OpenSSL 1.1.1 or better for SHA3-xxx hash support. The syntax is:

echo "some string" | openssl dgst -hashalgorithm

or

type "somefilespec" | openssl dgst -hashalgorithm

where "hashalgorithm" is any of:

blake2b512                blake2s256                md4
md5                       md5-sha1                  mdc2
ripemd                    ripemd160                 rmd160
sha1                      sha224                    sha256
sha3-224                  sha3-256                  sha3-384
sha3-512                  sha384                    sha512
sha512-224                sha512-256                shake128
shake256                  sm3                       ssl3-md5
ssl3-sha1                 whirlpool

So for implementing this in PowerShell, it's as easy as creating the command in a string, then calling it with "Invoke-Expression" (shortened to "iex" in the examples below).

So for now, until Microsoft rolls better support for SHA3 family of hashing algorithms, my quick-and-dirty implementation for the newer, shinier hash algorithms is below. Note that if OpenSSL isn't in the path, I've got a variable pointed to the path to the binary (update this variable to match your install). In any "real" code you would put this in a config file of course (because we all need more config files in our life right?)

$OpenSSLPath = "C:\openssl-1.1.1h\bin\"

function Get-StringHash-OpenSSL ( [String] $InputString, $HashAlgo )

{

$QT = "`""

$cmd = "echo " + $QT + $InputString + $QT + " | " + $OpenSSLPath + "openssl.exe dgst -" + $HashAlgo

$callcmd = iex $cmd

$callcmd.split(" ")[1]

}

$hash = get-stringhash-openssl "CQ CQ CQ" "SHA3-256"

$hash

5b960a5284843bb23af5e249c8692bd6d831645cc5070d501b4cef3e94d6983e

$OpenSSLPath = "C:\openssl-1.1.1h\bin\"

function Get-FileHash-OpenSSL ( [String] $InputFileSpec, $HashAlgo )

{

$QT = "`""

$cmd = "type " + $QT + $InputFileSpec + $QT + " | " + $OpenSSLPath + "openssl.exe dgst -" + $HashAlgo

$callcmd = iex $cmd

$callcmd.split(" ")[1]

}

$hash = get-FileHash-OpenSSL "c:\windows\system32\cmd.exe" "Sha3-512"

$hash

0cacd8c85b44eed57101fee1431434278319dc441aee26354f811b483a30ff7861ecc88f4c90791e941e49dcb124a975d9eb301
e5d715a4e80ee918ea2f5f844

If you've worked out a way to get these algorithms into PowerShell without IEX or any 3rd party installs, please share using our comment form.

(And yes, I did riff on the title of Mark Baggett's presentation next week - Tech Tuesday Workshop - O Hacker, Where Art Thou?: A Hands-On Python Workshop for Geolocating Attackers https://www.sans.org/webcasts/hacker-art-thou-hands-on-python-workshop-geolocating-attackers-115340 )

===============
Rob VandenBrink
www.coherentsecurity.com

Rob VandenBrink

571 Posts
ISC Handler

May 15th 2020

FWIW, it looks to me like Get-StringHash-OpenSSL appends CRLF to the input string. I believe I verified this for MD5 hashes since I have an independent MD5 hash program. I don't know about the other hashes. I tested my independent MD5 hash program against some examples in the MD5 Wikipedia article; they matched. When I append CRLF to the input string in the independent MD5 hash program, I get the results that Get-StringHash-OpenSSL gives.

I have spent way too much time on this trying to figure something out in PowerShell. The intent of the code in the above function is to pass the string as STDIN to openssl (which seems to only accept file input). In DOS, the trivial chop.pl Perl program, "while ( <> ) { chop(); print; }", removes the trailing CRLF from each input line. I am a Powershell neophyte. When I put this program in the PowerShell pipeline, the CRLF magically reappears. Is there a way to send the string (sans CRLF) directly to openssl without creating an intermediate file?

robv

23 Posts

A few answers ....

There isn't actually a CRLF at the end of $hash, you can verify this with :
foreach($c in $hash.tochararray()) { write-host ([byte][char]$c)}
(you'll only see the ascii values expected)

Or even simpler you can check with:
> $hash.Length
64

> $hash[63]
e
(ie - the last character is an "e")

However, write-host does append a newline return by default, so you'll see it in the output - there is in fact a newline character to kill here. You can remove it as below:
write-host -NoNewline $hash

I'll correct the code in my github (it's not there yet, but should be soon)

ps - if you needed to remove the trailing CR or LF chars from a string that has them, there are more than a few ways to do it. You can use "replace" or "split/join" (in multiple different ways each) - I tend to use replace most often. If anyone else has other methods they're welcome to append to this thread :-)

Rob VandenBrink

571 Posts
ISC Handler

Thank you for taking the time to reply. I'm a little more confused now, however. The variable $hash you refer to in your answer -- what is that? Is it the output of the Get-StringHash-OpenSSL call? What I referred to in my first reply was inside the function. $cmd is set to "echo ...", and in my testing the effect is to append a CRLF to the string that openssl sees. Giving the function "abc" does not compute the hash of "abc"; it computes the hash of "abcCRLF" (at least for MD5). Am I wrong about this? This is why I was trying to figure out how to pipe a string (to STDIN) without a CRLF at the end (so as, as in your original intent, to avoid creating an intermediate file). With everything I tried I ended up with a CRLF messing things up. Am I missing something obvious?

P.S. As a PowerShell neophyte, I am learning some weird things. For example, I can put two "-" characters in the names of functions as in your original diary, but if I put them in a so-called module and import the module, PowerShell squawks. (I have to remove the second "-".) I understand the "why the squawk"; what I don't understand is why it allows the second "-" in a script. Another weird thing is this: once you import a module, if you update the source and then import it again, it does not "take." In my experience I had to remove-module and then import-module again. It's as if PowerShell caches the file and doesn't bother to check if it's been updated. Maybe there's something else going on here, but it confused the heck out of me until I found the "remove-module workaround."

robv

23 Posts

FWIW, here is a kludgey workaround for the problem I claim exists. In the Get-StringHash-OpenSSL function, change the "$cmd =" line to the following.

$cmd = "cmd /c " + $QT + "echo " + $InputString + "|chop|" + $OpenSSLPath + "openssl.exe dgst -" + $HashAlgo + $QT

This sort of works because DOS pipes do what you want and expect, but PS pipes (or something) don't. Somehow a CRLF always gets inserted in the PS-controlled pipeline resulting in openssl seeing a string with CRLF appended. I can't find a PS way around this, so used DOS. But this has its own pitfalls I won't really go into here (for example, notice there's no trailing space after $InputString).

Finally, here is the guts of a simple Perl chop program that works in DOS. I wrote it this way to help with debug and to see all characters, etc. (Without binmode, Perl would change the CRLF to a LF, i.e., a "\n".)

$CR = chr(13);
$LF = chr(10);

binmode STDIN;

while ( $u = <STDIN> ) {
if ( $verbose ) {
print STDERR ++$ln , ':' , $u;
print STDERR length($u) , ':(' , join( ',' , map( ord($_) , split( '' , $u ) ) ) , '):' , $u;
}
$u =~ s/[$CR$LF]+$//o;
print STDOUT $u;
}

P.S. I started fooling around with the "file" version of the function, and it has the same perceived problem as the "string" version, namely, a CRLF is appended to the file contents. Since "openssl.exe dgst" allows a file parameter, changing the $cmd variable as follows is a fix. I must point out that either I am way off base here and missing something important (anyone?) or there is a real effort to use ECHO and TYPE and pipes in PS in these functions. Why, I don't know. Both original functions are broken, right? I am all ears, as they say. I expect this to be my last follow-up on the subject, although some corroboration would be nice. (BTW, being a neophyte PS programmer, I thought this diary would be a good way to study PS. That's how I got interested in this.)

$cmd = $OpenSSLPath + "openssl.exe dgst -" + $HashAlgo + " " + $QT + $InputFileSpec + $QT

robv

23 Posts