Infinite Loop: Theory and Practice Revisited.

I’ve unlocked a new achievement as a blogger, or a new milestone as a life-form. As a dinosaur telling the same old stories over and over again.

I started drafting a blog post, as I always do since a while: I do it in my mind only, twist and turn in for days or weeks – until I am ready to write it down in one go. Today I wanted to release a post called On Learning (2) or the like. I knew I had written an early post with a similar title, so I expected this to be a loosely related update. But then I checked the old On Learning post: I found not only the same general ideas but the same autobiographical anecdotes I wanted to use now – even  in the same order.

2014 I had looked back on being both a teacher and a student for the greater part of my professional life, and the patterns were always the same – be the field physics, engineering, or IT security. I had written this post after a major update of our software for analyzing measurement data. This update had required me to acquire new skills, which was a delightful learning experience. I tried to reconcile very different learning modes: ‘Book learning’ about so-called theory, including learning for the joy of learning, and solving problems hands-on based on the minimum knowledge absolutely required.

It seems I like to talk about the The Joys of Theory a lot – I have meta-posted about theoretical physics in general, more than oncegeneral relativity as an example, and about computer science. I searched for posts about hands-on learning now – there aren’t any. But every post about my own research and work chronicles this hands-on learning in a non-meta explicit way. These are the posts listed on the heat pump / engineering page,  the IT security / control page, and some of the physics posts about the calculations I used in my own simulations.

Now that I am wallowing in nostalgia and scrolling through my old posts I feel there is one possibly new insight: Whenever I used knowledge to achieve a result that I really needed to get some job done, I think about this knowledge as emerging from hands-on tinkering and from self-study. I once read that many seasoned software developers also said that in a survey about their background: They checked self-taught despite having university degrees or professional training.

This holds for the things I had learned theoretically – be it in a class room or via my morning routine of reading textbooks. I learned about differential equations, thermodynamics, numerical methods, heat pumps, and about object-oriented software development. Yet when I actually have to do all that, it is always like re-learning it again in a more pragmatic way, even if the ‘class’ was very ‘applied’, not much time had passed since learning only, and I had taken exams. This is even true for the archetype all self-studied disciplines – hacking. Doing it – like here  – white-hat-style 😉 – is always a self-learning exercise, and reading about pentesting and security happens in an alternate universe.

The difference between these learning modes is maybe not only in ‘the applied’ versus ‘the theoretical’, but it is your personal stake in the outcome that matters – Skin In The Game. A project done by a group of students for the final purpose of passing a grade is not equivalent to running this project for your client or for yourself. The point is not if the student project is done for a real-life client, or the task as such makes sense in the real world. The difference is whether it feels like an exercise in an gamified system, or whether the result will matter financially / ‘existentially’ as you might try to empress your future client or employer or use the project results to build your own business. The major difference is in weighing risks and rewards, efforts and long-term consequences. Even ‘applied hacking’ in Capture-the-Flag-like contests is different from real-life pentesting. It makes all the difference if you just loose ‘points’ and miss the ‘flag’, or if you inadvertently take down a production system and violate your contract.

So I wonder if the Joy of Theoretical Learning is to some extent due to its risk-free nature. As long as you just learn about all those super interesting things just because you want to know – it is innocent play. Only if you finally touch something in the real world and touching things has hard consequences – only then you know if you are truly ‘interested enough’.

Sorry, but I told you I will post stream-of-consciousness-style now and then 🙂

I think it is OK to re-use the image of my beloved pre-1900 physics book I used in the 2014 post:

Where Are the Files? [Winsol – UVR16x2]

Recently somebody has asked me where the log files are stored. This question is more interesting then it seems.

We are using the freely programmable controller UVR16x2 (and its predecessor) UVR1611) …

.. and their Control and Monitoring Interface – CMI:The CMI is a data logger and runs a web server. It logs data from the controllers (and other devices) via CAN bus – I have demonstrated this in a contrived example recently, and described the whole setup in this older post.

IT / smart home nerds asked me why there are two ‘boxes’ as other solutions only use a ‘single box’ as both controller and logger. I believe separating these functions is safer and more secure: A logger / web server should not be vital to run the controller, and any issues with these auxiliary components must impact the controller’s core functions.

Log files are stored on the CMI in a proprietary format, and they can retrieved via HTTP using the software Winsol. Winsol lets you visualize data for 1 or more days, zoom in, define views etc. – and data can be exported as CSV files. This is the tool we use for reverse engineering hydraulics and control logic (German blog post about remote hydraulics surgery):

In the latest versions of Winsol, log files are per default stored in the user’s profile on Windows:
C:\Users\[Username]\Documents\Technische Alternative\Winsol

I had never paid much attention to this; I had always changed that path in the configuration to make backup and automation easier. The current question about the log files’ location was actually about how I managed to make different users work with the same log files.

The answer might not be obvious because of the historical location of the log files:

Until some version of Winsol in use in 2017 log files were by stored in the Program Files folder, or at least Winsol tried to use that folder. Windows does not allow this anymore for security reasons.

If Winsol is upgraded from an older version, settings might be preserved. I did my tests  with Winsol 2.07 upgraded from an earlier version. I am a bit vague about versions as I did not test different upgrade paths in detail My point is users of control system’s software tend to be conservative when it comes to changing a running system – an older ‘logging PC’ with an older or upgraded version of Winsol is not an unlikely setup.

I started debugging on Windows 10 with the new security feature Controlled Folder Access enabled. CFA, of course, did not know Winsol, considered it an unfriendly app … to be white-listed.

Then I was curious about the default log file folders, and I saw this:

In the Winsol file picker dialogue (to the right) the log folders seem to be in the Program Files folder:
C:\Program Files\Technische Alternative\Winsol\LogX
But in Windows Explorer (to the left) there are no log files at that location.

What does Microsoft Sysinternals Process Monitor say?

There is a Reparse Point, and the file access is redirected to the folder:
C:\Users\[User]\AppData\Local\VirtualStore\Program Files\Technische Alternative\Winsol
Selecting this folder directly in Windows Explorer shows the missing files:

This location can be re-configured in Winsol to allow different users to access the same files (Disclaimer: Perhaps unsupported by the vendor…)

And there are also some truly user-specific configuration files in the user’s profile, in
C:\Users\[User]\AppData\Roaming\Technische Alternative\Winsol

Winsol.xml is e.g. for storing the list of ‘clients’ (logging profiles) that are included in automated processing of log files, and cookie.txt is the logon cookie for access to the online logging portal provided by Technische Alternative. If you absolutely want to switch Windows users *and* switch logging profiles often *and* sync those you have to tinker with Winsol.xml, e.g. by editing it using a script (Disclaimer again: Unlikely to be a supported way of doing things ;-))

As a summary, I describe the steps required to migrate Winsol’s configuration to a new PC and prepare it for usage by different users.

  • Install the latest version of Winsol on the target PC.
  • If you use Controlled Folder Access on Windows 10: Exempt Winsol as a friendly app.
  • Copy the contents of C:\Users\[User]\AppData\Roaming\Technische Alternative\Winsol from the user’s profile on the old machine to the new machine (user-specific config files).
  • If the log file folder shows up at a different path on the two machines – for example when using the same folder via a network share – edit the path in Winsol.xml or configure it in General Settings in Winsol.
  • Copy your existing log data to this new path. LogX contains the main log files, Infosol contain clients’ data. The logging configuration for each client, e.g. the IP address or portal name of the logger, is included in the setup.xml file in the root of each client’s folder.

Note: If you skip some Winsol versions on migrating/upgrading the structure of files might have changed – be careful! Last time that happened by the end of 2016 and Data Kraken had to re-configure some tentacles.

Cloudy Troubleshooting

Actors:

  • Cloud: Service provider delivering an application over the internet.
  • Client: Business using the Cloud
  • Telco: Service provider operating part of the network infrastructure connecting them.
  • elkement: Somebody who always ends up playing intermediary.

~

Client: Cloud logs us off ever so often! We can’t work like this!

elkement: Cloud, what timeouts do you use? Client was only idle for a short break and is logged off.

Cloud: Must be something about your infrastructure – we set the timeout to 1 hour.

Client: It’s becoming worse – Cloud logs us off every few minutes even we are in the middle of working.

[elkement does a quick test. Yes, it is true.]

elkement: Cloud, what’s going on? Any known issue?

Cloud: No issue in our side. We have thousands of happy clients online. If we’d have issues, our inboxes would be on fire.

[elkement does more tests. Different computers at Client. Different logon users. Different Client offices. Different speeds of internet connections. Computers at elkement office.]

elkement: It is difficult to reproduce. It seems like it works well for some computers or some locations for some time. But Cloud – we did not have any issues of that kind in the last year. This year the troubles started.

Cloud: The timing of our app is sensitive: If network cards in your computers turn on power saving that might appear as a disconnect to us.

[elkement learns what she never wanted to know about various power saving settings. To no avail.]

Cloud: What about your bandwidth?… Well, that’s really slow. If all people in the office are using that connection we can totally understand why our app sees your users disappearing.

[elkement on a warpath: Tracking down each application eating bandwidth. Learning what she never wanted to know about tuning the background apps, tracking down processes.]

elkement: Cloud, I’ve throttled everything. I am the only person using Clients’ computers late at night, and I still encounter these issues.

Cloud: Upgrade the internet connection! Our protocol might choke on a hardly noticeable outage.

[elkement has to agree. The late-night tests were done over a remote connections; so measurement may impact results, as in quantum physics.]

Client: Telco, we buy more internet!

[Telco installs more internet, elkement measures speed. Yeah, fast!]

Client: Nothing has changed, Clouds still kicks us out every few minutes.

elkement: Cloud, I need to badger you again….

Cloud: Check the power saving settings of your firewalls, switches, routers. Again, you are the only one reporting such problems.

[The router is a blackbox operated by Telco]

elkement: Telco, does the router use any power saving features? Could you turn that off?

Telco: No we don’t use any power saving at all.

[elkement dreams up conspiracy theories: Sometimes performance seems to degrade after business hours. Cloud running backup jobs? Telco’s lines clogged by private users streaming movies? But sometimes it’s working well even in the location with the crappiest internet connection.]

elkement: Telco, we see this weird issue. It’s either Cloud, Client’s infrastructure, or anything in between, e.g. you. Any known issues?

Telco: No, but [proposal of test that would be difficult to do]. Or send us a Wireshark trace.

elkement: … which is what I planned to do anyway…

[elkement on a warpath 2: Sniffing, tracing every process. Turning off all background stuff. Looking at every packet in the trace. Getting to the level where there are no other packets in between the stream of messages between Client’s computers and Cloud’s servers.]

elkement: Cloud, I tracked it down. This is not a timeout. Look at the trace: Server and client communicating nicely, textbook three-way handshake, server says FIN! And no other packet in the way!

Cloud: Try to connect to a specific server of us.

[elkement: Conspiracy theory about load balancers]

elkement: No – erratic as ever. Sometimes we are logged off, sometimes it works with crappy internet. Note that Client could work during vacation last summer with supper shaky wireless connections.

[Lots of small changes and tests by elkement and Cloud. No solution yet, but the collaboration is seamless. No politics and finger-pointing who to blame – just work. The thing that keeps you happy as a netadmin / sysadmin in stressful times.]

elkement: Client, there is another interface which has less features. I am going to test it…

[elkement: Conspiracy theory about protocols. More night-time testing].

elkement: Client, Other Interface has the same problems.

[elkement on a warpath 3: Testing again with all possible combinations of computers, clients, locations, internet connections. Suddenly a pattern emerges…]

elkement: I see something!! Cloud, I believe it’s user-dependent. Users X and Y are logged off all the time while A and B aren’t.

[elkement scratches head: Why was this so difficult to see? Tests were not that unambiguous until now!]

Cloud: We’ve created a replacement user – please test.

elkement: Yes – New User works reliably all the time! 🙂

Client: It works –  we are not thrown off in the middle of work anymore!

Cloud: Seems that something about the user on our servers is broken – never happened before…

elkement: But wait 😦 it’s not totally OK: Now logged off after 15 minutes of inactivity? But never mind – at least not as bad as logged off every 2 minutes in the middle of some work.

Cloud: Yeah, that could happen – an issue with Add-On Product. But only if your app looks idle to our servers!

elkement: But didn’t you tell us that every timeout ever is no less than 1 hour?

Cloud: No – that 1 hour was another timeout …

elkement: Wow – classic misunderstanding! That’s why it is was so difficult to spot the pattern. So we had two completely different problems, but both looked like unwanted logoffs after a brief period, and at the beginning both weren’t totally reproducible.

[elkement’s theory validated again: If anything qualifies elkement for such stuff at all it was experience in the applied physics lab – tracking down the impact of temperature, pressure and 1000 other parameters on the electrical properties of superconductors… and trying to tell artifacts from reproducible behavior.]

~

Cloudy

Logging Fun with UVR16x2: Photovoltaic Generator – Modbus – CAN Bus

The Data Kraken wants to grow new tentacles.

I am playing with the CMI – Control and Monitoring Interface – the logger / ‘ethernet gateway’ connected to our control units (UVR1611, UVR16x2) via CAN bus. The CMI has become a little Data Kraken itself: Inputs and outputs can be created for CAN bus and Modbus, and data from most CAN devices can also be logged via JSON.

Are these features useful to integrate the datalogger of our photovoltaics inverter – Fronius Symo 4.5-3-M? I am now logging data to an USB stick and feed the CSV files to the SQL Server Data Kraken. The USB logger’s logging interval is 5 minutes whereas Modbus TCP allows for logging every few seconds. The inverter has built-in energy management features, but it can only ‘signal’ via a relay which also requires proper wiring. Modbus TCP, on the other hand, could use the existing WLAN connection of the inverter and the control unit could do something smarter with the sensor reading of the output power.

My motivation is to test if you – as an UVR16x2 user – can re-use a logger you  already have – the CMI – as much as possible, avoiding the need to run another ‘logging server’ all the time (Also my SQL Server is for analysis, not for real-time logging). I know that there are many open source Modbus clients available and that it is easy to write a Python script.

Activate Modbus on the inverter: I prefer floating numbers to integers plus a scaling factor, and I turn off the option to make changes via Modbus:

Modbus settings, Fronius Datalogger, inverter’s local web server. 502 is the TCO standard port. The alternative to floating numbers is integers plus a varying scaling factor (SF), to be found in another register.

Check Fronius documentation of its Modbus registers: The document is currently available here. There are different sets of registers related to the inverter or associated with one string of PV panels:

PDF p.97, Common Inverter Model. For logging AC output power you need:  Address 40092, type of register 3 (read and hold), datatype float32 (‘corresponds’ to two 16bit integer register, thus size 2).

The address to be configured on a Modbus client is smaller than this address by 1 – so 40091 needs to be set to log AC power.

Using these configuration parameters an analog Modbus Input is added at the CMI. The signal is ‘digital’ – but in field-bus-language everything that is not a single bit – 0/1 – seems to be ‘analog’.

Modbus input at the CMI. Input value:  32bits read from the bus interpreted as an integer. Actual value: Integer part of the ‘true value’ = the 32bits interpreted as 32-bit float.

Yes, I checked the network trace 😉 as the byte order dropdown menu confused me: According to the Modbus protocol specification Big Endian is required, not an option.

Factors and data types: Only integer values are understood by CAN devices. Decimal places might be indicated by a scaling factor. The PV power value in Watt has enough significant digits; so the integer part of the float number is fine. But for current in Ampere – typically about 15A maximum – a Factor of 10 would be better. It would not have helped to select int + scaling factor at the inverter: The scaling factor would be stored in a second register, there is a different factor for every parameter, and you cannot configure another ‘scaling factor register’ per input at the CMI. Theoretically you could log the scaling factor separately and re-scale the value in a custom application – but then I would use a separate, custom logger.

In any case, if you screw this up, you see non-sensible numbers of the CAN bus: Slowly evolving positive values – like PV power on a sunny day – are displayed as wild variations of signed integers between -32000 and 32000 😉

Where are the ‘logged’ data? The CMI is first and foremost the data logger for the control units. The CMI does not immediately store the data from Modbus inputs in a  local ‘logging database’. All I have achieved so far is to display the value on the Settings page. The CMI can only log values from the CAN bus or DL bus. So we need an…

… Analog CAN Output at the CMI:

The CMI has the default node number 56 on the CAN bus. Other CAN devices on the bus can query it for this parameter by specifying node 56 and output no 1.

These are the devices on our CAN bus:

CAN bus displayed on the CMI’s website. UVR1611 and UVR16x2 controllers can be managed by clicking their icons – which brings up a web page that resembles the controller’s local display.

The CMI’s Logging page looks tempting – can we simply select the CMI itself as a CAN logging source – CAN 56?

Configuration of the devices the CMI logs data from, via CAN bus. CAN 1 – UVR1611, CAN 2 – UVR16x3, CAN 41 – energy meter CAN-EZ.

Nothing stops you from selecting CAN 56 in this dropdown menu, but it does not end well:

CAN error message displayed at the logger CMI when you try to configure the CMI also as a logging source.

We need a round-trip: Data needs to be sent to a supported device first – one of the controllers on the CAN bus. We need an…

… Analog CAN input / network variable at the UVR16x2:

Configuration of a CAN input at the controller UVR16x2 (via CMI’s web interface to the controller).

The value of AC power is displayed as integer without scaling. Had a factor of 10 been used at the Modbus input it would be ‘corrected’ here, using the Unit called dimensionless,1.

logging-uvr16x2-can-network-input-can-value-display

Values received by the controller UVR16x2 over CAN bus.

Result of all this: UVR16x2 knows PV power and can use it do magic smart things when controlling the heat pump. On the other hand, CMI can log this value – in the same way it logs all other sensor readings.

Log files are retrieved by Winsol, the free logging software for the CMI …

Logged visualized with Winsol. Logfiles are downloaded from the CMI on the internal LAN or via Technsche Alternative’s web portal. PV power (PV.Leistung.Watt) is displayed together with global radiation on a vertical plane (GBS, at the solar/air collector for the theat pump), ambient temperature (red), temperature of solar/air collector (orange)

… or logging is configured at the web portal cmi.ta.co.at …

Configuration of logging at cmi.ta.co.at: Supported loggers are UVR1611 and UVR16x2. Values to be logged are selected from all direct inputs / outputs / functions and from CAN network inputs and outputs.

… and data can be viewed online:

Data visualized at cmi.ta.co.at. Data logged via CAN are sent from the CMI to the web portal.

Using this kind of logging for all values the inverter provides would be costly: It’s not just a column you add to a log file, but you occupy one of the limited inputs and outputs at the CMI and the controller. If you really need to know the voltage between phase 1 and 2 or apparent power you better stick with the USB file or use a separate Modbus logger like a Rasbperry Pi. This project is great and documented very well – data acqusition from a Symo inverter using Python plus a web front end.

Sending Modbus data back and forth from the CMI to UVR controllers is only worth the efforts if you need them for control, not for ‘nice-to-have’ logging.

Let Your Hyperlinks Live Forever!

It is the the duty of a Webmaster to allocate URIs which you will be able to stand by in 2 years, in 20 years, in 200 years. This needs thought, and organization, and commitment. (https://www.w3.org/Provider/Style/URI)

Joel Spolsky did it:

 I’m bending over backwards not to create “linkrot” — all old links to Joel on Software stories have been replaced with redirects, so they should still work. (November 2001)

More than once:

I owe a huge debt of gratitude to [several people] for weeks of hard work on creating this almost perfect port of 16 years of cruft, preserving over 1000 links with redirects… (December 2016).

Most of the outgoing URLs linked by Joel of Software have rotted, with some notable exceptions: Jakob Nielsen’s URLs do still work, so they live what he preached – in 1998:

… linkrot contributes to dissolving the very fabric of the Web: there is a looming danger that the Web will stop being an interconnected universal hypertext and turn into a set of isolated info-islands. Anything that reduces the prevalence and usefulness of cross-site linking is a direct attack on the founding principle of the Web.

No excuses if you are not Spolsky- or Nielsen-famous – I did it too, several times. In 2015 I rewrote the application for my websites from scratch and redirected every single .asp URL to a new friendly URL at a new subdomain.

I am obsessed with keeping old URLs working. I don’t like it if websites are migrated to a new content management system, changing all the URLs.

I checked all that again when migrating to HTTPS last year.

So I am a typical nitpicking dinosaur, waxing nostalgic about the time when web pages were still pages, and when Hyperlinks Subverted Hierarchy. When browsers were not yet running an OS written in Javascript and hogging 70% of your CPU for ad-tracking or crypto-mining.

The dinosaur is grumpy when it has to fix outgoing URLs on this blog. So. Many. Times. Like every second time I test a URL that shows up in my WordPress statistics as clicked, it 404s. Then I try to find equivalent content on the same site if the domain does still exist – and had not been orphaned and hijacked by malvertizers. If I am not successful I link to a version of this content on web.archive.org, track down the content owner’s new site, or find similar content elsewhere.

My heart breaks when I see that it’s specifically the interesting, unusual content that users want to follow from here – like hard-to-find historical information on how to build a heat pump from clay tablets and straw. My heart breaks even more when the technical content on the target site gets dumbed down more and more with every URL breaking website overhaul. But OK – you now have this terrific header image with a happy-people-at-work stock photo that covers all my desktop so that I have to scroll for anything, and the dumbed down content is shown in boxes that pop up and whirl – totally responsive, though clunky on a desktop computer.

And, yes: I totally know that site owners don’t own me anything. Just because you hosted that rare and interesting content for the last 10 years does not mean you have to do that forever.

But you marketing ninjas and website wranglers neglected an important point: We live in the age of silly gamification that makes 1990s link building pale: I like yours and you like mine. Buy Followers. Every time I read a puffed up Case Study for a project I was familiar with as an insider, I was laughing for minutes and then checked if it was not satire.

In this era of fake word-of-mouth marketing you get incoming links. People say something thoughtful, maybe even nice about you just because they found your content interesting and worth linking not because you play silly games of reciprocating. The most valuable links are set by people you don’t know and who did not anticipate you will ever notice their link. As Nassim Taleb says: Virtue is what you do when nobody is looking.

I would go to great lengths not to break links to my sites in those obscure DIY forums whose posts are hardly indexed by search engines. At least I would make a half-hearted attempt at redirecting to a custom 404 page that explains where you might the moved content. Or just keep the domain name intact. Which of course means not to register a catchy domain name for every product in the first place. Which I consider bad practice anyway – training users to fall for phishing, by getting them used to jumping from one weird but legit domain to another.

And, no, I don’t blame you personally, poor stressed out web admin who had to get the new site up and running before April 1st, because suits in your company said the world would come to an end otherwise. I just think that our internet culture that embraces natural linkrot so easily is as broken as the links.

I tag this as Rant, but it is a Plea: I beg you, I implore you to invest just a tiny part of the time, budget and efforts you allocated to Making the Experience of Your Website Better to making some attempt at keeping your URLs intact. They are actually valuable for others – something you should be proud of.

Reverse Engineering Fun

Recently I read a lot about reverse engineering –  in relation to malware research. I for one simply wanted to get ancient and hardly documented HVAC engineering software to work.

The software in question should have shown a photo of the front panel of a device – knobs and displays – augmented with current system’s data, and you could have played with settings to ‘simulate’ the control unit’s behavior.

I tested it on several machines, to rule out some typical issues quickly: Will in run on Windows 7? Will it run on a 32bit system? Do I need to run it was Administrator? None of that helped. I actually saw the application’s user interface coming up once, on the Win 7 32bit test machine I had not started in a while. But I could not reproduce the correct start-up, and in all other attempts on all other machines I just encountered an error message … that used an Asian character set.

I poked around the files and folders the application uses. There were some .xls and .xml files, and most text was in the foreign character set. The Asian error message was a generic Windows dialogue box: You cannot select the text within it directly, but the whole contents of such error messages can be copied using Ctrl+C. Pasting it into Google Translate it told me:

Failed to read the XY device data file

Checking the files again, there was an on xydevice.xls file, and I wondered if the relative path from exe to xls did not work, or if it was an issue with permissions. The latter was hard to believe, given that I simply copied the whole bunch of files, my user having the same (full) permissions on all of them.

I started Microsoft Sysinternals Process Monitor to check if the application was groping in vain for the file. It found the file just fine in the right location:

Immediately before accessing the file, the application looped through registry entries for Microsoft JET database drivers for Office files – the last one it probed was msexcl40.dll – a  database driver for accessing Excel files.

There is no obvious error in this dump: The xls file was closed before the Windows error popup was brought up; so the application had handled the error somehow.

I had been tinkering a lot myself with database drivers for Excel spreadsheets, Access databases, and even text files – so that looked like a familiar engineering software hack to me 🙂 On start-up the application created a bunch of XML files – I saw them once, right after I saw the GUI once in that non-reproducible test. As far as I could decipher the content in the foreign language, the entries were taken from that problematic xls file which contained a formatted table. It seemed that the application was using a sheet in the xls file as a database table.

What went wrong? I started Windows debugger WinDbg (part of the Debugging tools for Windows). I tried to go the next unhandled or handled exception, and I saw again that it stumbled over msexec40.dll:

But here was finally a complete and googleable error message in nerd speak:

Unexpected error from external database driver (1).

This sounded generic and I was not very optimistic. But this recent Microsoft article was one of the few mentioning the specific error message – an overview of operating system updates and fixes, dated October 2017. It describes exactly the observed issue with using the JET database driver to access an xls file:

Finally my curious observation of the non-reproducible single successful test made sense: When I started the exe on the Win 7 test client, this computer had been started the first time after ~3 months; it was old and slow, and it was just processing Windows Updates – so at the first run the software had worked because the deadly Windows Update had not been applied yet.

Also the ‘2007 timeframe’ mentioned was consistent – as all the application’s executable files were nearly 10 years old. The recommended strategy is to use a more modern version of the database driver, but Microsoft also states they will fix it again in a future version.

So I did not get the software to to run, as I obviously cannot fix somebody else’s compiled code – but I could provide the exact information needed by the developer to repair it.

But the key message in this post is that it was simply a lot of fun to track this down 🙂

Simulating Life-Forms (2): Cooling Energy

I found this comprehensive research report:
Energy Use in the Australian Residential Sector 1986–2020 (June 2008)
(several PDFs for download, click the link Energy Use… to display them)

There are many interesting results – and the level of detail is impressive: The authors modelled the energy used per appliance type, by e.g. factoring in how building types change slowly over time or by modelling the development of TV sets and their usage. Occupancy factors for buildings are determined from assumptions about typical usage profiles called Stay At Home, At Work or Night Owl.

I zoom in on simulating and predicting usage of air conditioning and thus cooling energy:

They went to great lengths to simulate the behavior of home owners to model operations of air conditioning and thus total cooling energy for a season, for a state or the whole country.

The authors investigated the official simulation software used for rating buildings (from …part2.pdf):

In the AccuRate software, once cooling is invoked the
program continues to assume that the occupant is willing to
tolerate less than optimal comfort conditions and will therefore terminate cooling if in the absence of such cooling the internal temperature would not rise above the summer neutral temperature noted in Table 57, + 2.5oC plus allowances for humidity and air movement as applicable. While this may be appropriate for rating purposes, it is considered to be an unlikely form of behaviour to be adopted by householders in the field and as such this assumption is likely to underestimate the potential space cooling demand. This theory is supported by the survey work undertaken by McGreggor in South Australia.

This confirms what I am saying all the time: The more modern a building is, or generally nowadays given ‘modern’ home owners’ requirements, the more important would it be to actually simulate humans’ behavior, on top of the physics and the control logic.

The research study also points out e.g. that AC usage has been on the rise, because units got affordable, modern houses are built with less focus on shading, and home owners demand higher standards of comfort. Ducted cooling systems that cover the cooling load of the whole house are being implemented, and they replace systems for cooling single zones only. Those ducted systems have a rated output cooling power greater than 10kW – so the authors (and it seems Australian governmental decision makers) are worried about the impact on the stability of the power grid on hot days [*].

Once AC had been turned on for the first time in the hot season, home owners don’t switch it off again when the theoretical ‘neutral’ summer temperature would be reached again, but they keep it on and try to maintain a lower temperature (22-23°C) that is about constant irrespective of temperature outside. So small differences in actual behavior cause huge error bars in total cooling energy for a season:

The impact of this resetting of the cooling thermostat operation was found to be significant. A comparison was undertaken between cooling loads determined using the AccuRate default thermostat settings and the modified settings as described above. A single-storey brick veneer detached dwelling with concrete slab on ground floor and ceiling insulation was used for the comparison. The comparison was undertaken in both the Adelaide and the Darwin climate zones. In Adelaide the modified settings produced an increased annual cooling load 64% higher than that using the AccuRate default settings.

The report also confirms my anecdotal evidence: In winter (colder regions) people heat rooms to higher temperatures than ‘expected’; in summer (warmer regions) people want to cool to a lower temperature:

This is perhaps not surprising, de Dear notes that: “preferred temperature for a particular building did not necessarily coincide with thermal neutrality, and this semantic discrepancy was most evident in HVAC buildings where preference was depressed below neutrality in warm climates and elevated above neutrality in cold climates (ie people preferred to feel cooler than neutral in warm climates, and warmer than neutral in cold climates)” (Richard de Dear et al 1997, P xi).

I noticed that the same people who (over-)heat their rooms to 24°C in winter might want to cool to 20°C in summer. In middle Europe AC in private homes has been uncommon, but I believe it is on the rise, too, also because home owners got accustomed to a certain level of cooling when they work in typical office buildings.

My conclusion is (yet again) that you cannot reliably ‘predict’ cooling energy. It’s already hard to do so for heating energy for low energy houses, but nearly impossible for cooling energy. All you can do – from a practical / system’s design perspective – is to make sure that there is an ‘infinite’ source of cooling energy available.

_________________________________

[*] Edit: And it actually happenend in February 2017.