Reverse Engineering Fun

Recently I read a lot about reverse engineering –  in relation to malware research. I for one simply wanted to get ancient and hardly documented HVAC engineering software to work.

The software in question should have shown a photo of the front panel of a device – knobs and displays – augmented with current system’s data, and you could have played with settings to ‘simulate’ the control unit’s behavior.

I tested it on several machines, to rule out some typical issues quickly: Will in run on Windows 7? Will it run on a 32bit system? Do I need to run it was Administrator? None of that helped. I actually saw the application’s user interface coming up once, on the Win 7 32bit test machine I had not started in a while. But I could not reproduce the correct start-up, and in all other attempts on all other machines I just encountered an error message … that used an Asian character set.

I poked around the files and folders the application uses. There were some .xls and .xml files, and most text was in the foreign character set. The Asian error message was a generic Windows dialogue box: You cannot select the text within it directly, but the whole contents of such error messages can be copied using Ctrl+C. Pasting it into Google Translate it told me:

Failed to read the XY device data file

Checking the files again, there was an on xydevice.xls file, and I wondered if the relative path from exe to xls did not work, or if it was an issue with permissions. The latter was hard to believe, given that I simply copied the whole bunch of files, my user having the same (full) permissions on all of them.

I started Microsoft Sysinternals Process Monitor to check if the application was groping in vain for the file. It found the file just fine in the right location:

Immediately before accessing the file, the application looped through registry entries for Microsoft JET database drivers for Office files – the last one it probed was msexcl40.dll – a  database driver for accessing Excel files.

There is no obvious error in this dump: The xls file was closed before the Windows error popup was brought up; so the application had handled the error somehow.

I had been tinkering a lot myself with database drivers for Excel spreadsheets, Access databases, and even text files – so that looked like a familiar engineering software hack to me 🙂 On start-up the application created a bunch of XML files – I saw them once, right after I saw the GUI once in that non-reproducible test. As far as I could decipher the content in the foreign language, the entries were taken from that problematic xls file which contained a formatted table. It seemed that the application was using a sheet in the xls file as a database table.

What went wrong? I started Windows debugger WinDbg (part of the Debugging tools for Windows). I tried to go the next unhandled or handled exception, and I saw again that it stumbled over msexec40.dll:

But here was finally a complete and googleable error message in nerd speak:

Unexpected error from external database driver (1).

This sounded generic and I was not very optimistic. But this recent Microsoft article was one of the few mentioning the specific error message – an overview of operating system updates and fixes, dated October 2017. It describes exactly the observed issue with using the JET database driver to access an xls file:

Finally my curious observation of the non-reproducible single successful test made sense: When I started the exe on the Win 7 test client, this computer had been started the first time after ~3 months; it was old and slow, and it was just processing Windows Updates – so at the first run the software had worked because the deadly Windows Update had not been applied yet.

Also the ‘2007 timeframe’ mentioned was consistent – as all the application’s executable files were nearly 10 years old. The recommended strategy is to use a more modern version of the database driver, but Microsoft also states they will fix it again in a future version.

So I did not get the software to to run, as I obviously cannot fix somebody else’s compiled code – but I could provide the exact information needed by the developer to repair it.

But the key message in this post is that it was simply a lot of fun to track this down 🙂

Simulating Life-Forms (2): Cooling Energy

I found this comprehensive research report:
Energy Use in the Australian Residential Sector 1986–2020 (June 2008)
(several PDFs for download, click the link Energy Use… to display them)

There are many interesting results – and the level of detail is impressive: The authors modelled the energy used per appliance type, by e.g. factoring in how building types change slowly over time or by modelling the development of TV sets and their usage. Occupancy factors for buildings are determined from assumptions about typical usage profiles called Stay At Home, At Work or Night Owl.

I zoom in on simulating and predicting usage of air conditioning and thus cooling energy:

They went to great lengths to simulate the behavior of home owners to model operations of air conditioning and thus total cooling energy for a season, for a state or the whole country.

The authors investigated the official simulation software used for rating buildings (from …part2.pdf):

In the AccuRate software, once cooling is invoked the
program continues to assume that the occupant is willing to
tolerate less than optimal comfort conditions and will therefore terminate cooling if in the absence of such cooling the internal temperature would not rise above the summer neutral temperature noted in Table 57, + 2.5oC plus allowances for humidity and air movement as applicable. While this may be appropriate for rating purposes, it is considered to be an unlikely form of behaviour to be adopted by householders in the field and as such this assumption is likely to underestimate the potential space cooling demand. This theory is supported by the survey work undertaken by McGreggor in South Australia.

This confirms what I am saying all the time: The more modern a building is, or generally nowadays given ‘modern’ home owners’ requirements, the more important would it be to actually simulate humans’ behavior, on top of the physics and the control logic.

The research study also points out e.g. that AC usage has been on the rise, because units got affordable, modern houses are built with less focus on shading, and home owners demand higher standards of comfort. Ducted cooling systems that cover the cooling load of the whole house are being implemented, and they replace systems for cooling single zones only. Those ducted systems have a rated output cooling power greater than 10kW – so the authors (and it seems Australian governmental decision makers) are worried about the impact on the stability of the power grid on hot days [*].

Once AC had been turned on for the first time in the hot season, home owners don’t switch it off again when the theoretical ‘neutral’ summer temperature would be reached again, but they keep it on and try to maintain a lower temperature (22-23°C) that is about constant irrespective of temperature outside. So small differences in actual behavior cause huge error bars in total cooling energy for a season:

The impact of this resetting of the cooling thermostat operation was found to be significant. A comparison was undertaken between cooling loads determined using the AccuRate default thermostat settings and the modified settings as described above. A single-storey brick veneer detached dwelling with concrete slab on ground floor and ceiling insulation was used for the comparison. The comparison was undertaken in both the Adelaide and the Darwin climate zones. In Adelaide the modified settings produced an increased annual cooling load 64% higher than that using the AccuRate default settings.

The report also confirms my anecdotal evidence: In winter (colder regions) people heat rooms to higher temperatures than ‘expected’; in summer (warmer regions) people want to cool to a lower temperature:

This is perhaps not surprising, de Dear notes that: “preferred temperature for a particular building did not necessarily coincide with thermal neutrality, and this semantic discrepancy was most evident in HVAC buildings where preference was depressed below neutrality in warm climates and elevated above neutrality in cold climates (ie people preferred to feel cooler than neutral in warm climates, and warmer than neutral in cold climates)” (Richard de Dear et al 1997, P xi).

I noticed that the same people who (over-)heat their rooms to 24°C in winter might want to cool to 20°C in summer. In middle Europe AC in private homes has been uncommon, but I believe it is on the rise, too, also because home owners got accustomed to a certain level of cooling when they work in typical office buildings.

My conclusion is (yet again) that you cannot reliably ‘predict’ cooling energy. It’s already hard to do so for heating energy for low energy houses, but nearly impossible for cooling energy. All you can do – from a practical / system’s design perspective – is to make sure that there is an ‘infinite’ source of cooling energy available.

_________________________________

[*] Edit: And it actually happenend in February 2017.

The Orphaned Internet Domain Risk

I have clicked on company websites of social media acquaintances, and something is not right: Slight errors in formatting, encoding errors for special German characters.

Then I notice that some of the pages contain links to other websites that advertize products in a spammy way. However, the links to the spammy sites are embedded in this alleged company websites in a subtle way: Using the (nearly) correct layout, or  embedding the link in a ‘news article’ that also contains legit product information – content really related to the internet domain I am visiting.

Looking up whois information tells me that these internet domain are not owned by my friends anymore – consistent with what they actually say on the social media profiles. So how come that they ‘have given’ their former domains to spammers? They did not, and they didn’t need to: Spammers simply need to watch out for expired domains, seize them when they are available – and then reconstruct the former legit content from public archives, and interleave it with their spammy messages.

The former content of legitimate sites is often available on the web archive. Here is the timeline of one of the sites I checked:

Clicking on the details shows:

  • Last display of legit content in 2008.
  • In 2012 and 2013 a generic message from the hosting provider was displayed: This site has been registered by one of our clients
  • After that we see mainly 403 Forbidden errors – so the spammers don’t want their site to be archived – but at one time a screen capture of the spammy site had been taken.

The new site shows the name of the former owner at the bottom but an unobtrusive link had been added, indicating the new owner – a US-based marketing and SEO consultancy.

So my take away is: If you ever feel like decluttering your websites and free yourself of your useless digital possessions – and possibly also social media accounts, think twice: As soon as your domain or name is available, somebody might take it, and re-use and exploit your former content and possibly your former reputation for promoting their spammy stuff in a shady way.

This happened a while ago, but I know now it can get much worse: Why only distribute marketing spam if you can distribute malware through channels still considered trusted? In this blog post Malwarebytes raises the question if such practices are illegal or not – it seems that question is not straight-forward to answer.

Visitors do not even have to visit the abandoned domain explicitly to get hacked by malware served. I have seen some reports of abandoned embedded plug-ins turned into malicious zombies. Silly example: If you embed your latest tweets, Twitter goes out-of-business, and its domains are seized by spammers – you Follow Me icon might help to spread malware.

If a legit site runs third-party code, they need to trust the authors of this code. For example, Equifax’ website recently served spyware:

… the problem stemmed from a “third-party vendor that Equifax uses to collect website performance data,” and that “the vendor’s code running on an Equifax Web site was serving malicious content.”

So if you run any plug-ins, embedded widgets or the like – better check out regularly if the originating domain is still run by the expected owner – monitor your vendors often; and don’t run code you do not absolutely need in the first place. Don’t use embedded active badges if a simple link to your profile would do.

Do a painful boring inventory and assessment often – then you will notice how much work it is to manage these ‘partners’ and rather stay away from signing up and registering for too much services.

Update 2017-10-25: And as we speak, we learn about another example – snatching a domain used for a Dell backup software, preinstalled on PCs.

Data for the Heat Pump System: Heating Season 2016-2017

I update the documentation of measurement data [PDF] about twice a year. This post is to provide a quick overview for the past season.

The PDF also contains the technical configuration and sizing data. Based on typical questions from an ‘international audience’ I add a summary here plus some ‘cultural’ context:

Building: The house is a renovated, nearly 100-year old building in Eastern Austria: a typical so-called ‘Streckhof’ – an elongated, former small farmhouse. Some details are mentioned here. Heating energy for space heating of two storeys (185m2) and hot water is about 17.000-20.000kWh per year. The roof / attic had been rebuilt in 2008, and the facade was thermally insulated. However, the major part of the house is without an underground level, so most energy is lost via ground. Heating only the ground floor (75m2) with the heat pump reduces heating energy only by 1/3.

Climate: This is the sunniest region of Austria – the lowlands of the Pannonian Plain bordering Hungary. We have Pannonian ‘continental’ climate with low precipitation. Normally, monthly average temperatures in winter are only slightly below 0°C in January, and weeks of ‘ice days’ in a row are very rare.

Heat energy distribution and storage (in the house): The renovated first floor has floor loops while at the ground floor mainly radiators are used. Wall heating has been installed in one room so far. A buffer tank is used for the heating water as this is a simple ‘on-off’ heat pump always operating at about its rated power. Domestic hot water is heated indirectly using a hygienic storage tank.

Heating system. An off-the-shelf, simple brine-water heat pump uses a combination of an unglazed solar-air collector and an underwater water tank as a heat source. Energy is mainly harvested from rather cold air via convection.

Addressing often asked questions: Off-the-shelf =  Same type of heat pump as used with geothermal systems. Simple: Not-smart, not trying to be the universal energy management system, as the smartness in our own control unit and logic for managing the heat source(s). Brine: A mixture of glycol and water (similar to the fluid used with flat solar thermal collectors) = antifreeze as the temperature of brine is below 0°C in winter. The tank is not a seasonal energy storage but a buffer for days or weeks. In this post hydraulics is described in detail, and typical operating conditions throughout a year. Both tank and collector are needed: The tank provides a buffer of latent energy during ‘ice periods’ and it allows to harvest more energy from air, but the collector actually provides for about 75% of the total ambient energy the heat pump needs in a season.

Tank and collector are rather generously sized in relation to the heating demands: about 25m3 volume of water (total volume +10% freezing reserve) and 24m2 collector area.

The overall history of data documented in the PDF also reflects ongoing changes and some experiments, like heating the first floor with a wood stove, toggling the effective area of the collector used between 50% and 100%, or switching off the collector to simulate a harsher winter.

Data for the past season

Finally we could create a giant ice cube naturally. 14m3 of ice had been created in the coldest January since 30 years. The monthly average temperature was -3,6°C, 3 degrees below the long-term average.

(Re the oscillations of the ice volume are see here and here.)

We heated only the ground floor in this season and needed 16.600 kWh (incl. hot water) – about the same heating energy as in the previous season. On the other hand, we also used only half of the collector – 12m2. The heating water inlet temperatures for radiators was even 37°C in January.

For the first time the monthly performance factor was well below 4. The performance factor is the ratio of output heating energy and input electrical energy for heat pump and brine pump. In middle Europe we measure both energies in kWh 😉 The overall seasonal performance factor was 4,3.

The monthly performance factor is a bit lower again in summer, when only hot water is heated (and thus the heat pump’s COP is lower because of the higher target temperature).

Per day we needed about 100kWh of heating energy in January, while the collector could not harvest that much:

In contrast to the season of the Ice Storage Challenge, also the month before the ‘challenge’ (Dec. 2016) was not too collector-friendly. But when the ice melted again, we saw the usual large energy harvests. Overall, the collector could contribute not the full ‘typical’ 75% of ambient energy this season.

(Definitions, sign conventions explained here.)

But there was one positive record, too. In a hot summer of 2017 we consumed the highest cooling energy so far – about 600kWh. The floor loops are used for passive cooling; the heating buffer tank is used to transfer heat from the floor loops to the cold underground tank. In ‘colder’ summer nights the collector is in turn used to cool the tank, and every time hot tap water is heated up the tank is cooled, too.

Of course the available cooling power is just a small fraction of what an AC system for the theoretical cooling load would provide for. However, this moderate cooling is just what – for me – makes the difference between unbearable and OK on really hot days with more than 35°C peak ambient temperature.

Computers, Science, and History Thereof

I am reading three online resources in parallel – on the history and the basics of computing, computer science, software engineering, and the related culture and ‘philosophy’. An accidental combination I find most enjoyable.

Joel on Software: Joel Spolsky’s blog – a collection of classic essays. What every developer needs to know about Unicode. New terms like Astronaut Architects and Leaky Abstractions. How to start a self-funded software company, how to figure out the price of software, how to write functional specifications. Bringing back memories of my first encounters with Microsoft VBA. He has the best examples – Martian Headsets to explain web standards.

The blog started in 1999 – rather shortly after I had entered the IT industry. So it is an interesting time capsule, capturing technologies and trends I was sort of part of – including the relationship with one large well-known software company.

Somewhere deep in Joel’s blog I found references to another classic; it was in an advice on how to show passion as an applicant for a software developer job. Tell them how reading this moved you to tears:

Structure and Interpretation of Computer Programs. I think I have found the equivalent to Feynman’s Physics Lectures in computer science! I have hardly ever read a textbook or attended a class that was both so philosophically insightful and useful in a hands-on, practical way. Using Scheme (Lisp) as an example, important concepts are introduced step-by-step, via examples, viewed from different perspectives.

It was amazing how far you can get with purely Functional Programming. I did not even notice that they had not used a single assignment (Data Mutation) until far into the course.

The quality of the resources made available for free is incredible – which holds for all the content I am praising in this post: Full textbook, video lectures with transcripts, slides with detailed comments. It is also good to know and reassuring that despite the allegedly fast paced changes of technology, basic concepts have not changed that much since decades.

But if you are already indulging in nostalgic thoughts why not catch up on the full history of computing?

Creatures of Thought. A sublime book-like blog on the history of computing – starting from with the history of telephone networks and telegraphs, covering computing machines – electro-mechanical or electronic, related and maybe unappreciated hardware components like the relay, and including biographic vignettes of the heroes involved.

The author’s PhD thesis (available for download on the About page) covers the ‘information utility’ vision that was ultimately superseded by the personal computer. This is an interesting time capsule for me as well, as this story ends about where my personal journey started – touching personal PCs in the late 1980s, but having been taught the basics of programming via sending my batch jobs to an ancient mainframe.

From such diligently done history of engineering I can only learn not to rush to any conclusions. There are no simple causes and effects, or unambiguous stories about who invented what and who was first. It’s all subtle evolution and meandering narratives, randomness and serendipity. Quoting from the post that indicates the beginning of the journey, on the origins of the electric telegraph:

Our physics textbooks have packaged up the messy past into a tidy collection of concepts and equations, eliding centuries of development and conflict between competing schools of thought. Ohm never wrote the formula V = IR, nor did Maxwell create Maxwell’s equations.

Though I will not attempt to explore all the twists and turns of the intellectual history of electricity, I will do my best to present ideas as they existed at the time, not as we retrospectively fit them into our modern categories.

~

Phone, 1970s, Austria

The kind of phone I used at the time when the video lectures for Structure and Interpretation of Computer Programs had been recorded and when I submitted my batch jobs of Fortran code to be compiled. I have revived the phone now and then.

 

Heat Transport: What I Wrote So Far.

Don’t worry, The Subversive Elkement will publish the usual silly summer posting soon! Now am just tying up loose ends.

In the next months I will keep writing about heat transport: Detailed simulations versus maverick’s rules of thumb, numerical solutions versus insights from the few things you can solve analytically, and applications to our heat pump system.

So I checked what I have already written – and I discovered a series which does not show up as such in various lists, tags, categories:

[2014-12-14] Cistern-Based Heat Pump – Research Done in 1993 in Iowa. Pioneering work, but the authors dismissed a solar collector for economic reasons. They used a steady-state estimate of the heat flow from ground to the tank, and did not test the setup in winter.

Cistern-Based Water-Source Heat Pump System Design, 1993[2015-01-28] More Ice? Exploring Spacetime of Climate and Weather. A simplified simulation based on historical weather data – only using daily averages. Focus: Estimate of the maximum volume of ice per season, demonstration of yearly variations. As explained later (2017) in more detail I had to use information from detailed simulations though – to calculate the energy harvested by the collector correctly in such a simple model.

Simple simulations of volume of ice[2015-04-01] Ice Storage Challenge: High Score! Our heat pump created an ice cube of about 15m3 because we had turned the collector off. About 10m3 of water remained unfrozen, most likely when / because the ice cube touched ground. Some qualitative discussions of heat transport phenomena involved and of relevant thermal parameters.

Ice formation during the 'ice storage challenge'[2016-01-07] How Does It Work? (The Heat Pump System, That Is) Our system, in a slide-show of operating statuses throughput a typical year. For each period typical temperatures are given and the ‘typical’ direction of heat flow.

System in September - typical operations conditions[2016-01-22] Temperature Waves and Geothermal Energy. ‘Geothermal’ energy used by heat pumps is mainly stored solar energy. A simple model: The temperature at the surface of the earth varies sinusoidally throughout the year – this the boundary condition for the heat equation. This differential equation links the temporal change of temperature to its spatial variation. I solve the equation, show some figures, and check how results compare to the thermal diffusivity of ground obtained from measurements.

Measured 'wave' and propagation time[2016-03-01] Rowboats, Laser Pulses, and Heat Energy (Boring Title: Dimensional Analysis). Re-visiting heat transport and heat diffusion length, this time only analyzing dimensional relationships. By looking at the heat equation (without the need to solve it) a characteristic length can be calculated: ‘How far does heat get in a certain time?’

Temperature waves in ground - attenuation length of about 10 meters[2017-02-05] Earth, Air, Water, and Ice. Data analysis of the heating season 2014/15 (when we turned off the solar/air collector to simulate a harsher winter) – and an attempt to show energy storages, heat exchangers, and heat flows in one schematic. From the net energy ‘in the tank’ the contribution of ground can be calculated.

Energy storage, heat exchangers, heat flow[2017-02-22] Ice Storage Hierarchy of Needs. Continued from the previous post – bird’s eye view: How much energy comes from which sources, and which input parameters are critical? I try to answer when and if simple energy accounting makes sense in comparison to detailed simulations.

Hierarchy of needs - ambient energy in ice months[2017-05-02] Simulating Peak Ice. I compare measurements of the level in the tank with simulations of the evolution of the volume of ice. Simulations (1-minute intervals) comprise a model of the control logic, the varying performance factor of the heat pump, heat transport in ground, energy balances for the hot and cold tanks, and the heat exchangers connected in series.

Simulations of brine and tank temperature and volume of ice, based on system state in 1-minute intervals.(Adding the following after having published this post. However, there is no guarantee I will update this post forever ;-))

[2017-08-17] Simulations: Levels of Consciousness. Bird’s Eye View: How does simulating heat transport fit into my big picture of simulating the heat pump system or buildings or heating systems in general? I consider it part of the ‘physics’ layer of a hierarchy of levels.

Simulation - levels of consciousnessPlanned episodes? Later this year (2017) or next year I might write about the error made when considering simplified geometry – like modeling a linear 1D flow when the actualy symmetry is e.g. spherical.

Other People Have Lives – I Have Domains

These are just some boring update notifications from the elkemental Webiverse.

The elkement blog has recently celebrated its fifth anniversary, and the punktwissen blog will turn five in December. Time to celebrate this – with new domain names that says exactly what these sites are – the ‘elkement.blog‘ and the ‘punktwissen.blog‘.

Actually, I wanted to get rid of the ads on both blogs, and with the upgrade came a free domain. WordPress has a detailed cookie policy – and I am showing it dutifully using the respective widget, but they have to defer to their partners when it comes to third-party cookies. I only want to worry about research cookies set by Twitter and Facebook, but not by ad providers, and I am also considering to remove social media sharing buttons and the embedded tweets. (Yes, I am thinking about this!)

On the websites under my control I went full dinosaur, and the server sends only non-interactive HTML pages sent to the client, not requiring any client-side activity. I now got rid of the last half-hearted usage of a session object and the respective cookie, and I have never used any social media buttons or other tracking.

So there are no login data or cookies to protect, but yet I finally migrated all sites to HTTPS.

It is a matter of principle: I of all website owners should use https. Since 15 years I have been planning and building Public Key Infrastructures and troubleshooting X.509 certificates.

But of course I fear Google’s verdict: They have announced long ago to HTTPS is considered a positive ranking by its search engine. Pages not using HTTPS will be tagged as insecure using more and more terrifying icons – e.g. http-only pages with login buttons already display a striked-through padlock in Firefox. In the past years I migrated a lot of PKIs from SHA1 to SHA256 to fight the first wave of Insecure icons.

Finally Let’s Encrypt has started a revolution: Free SSL certificates, based on domain validation only. My hosting provider uses a solution based on Let’s Encrypt – using a reverse proxy that does the actual HTTPS. I only had to re-target all my DNS records to the reverse proxy – it would have been very easy would it not have been for all my already existing URL rewriting and tweaking and redirecting. I also wanted to keep the option of still using HTTP in the future for tests and special scenario (like hosting a revocation list), so I decided on redirecting myself in the application(s) instead of using the offered automated redirect. But a code review and clean-up now and then can never hurt 🙂 For large complex sites the migration to HTTPS is anything but easy.

In case I ever forget which domains and host names I use, I just need to check out this list of Subject Alternative Names again:

(And I have another certificate for the ‘test’ host names that I need for testing the sites themselves and also for testing various redirects ;-))

WordPress.com also uses Let’s Encrypt (Automattic is a sponsor), and the SAN elkement.blog is lumped together with several other blog names, allegedly the ones which needed new certificates at about the same time.

It will be interesting what the consequences for phishing websites will be. Malicious websites will look trusted as being issued certificates automatically, but revoking a certificate might provide another method for invalidating a malicious website.

Anyway, special thanks to the WordPress.com Happiness Engineers and support staff at my hosting provider Puaschitz IT. Despite all the nerdiness displayed on this blog I prefer hosted / ‘shared’ solutions when it comes to my own websites because I totally like it when somebody else has to patch the server and deal with attacks. I am an annoying client – with all kinds of special needs and questions – thanks for the great support! 🙂