Cloudy Troubleshooting (2)

Unrelated to part 1 – but the same genre.

Actors this time:

  • File Cloud: A cloud service for syncing and sharing files. We won’t drop a brand name, will we?
  • Client: Another user of File Cloud.
  • [Redacted]: Once known for reliability and as The Best Network.
  • Dark Platform: Wannabe hackers’ playground.
  • elkement: Somebody who sometimes just wants to be an end user, but always ends up sniffing and debugging.

There are no dialogues with human life-forms this time, only the elkement’s stream of consciousness, interacting with the others via looking at things at a screen.

elkement: Time for a challenging Sunday hack!

elkement connects to the The Dark Platform. Hardly notices anything in the real world anymore. But suddenly elkement looks at the clock – and at File Cloud’s icon next to it.

elkement: File Cloud, what’s going on?? Seems you have a hard time Connecting… for hours now? You have not even synced my hacker notes from yesterday evening?

elkement tries to avoid to look at File Cloud, but it gets too painful.

elkement: OK – let’s consider the File Cloud problem the real Sunday hacker’s challenge…

elkement walks through the imaginary checklist:

  • File Cloud mentioned on DownDetector website? No.
  • Users tweeting about outage? No.
  • Do the other cloudy apps work fine? Yes.
  • Do other web sites work fine? Yes.
  • Does my router needs its regular reboots because it’s DNS server got stuck? No.
  • Should I perhaps try the usual helpdesk recommendation? Yes. (*)

(*) elkement turns router and firewall off and on again. Does not help.

elkement gets worried about Client using File Cloud, too. Connects to Client’s network – via another cloudy app (that obviously also works).

  • Does Client has the same issues? Yes and No – Yes at one site, No at another site.

elkement: Oh no – do I have to setup a multi-dimensional test matrix again to check for weird dependencies?

Coffee Break. Leaving the hacker’s cave. Gardening.

elkement: OK, let’s try something new!

elkement connects to super shaky mobile internet via USB tethering on the smart phone.

  • Does an alternative internet connection fix File Cloud? Yes!!

elkement: Huh!? Will now again somebody explain to me that a protocol (File Cloud) is particularly sensitive to hardly noticeable network disconnects? Is it maybe really a problem with [Redacted] this time?

elkement checks out DownDetector – and there they are the angry users and red spots on the map. They mention that seemingly random websites and applications fail. And that [Redacted] is losing packets.

elkement: Really? Only packets for File Cloud?

elkement starts sniffing. Checks IP addresses.

(elkement: Great, whois does still work, despite the anticipated issues with GDPR!)

elkement spots communication with File Cloud. File Cloud client and server are stuck in a loop of misunderstandings. File Cloud client is rude and says: RST, then starts again. Says Hello. They never shake hands as a previous segment was not captured.

elkement: But why does all the other stuff work??

elkement googles harder. Indeed, some other sites might be slower – not The Dark Platform, fortunately. Now finally Google and duckduckgo stop working, too. 

elkement: I can’t hack without Google.

elkement hacks something without Google though. Managed to ignore File Cloud’s heartbreaking connection attempts.

A few hours later it’s over. File Cloud syncs hacker notes. Red spots on DownDetector start to fade out while the summer sun is setting.

~

FIN, ACK

Infinite Loop: Theory and Practice Revisited.

I’ve unlocked a new achievement as a blogger, or a new milestone as a life-form. As a dinosaur telling the same old stories over and over again.

I started drafting a blog post, as I always do since a while: I do it in my mind only, twist and turn in for days or weeks – until I am ready to write it down in one go. Today I wanted to release a post called On Learning (2) or the like. I knew I had written an early post with a similar title, so I expected this to be a loosely related update. But then I checked the old On Learning post: I found not only the same general ideas but the same autobiographical anecdotes I wanted to use now – even  in the same order.

2014 I had looked back on being both a teacher and a student for the greater part of my professional life, and the patterns were always the same – be the field physics, engineering, or IT security. I had written this post after a major update of our software for analyzing measurement data. This update had required me to acquire new skills, which was a delightful learning experience. I tried to reconcile very different learning modes: ‘Book learning’ about so-called theory, including learning for the joy of learning, and solving problems hands-on based on the minimum knowledge absolutely required.

It seems I like to talk about the The Joys of Theory a lot – I have meta-posted about theoretical physics in general, more than oncegeneral relativity as an example, and about computer science. I searched for posts about hands-on learning now – there aren’t any. But every post about my own research and work chronicles this hands-on learning in a non-meta explicit way. These are the posts listed on the heat pump / engineering page,  the IT security / control page, and some of the physics posts about the calculations I used in my own simulations.

Now that I am wallowing in nostalgia and scrolling through my old posts I feel there is one possibly new insight: Whenever I used knowledge to achieve a result that I really needed to get some job done, I think about this knowledge as emerging from hands-on tinkering and from self-study. I once read that many seasoned software developers also said that in a survey about their background: They checked self-taught despite having university degrees or professional training.

This holds for the things I had learned theoretically – be it in a class room or via my morning routine of reading textbooks. I learned about differential equations, thermodynamics, numerical methods, heat pumps, and about object-oriented software development. Yet when I actually have to do all that, it is always like re-learning it again in a more pragmatic way, even if the ‘class’ was very ‘applied’, not much time had passed since learning only, and I had taken exams. This is even true for the archetype all self-studied disciplines – hacking. Doing it – like here  – white-hat-style 😉 – is always a self-learning exercise, and reading about pentesting and security happens in an alternate universe.

The difference between these learning modes is maybe not only in ‘the applied’ versus ‘the theoretical’, but it is your personal stake in the outcome that matters – Skin In The Game. A project done by a group of students for the final purpose of passing a grade is not equivalent to running this project for your client or for yourself. The point is not if the student project is done for a real-life client, or the task as such makes sense in the real world. The difference is whether it feels like an exercise in an gamified system, or whether the result will matter financially / ‘existentially’ as you might try to empress your future client or employer or use the project results to build your own business. The major difference is in weighing risks and rewards, efforts and long-term consequences. Even ‘applied hacking’ in Capture-the-Flag-like contests is different from real-life pentesting. It makes all the difference if you just loose ‘points’ and miss the ‘flag’, or if you inadvertently take down a production system and violate your contract.

So I wonder if the Joy of Theoretical Learning is to some extent due to its risk-free nature. As long as you just learn about all those super interesting things just because you want to know – it is innocent play. Only if you finally touch something in the real world and touching things has hard consequences – only then you know if you are truly ‘interested enough’.

Sorry, but I told you I will post stream-of-consciousness-style now and then 🙂

I think it is OK to re-use the image of my beloved pre-1900 physics book I used in the 2014 post:

Where Are the Files? [Winsol – UVR16x2]

Recently somebody has asked me where the log files are stored. This question is more interesting then it seems.

We are using the freely programmable controller UVR16x2 (and its predecessor) UVR1611) …

.. and their Control and Monitoring Interface – CMI:The CMI is a data logger and runs a web server. It logs data from the controllers (and other devices) via CAN bus – I have demonstrated this in a contrived example recently, and described the whole setup in this older post.

IT / smart home nerds asked me why there are two ‘boxes’ as other solutions only use a ‘single box’ as both controller and logger. I believe separating these functions is safer and more secure: A logger / web server should not be vital to run the controller, and any issues with these auxiliary components must impact the controller’s core functions.

Log files are stored on the CMI in a proprietary format, and they can retrieved via HTTP using the software Winsol. Winsol lets you visualize data for 1 or more days, zoom in, define views etc. – and data can be exported as CSV files. This is the tool we use for reverse engineering hydraulics and control logic (German blog post about remote hydraulics surgery):

In the latest versions of Winsol, log files are per default stored in the user’s profile on Windows:
C:\Users\[Username]\Documents\Technische Alternative\Winsol

I had never paid much attention to this; I had always changed that path in the configuration to make backup and automation easier. The current question about the log files’ location was actually about how I managed to make different users work with the same log files.

The answer might not be obvious because of the historical location of the log files:

Until some version of Winsol in use in 2017 log files were by stored in the Program Files folder, or at least Winsol tried to use that folder. Windows does not allow this anymore for security reasons.

If Winsol is upgraded from an older version, settings might be preserved. I did my tests  with Winsol 2.07 upgraded from an earlier version. I am a bit vague about versions as I did not test different upgrade paths in detail My point is users of control system’s software tend to be conservative when it comes to changing a running system – an older ‘logging PC’ with an older or upgraded version of Winsol is not an unlikely setup.

I started debugging on Windows 10 with the new security feature Controlled Folder Access enabled. CFA, of course, did not know Winsol, considered it an unfriendly app … to be white-listed.

Then I was curious about the default log file folders, and I saw this:

In the Winsol file picker dialogue (to the right) the log folders seem to be in the Program Files folder:
C:\Program Files\Technische Alternative\Winsol\LogX
But in Windows Explorer (to the left) there are no log files at that location.

What does Microsoft Sysinternals Process Monitor say?

There is a Reparse Point, and the file access is redirected to the folder:
C:\Users\[User]\AppData\Local\VirtualStore\Program Files\Technische Alternative\Winsol
Selecting this folder directly in Windows Explorer shows the missing files:

This location can be re-configured in Winsol to allow different users to access the same files (Disclaimer: Perhaps unsupported by the vendor…)

And there are also some truly user-specific configuration files in the user’s profile, in
C:\Users\[User]\AppData\Roaming\Technische Alternative\Winsol

Winsol.xml is e.g. for storing the list of ‘clients’ (logging profiles) that are included in automated processing of log files, and cookie.txt is the logon cookie for access to the online logging portal provided by Technische Alternative. If you absolutely want to switch Windows users *and* switch logging profiles often *and* sync those you have to tinker with Winsol.xml, e.g. by editing it using a script (Disclaimer again: Unlikely to be a supported way of doing things ;-))

As a summary, I describe the steps required to migrate Winsol’s configuration to a new PC and prepare it for usage by different users.

  • Install the latest version of Winsol on the target PC.
  • If you use Controlled Folder Access on Windows 10: Exempt Winsol as a friendly app.
  • Copy the contents of C:\Users\[User]\AppData\Roaming\Technische Alternative\Winsol from the user’s profile on the old machine to the new machine (user-specific config files).
  • If the log file folder shows up at a different path on the two machines – for example when using the same folder via a network share – edit the path in Winsol.xml or configure it in General Settings in Winsol.
  • Copy your existing log data to this new path. LogX contains the main log files, Infosol contain clients’ data. The logging configuration for each client, e.g. the IP address or portal name of the logger, is included in the setup.xml file in the root of each client’s folder.

Note: If you skip some Winsol versions on migrating/upgrading the structure of files might have changed – be careful! Last time that happened by the end of 2016 and Data Kraken had to re-configure some tentacles.

Cloudy Troubleshooting

Actors:

  • Cloud: Service provider delivering an application over the internet.
  • Client: Business using the Cloud
  • Telco: Service provider operating part of the network infrastructure connecting them.
  • elkement: Somebody who always ends up playing intermediary.

~

Client: Cloud logs us off ever so often! We can’t work like this!

elkement: Cloud, what timeouts do you use? Client was only idle for a short break and is logged off.

Cloud: Must be something about your infrastructure – we set the timeout to 1 hour.

Client: It’s becoming worse – Cloud logs us off every few minutes even we are in the middle of working.

[elkement does a quick test. Yes, it is true.]

elkement: Cloud, what’s going on? Any known issue?

Cloud: No issue in our side. We have thousands of happy clients online. If we’d have issues, our inboxes would be on fire.

[elkement does more tests. Different computers at Client. Different logon users. Different Client offices. Different speeds of internet connections. Computers at elkement office.]

elkement: It is difficult to reproduce. It seems like it works well for some computers or some locations for some time. But Cloud – we did not have any issues of that kind in the last year. This year the troubles started.

Cloud: The timing of our app is sensitive: If network cards in your computers turn on power saving that might appear as a disconnect to us.

[elkement learns what she never wanted to know about various power saving settings. To no avail.]

Cloud: What about your bandwidth?… Well, that’s really slow. If all people in the office are using that connection we can totally understand why our app sees your users disappearing.

[elkement on a warpath: Tracking down each application eating bandwidth. Learning what she never wanted to know about tuning the background apps, tracking down processes.]

elkement: Cloud, I’ve throttled everything. I am the only person using Clients’ computers late at night, and I still encounter these issues.

Cloud: Upgrade the internet connection! Our protocol might choke on a hardly noticeable outage.

[elkement has to agree. The late-night tests were done over a remote connections; so measurement may impact results, as in quantum physics.]

Client: Telco, we buy more internet!

[Telco installs more internet, elkement measures speed. Yeah, fast!]

Client: Nothing has changed, Clouds still kicks us out every few minutes.

elkement: Cloud, I need to badger you again….

Cloud: Check the power saving settings of your firewalls, switches, routers. Again, you are the only one reporting such problems.

[The router is a blackbox operated by Telco]

elkement: Telco, does the router use any power saving features? Could you turn that off?

Telco: No we don’t use any power saving at all.

[elkement dreams up conspiracy theories: Sometimes performance seems to degrade after business hours. Cloud running backup jobs? Telco’s lines clogged by private users streaming movies? But sometimes it’s working well even in the location with the crappiest internet connection.]

elkement: Telco, we see this weird issue. It’s either Cloud, Client’s infrastructure, or anything in between, e.g. you. Any known issues?

Telco: No, but [proposal of test that would be difficult to do]. Or send us a Wireshark trace.

elkement: … which is what I planned to do anyway…

[elkement on a warpath 2: Sniffing, tracing every process. Turning off all background stuff. Looking at every packet in the trace. Getting to the level where there are no other packets in between the stream of messages between Client’s computers and Cloud’s servers.]

elkement: Cloud, I tracked it down. This is not a timeout. Look at the trace: Server and client communicating nicely, textbook three-way handshake, server says FIN! And no other packet in the way!

Cloud: Try to connect to a specific server of us.

[elkement: Conspiracy theory about load balancers]

elkement: No – erratic as ever. Sometimes we are logged off, sometimes it works with crappy internet. Note that Client could work during vacation last summer with supper shaky wireless connections.

[Lots of small changes and tests by elkement and Cloud. No solution yet, but the collaboration is seamless. No politics and finger-pointing who to blame – just work. The thing that keeps you happy as a netadmin / sysadmin in stressful times.]

elkement: Client, there is another interface which has less features. I am going to test it…

[elkement: Conspiracy theory about protocols. More night-time testing].

elkement: Client, Other Interface has the same problems.

[elkement on a warpath 3: Testing again with all possible combinations of computers, clients, locations, internet connections. Suddenly a pattern emerges…]

elkement: I see something!! Cloud, I believe it’s user-dependent. Users X and Y are logged off all the time while A and B aren’t.

[elkement scratches head: Why was this so difficult to see? Tests were not that unambiguous until now!]

Cloud: We’ve created a replacement user – please test.

elkement: Yes – New User works reliably all the time! 🙂

Client: It works –  we are not thrown off in the middle of work anymore!

Cloud: Seems that something about the user on our servers is broken – never happened before…

elkement: But wait 😦 it’s not totally OK: Now logged off after 15 minutes of inactivity? But never mind – at least not as bad as logged off every 2 minutes in the middle of some work.

Cloud: Yeah, that could happen – an issue with Add-On Product. But only if your app looks idle to our servers!

elkement: But didn’t you tell us that every timeout ever is no less than 1 hour?

Cloud: No – that 1 hour was another timeout …

elkement: Wow – classic misunderstanding! That’s why it is was so difficult to spot the pattern. So we had two completely different problems, but both looked like unwanted logoffs after a brief period, and at the beginning both weren’t totally reproducible.

[elkement’s theory validated again: If anything qualifies elkement for such stuff at all it was experience in the applied physics lab – tracking down the impact of temperature, pressure and 1000 other parameters on the electrical properties of superconductors… and trying to tell artifacts from reproducible behavior.]

~

Cloudy

Logging Fun with UVR16x2: Photovoltaic Generator – Modbus – CAN Bus

The Data Kraken wants to grow new tentacles.

I am playing with the CMI – Control and Monitoring Interface – the logger / ‘ethernet gateway’ connected to our control units (UVR1611, UVR16x2) via CAN bus. The CMI has become a little Data Kraken itself: Inputs and outputs can be created for CAN bus and Modbus, and data from most CAN devices can also be logged via JSON.

Are these features useful to integrate the datalogger of our photovoltaics inverter – Fronius Symo 4.5-3-M? I am now logging data to an USB stick and feed the CSV files to the SQL Server Data Kraken. The USB logger’s logging interval is 5 minutes whereas Modbus TCP allows for logging every few seconds. The inverter has built-in energy management features, but it can only ‘signal’ via a relay which also requires proper wiring. Modbus TCP, on the other hand, could use the existing WLAN connection of the inverter and the control unit could do something smarter with the sensor reading of the output power.

My motivation is to test if you – as an UVR16x2 user – can re-use a logger you  already have – the CMI – as much as possible, avoiding the need to run another ‘logging server’ all the time (Also my SQL Server is for analysis, not for real-time logging). I know that there are many open source Modbus clients available and that it is easy to write a Python script.

Activate Modbus on the inverter: I prefer floating numbers to integers plus a scaling factor, and I turn off the option to make changes via Modbus:

Modbus settings, Fronius Datalogger, inverter’s local web server. 502 is the TCO standard port. The alternative to floating numbers is integers plus a varying scaling factor (SF), to be found in another register.

Check Fronius documentation of its Modbus registers: The document is currently available here. There are different sets of registers related to the inverter or associated with one string of PV panels:

PDF p.97, Common Inverter Model. For logging AC output power you need:  Address 40092, type of register 3 (read and hold), datatype float32 (‘corresponds’ to two 16bit integer register, thus size 2).

The address to be configured on a Modbus client is smaller than this address by 1 – so 40091 needs to be set to log AC power.

Using these configuration parameters an analog Modbus Input is added at the CMI. The signal is ‘digital’ – but in field-bus-language everything that is not a single bit – 0/1 – seems to be ‘analog’.

Modbus input at the CMI. Input value:  32bits read from the bus interpreted as an integer. Actual value: Integer part of the ‘true value’ = the 32bits interpreted as 32-bit float.

Yes, I checked the network trace 😉 as the byte order dropdown menu confused me: According to the Modbus protocol specification Big Endian is required, not an option.

Factors and data types: Only integer values are understood by CAN devices. Decimal places might be indicated by a scaling factor. The PV power value in Watt has enough significant digits; so the integer part of the float number is fine. But for current in Ampere – typically about 15A maximum – a Factor of 10 would be better. It would not have helped to select int + scaling factor at the inverter: The scaling factor would be stored in a second register, there is a different factor for every parameter, and you cannot configure another ‘scaling factor register’ per input at the CMI. Theoretically you could log the scaling factor separately and re-scale the value in a custom application – but then I would use a separate, custom logger.

In any case, if you screw this up, you see non-sensible numbers of the CAN bus: Slowly evolving positive values – like PV power on a sunny day – are displayed as wild variations of signed integers between -32000 and 32000 😉

Where are the ‘logged’ data? The CMI is first and foremost the data logger for the control units. The CMI does not immediately store the data from Modbus inputs in a  local ‘logging database’. All I have achieved so far is to display the value on the Settings page. The CMI can only log values from the CAN bus or DL bus. So we need an…

… Analog CAN Output at the CMI:

The CMI has the default node number 56 on the CAN bus. Other CAN devices on the bus can query it for this parameter by specifying node 56 and output no 1.

These are the devices on our CAN bus:

CAN bus displayed on the CMI’s website. UVR1611 and UVR16x2 controllers can be managed by clicking their icons – which brings up a web page that resembles the controller’s local display.

The CMI’s Logging page looks tempting – can we simply select the CMI itself as a CAN logging source – CAN 56?

Configuration of the devices the CMI logs data from, via CAN bus. CAN 1 – UVR1611, CAN 2 – UVR16x3, CAN 41 – energy meter CAN-EZ.

Nothing stops you from selecting CAN 56 in this dropdown menu, but it does not end well:

CAN error message displayed at the logger CMI when you try to configure the CMI also as a logging source.

We need a round-trip: Data needs to be sent to a supported device first – one of the controllers on the CAN bus. We need an…

… Analog CAN input / network variable at the UVR16x2:

Configuration of a CAN input at the controller UVR16x2 (via CMI’s web interface to the controller).

The value of AC power is displayed as integer without scaling. Had a factor of 10 been used at the Modbus input it would be ‘corrected’ here, using the Unit called dimensionless,1.

logging-uvr16x2-can-network-input-can-value-display

Values received by the controller UVR16x2 over CAN bus.

Result of all this: UVR16x2 knows PV power and can use it do magic smart things when controlling the heat pump. On the other hand, CMI can log this value – in the same way it logs all other sensor readings.

Log files are retrieved by Winsol, the free logging software for the CMI …

Logged visualized with Winsol. Logfiles are downloaded from the CMI on the internal LAN or via Technsche Alternative’s web portal. PV power (PV.Leistung.Watt) is displayed together with global radiation on a vertical plane (GBS, at the solar/air collector for the theat pump), ambient temperature (red), temperature of solar/air collector (orange)

… or logging is configured at the web portal cmi.ta.co.at …

Configuration of logging at cmi.ta.co.at: Supported loggers are UVR1611 and UVR16x2. Values to be logged are selected from all direct inputs / outputs / functions and from CAN network inputs and outputs.

… and data can be viewed online:

Data visualized at cmi.ta.co.at. Data logged via CAN are sent from the CMI to the web portal.

Using this kind of logging for all values the inverter provides would be costly: It’s not just a column you add to a log file, but you occupy one of the limited inputs and outputs at the CMI and the controller. If you really need to know the voltage between phase 1 and 2 or apparent power you better stick with the USB file or use a separate Modbus logger like a Rasbperry Pi. This project is great and documented very well – data acqusition from a Symo inverter using Python plus a web front end.

Sending Modbus data back and forth from the CMI to UVR controllers is only worth the efforts if you need them for control, not for ‘nice-to-have’ logging.

Can the Efficiency Be Greater Than One?

This is one of the perennial top search terms for this blog.

Anticlimactic answer: Yes, because input and output are determined also by economics, not only by physics.

Often readers search for the efficiency of a refrigerator. Its efficiency, the ratio of output and input energies, is greater than 1 because the ambient energy is free. System’s operators are interested in the money they pay the utility, in relation to the resulting energy for cooling.

If you use the same thermodynamic machine either as a refrigerator or as a heat pump, efficiencies differ: The same input energy drives the compressor, but the relevant output energy is either the energy released to the ‘hot side’ at the condenser or the energy used for evaporating the refrigerant at the ‘cool side’:

The same machine / cycle is used as a heat pump for heating (left) or a refrigerator or AC for cooling (right). (This should just highlight the principles and does not include any hydraulic details, losses etc. related to detailed differences between refrigerators / ACs and heat pumps.)

For photovoltaic panels the definition has sort of the opposite bias: The sun does not send a bill – as PV installers say in their company’s slogan – but the free solar ambient energy is considered, and thus their efficiency is ‘only’ ~20%.

Half of our generator, now operational for three years: 10 panels, oriented south-east, 265W each, efficiency 16%. (The other 8 panels are oriented south-west).

When systems are combined, you can invent all kinds of efficiencies, depending on system boundaries. If PV panels are ‘included’ in a heat pump system (calculation-wise) the nominal electrical input energy becomes lower. If solar thermal collectors are added to any heating system, the electrical or fossil fuel input decreases.

Output energy may refer to energy measured directly at the outlet of the heat pump or boiler. But it might also mean the energy delivered to the heating circuits – after the thermal losses of a buffer tank have been accounted for. But not 100% of these losses are really lost, if the buffer tank is located in the house.

I’ve seen many different definitions in regulations and related software tools, and you find articles about how to game interpret these guidelines to your advantage. Tools and standards also make arbitrary assumptions about storage tank losses, hysteresis parameter and the like – factors that might be critical for efficiency.

Then there are scaling effects: When the design heat loads of two houses differ by a factor of 2, and the smaller house would use a scaled down heat pump (hypothetically providing 50% output power at the same efficiency), the smaller system’s efficiency is likely to be a bit lower. Auxiliary consumers of electricity – like heating circuit pumps or control systems – will not be perfectly scalable. But the smaller the required output energy is, the better it can be aligned with solar energy usage and storage by a ‘smart’ system – and this might outweigh the additional energy needed for ‘smartness’. Perhaps intermittent negative market prices of electricity could be leveraged.

Definitions of efficiency are also culture-specific, tailored to an academic discipline or industry sector. There are different but remotely related concepts of rating how useful a source of energy is: Gibbs Free Energy is the maximum work a system can deliver, given that pressure and temperature do not change during the process considered – for example in a chemical reaction. On the other hand, Exergy is the useful ‘available’ energy ‘contained’ in a (part of a) system: Sources of energy and heat are rated; e.g. heat energy is only mechanically useful up to the maximum efficiency of an ideal Carnot process. Thus exergy depends on the temperature of the environment where waste heat ends up. The exergy efficiency of a Carnot process is 1, as waste heat is already factored in. On the other hand, the fuel used to drive the process may or may not be included and it may or may not be considered pure exergy – if it is, energy and exergy efficiency would be the same again. If heat energy flows from the hot to the cold part of a system in a heat exchanger, no energy is lost – but exergy is.

You could also extend the system’s boundary spatially and on the time axis: Include investment costs or the cost of harm done to the environment. Consider the primary fuel / energy / exergy to ‘generate’ electricity: If a thermal power plant has 40% efficiency then the heat pump’s efficiency needs to be at least 2,5 to ‘compensate’ for that.

In summary, ‘efficiency’ is the ratio of an output and an input energy, and the definitions may be rather arbitrary as and these energies are determined by a ‘sampling’  time, system boundaries, and additional ‘ratings’.

Let Your Hyperlinks Live Forever!

It is the the duty of a Webmaster to allocate URIs which you will be able to stand by in 2 years, in 20 years, in 200 years. This needs thought, and organization, and commitment. (https://www.w3.org/Provider/Style/URI)

Joel Spolsky did it:

 I’m bending over backwards not to create “linkrot” — all old links to Joel on Software stories have been replaced with redirects, so they should still work. (November 2001)

More than once:

I owe a huge debt of gratitude to [several people] for weeks of hard work on creating this almost perfect port of 16 years of cruft, preserving over 1000 links with redirects… (December 2016).

Most of the outgoing URLs linked by Joel of Software have rotted, with some notable exceptions: Jakob Nielsen’s URLs do still work, so they live what he preached – in 1998:

… linkrot contributes to dissolving the very fabric of the Web: there is a looming danger that the Web will stop being an interconnected universal hypertext and turn into a set of isolated info-islands. Anything that reduces the prevalence and usefulness of cross-site linking is a direct attack on the founding principle of the Web.

No excuses if you are not Spolsky- or Nielsen-famous – I did it too, several times. In 2015 I rewrote the application for my websites from scratch and redirected every single .asp URL to a new friendly URL at a new subdomain.

I am obsessed with keeping old URLs working. I don’t like it if websites are migrated to a new content management system, changing all the URLs.

I checked all that again when migrating to HTTPS last year.

So I am a typical nitpicking dinosaur, waxing nostalgic about the time when web pages were still pages, and when Hyperlinks Subverted Hierarchy. When browsers were not yet running an OS written in Javascript and hogging 70% of your CPU for ad-tracking or crypto-mining.

The dinosaur is grumpy when it has to fix outgoing URLs on this blog. So. Many. Times. Like every second time I test a URL that shows up in my WordPress statistics as clicked, it 404s. Then I try to find equivalent content on the same site if the domain does still exist – and had not been orphaned and hijacked by malvertizers. If I am not successful I link to a version of this content on web.archive.org, track down the content owner’s new site, or find similar content elsewhere.

My heart breaks when I see that it’s specifically the interesting, unusual content that users want to follow from here – like hard-to-find historical information on how to build a heat pump from clay tablets and straw. My heart breaks even more when the technical content on the target site gets dumbed down more and more with every URL breaking website overhaul. But OK – you now have this terrific header image with a happy-people-at-work stock photo that covers all my desktop so that I have to scroll for anything, and the dumbed down content is shown in boxes that pop up and whirl – totally responsive, though clunky on a desktop computer.

And, yes: I totally know that site owners don’t own me anything. Just because you hosted that rare and interesting content for the last 10 years does not mean you have to do that forever.

But you marketing ninjas and website wranglers neglected an important point: We live in the age of silly gamification that makes 1990s link building pale: I like yours and you like mine. Buy Followers. Every time I read a puffed up Case Study for a project I was familiar with as an insider, I was laughing for minutes and then checked if it was not satire.

In this era of fake word-of-mouth marketing you get incoming links. People say something thoughtful, maybe even nice about you just because they found your content interesting and worth linking not because you play silly games of reciprocating. The most valuable links are set by people you don’t know and who did not anticipate you will ever notice their link. As Nassim Taleb says: Virtue is what you do when nobody is looking.

I would go to great lengths not to break links to my sites in those obscure DIY forums whose posts are hardly indexed by search engines. At least I would make a half-hearted attempt at redirecting to a custom 404 page that explains where you might the moved content. Or just keep the domain name intact. Which of course means not to register a catchy domain name for every product in the first place. Which I consider bad practice anyway – training users to fall for phishing, by getting them used to jumping from one weird but legit domain to another.

And, no, I don’t blame you personally, poor stressed out web admin who had to get the new site up and running before April 1st, because suits in your company said the world would come to an end otherwise. I just think that our internet culture that embraces natural linkrot so easily is as broken as the links.

I tag this as Rant, but it is a Plea: I beg you, I implore you to invest just a tiny part of the time, budget and efforts you allocated to Making the Experience of Your Website Better to making some attempt at keeping your URLs intact. They are actually valuable for others – something you should be proud of.