Our main data logger is the Control and Monitoring Interface of the freely programmable controller UVR16x2. There are two pieces of hardware you need for logging – the actual control unit and the logger connected to the controller via the CAN bus. This ‘architecture’ might be due to historical reasons, but I like the separation of critical control functions and nice-to-have logging. An operating system for control should do just that – I don’t want a web server and TCP/IP on the controller itself.

But sometimes it might be useful to send the controller a non-critical ‘nice-to-have’ signal. If you want to optimize the usage of photovoltaic energy, the controller should know about the PV ‘production’ and current consumption. As the CMI supports Modbus inputs and our Fronius Symo inverter has a Modbus interface, PV ‘generation’ data can be logged via Modbus. However, our smart meter has ‘only’ a JSON interface – and you can always scrape data from the website as last resort. But while the CMI is a web server with visualization a JSON API, it cannot log ‘from’ JSON.

My Data Kraken is a SQL Server that is fed log files from different data loggers – the largest one from the CMI, and others are from our smart power meter’s logger, from our photovoltaic inverter, and from the heat pump’s internal CAN bus. Separate log files provide for built-in backup – but what if I could send these data to the UVR16x2 in real time in addition?

Currently different loggers log data at different points of time, so you cannot simply join all these tables together. Of course you can simply overlay the data without the time stamps being exactly equal. As I am mainly interested in daily consumption or performance data, the Data Kraken creates a series of nested views, finally spitting out one row per day, and those views for different loggers can be joined again. So far I could not, say, calculate the photovoltaic power used directly at each instant.

I was looking for … a babel fish! A universal protocol translator that …

• collects data from other sources which UVR cannot directly process.
• processes these data and calculates derived values, without having to ‘waste’ UVR16x2’s building blocks for functions on that.
• and provides these data to UVR16x2 in a proper format as an input relevant for control.

As I had already tested logging from Modbus sources, the logical solution was:

Starting my own Modbus server at the Raspberry Pi we already use for logging from the heat pump’s CAN bus.

There is a great python library called pymodbus which makes this task easy:

You write values to your own registers by building a payload from values in different formats. Here is an example using the payload builder to create a ‘stack’ of registers containing payloads of different types. The payload builder can be fed values retrieved from another logger, or created by whatever complex calculation. These values are then ‘sent out’ via your Modbus server interface. That infamous usage of weather forecast data from some public API for optimizing battery storage could theoretically be one application for that.

I update the values ‘internally’ using the setvalues method of the context, like this. Another option would be to create a client connecting to your own local server and writing to the registers.

In order to change the values while the Modbus server runs without interruption, Python threading comes in handy. The ‘context’ is a global object, and one thread constantly updates the values in an infinite loop, and the other thread is for running the actual TCP server. Using log.debug lets you follow the details of the communication on the server – and helps with sorting out any byte ordering issues ;-)

On the CMI you configure the related inputs connecting to your Raspberry Pi Modbus server:

Currently the CMI reads the following from my ‘fake’ Modbus device:

Net electrical power measured by the smart meter: Total power for all three AC phases, and each phase separately (OBIS object model – “*2.8.0” and “*1.8.0”).

PV output power: The CMI also logs data from the PV inverter directly. This is a double Modbus hop ‘for fun’!

PV power used immediately and electrical power used in total in the building: Calculated from the smart meter’s output values (via JSON) and the PV inverter’s output (via Modbus).

PV utilization: The directly used PV power is related to either power consumed or power ‘generated’ right now. Over a season these values are ~30% (self-sufficiency quota) and ~35 (self-consumption quota).

Calculations based on the ‘current’ PV power should not be taken too seriously as you might encounter those alien “>100%” spikes. The default log files with 5-minute averages still makes sense.

To actually log these values to CMI’s log file and to use them as control input requires the configuration of a CAN input at the UVR, as described here.

The Babelfish is now translating from Modbus to Modbus, from JSON to Modbus, and from ‘internal calculations’ to Modbus – and it can be extended to translate from a foreign CAN bus to UVR’s in a sense.

Frequently asked question:

How can I log from the heat pump’s internal CAN bus to UVR16x2? It’s all CAN buses, why don’t you simply connect them instead of having a separate CAN logger installed on a Pi?

‘Connected to CAN bus’ does not necessarily mean that those devices speak the same dialect at all network levels of the protocol. While UVR uses CANopen, the heat pump’s CAN protocol is proprietary (though similar to CANopen). In addition, you might want to separate the control bus for the heat pump from your standard CAN bus. What if an exotic issue with one of the devices confuses the heat pump? I remember how an IP TV solution broke the CMI’s predecessor BL-NET, and the solution was to put the CMI into a different subnet, behind another router.

My Modbus Babel fish gathers the heat pump’s values from its internal can bus and saves them to a CSV file (… which is then fed to the SQL Server Kraken). The values could as well be served via my Modbus server.

The final glorious chain looks like this :-)

• CAN bus heat pump –> CAN sniffer on Raspberry Pi
• Python Modbus Server on Raspberry Pi –> Modbus input on CMI
• CAN output on the CMI –> CAN input on UVR16x2
• –> UVR uses CAN input in a control function
• –> CMI logs this value from the CAN bus ‘again’ –> into its log file