Data Logging with UVR1611 – FAQ

I have received several questions related to my article on data logging on this blog, or to my postings on monitoring and control on our German blog.

Thus I have decided to write the article I would have wanted to read when I once made myself familiar with this. The target audience for this article are IT guys / web developers / ambitious DIY enthusiasts trying to make sense of the interfaces provided by the freely programmable controller UVR1611.

We use this device as the main controller of heat pump systems we design, and for monitoring and optimizing heating systems in general.

UVR1611, customized welcome screen

This control unit receives data from sensors (temperature, flow, irradiation,…) and controls pumps and valves accordingly.

You interact with the unit via programming it directly – using a scroll wheel and buttons, but this should only be used for changing parameters such as a temperature set point. The control logic should rather be developed with a graphical programming application, called TAPPS. This software creates the functional data which have to be uploaded to the controller.

Snapshot of a part of the control logic of our heat pump system, as designed with TAPPS.

Sensors and manageable devices as valves talk to the control unit via traditional field buses, such as CAN bus (e.g. also used in cars’ internal networks) and so-called DL Bus. In order to access UVR1611 via a standard TCP/IP computer network you need a kind of gateway. This device does not only convert the field bus communication, but also serves as a repository for the logged measurement data. 

In our control network there are two different kinds of loggers, for ‘research purposes’. According to the vendor Technische Alternative GmbH, using two loggers in parallel is not supported and discouraged at is might cause issues. For us it works fine, but only try at your own risk:

Data loggers CMI and BL-NET

Data loggers by Technische Alternative: CMI (Control and Monitoring Interface) to the left, BL-NET (Boot Loader)  in the middle, standard ethernet switch to the right. The CAN bus cable is connected to both of them via the blue connector.

Two loggers – CMI and BL-NET – are connected to UVR1611 via CAN bus – a linear bus that needs to be terminated on both ends. Each of the devices is connected to the local computer network via standard ethernet wiring.

CMI is BL-NET’s successor although there might be no immediate reason to upgrade. Starting from scratch now, I would recommend CMI though.

This is how our local CAN bus looks like now, as displayed in CMI’s web interface:

Devices on CAN bus, displayed by CMI

Devices on CAN bus, displayed by CMI: Loggers CMI and BL-NET, plus and energy counter (CAN-EZ) and an extension of inputs/outputs (CAN-IO), UVR1611’s successor UVR16x2.

So there is a web server on CMI, which can be accessed locally. As described in the previous article, you can also access it via a ‘cloud-based’ portal.

In summary, this logger / gateway allows for the following:

1) Uploading functional data (programming logic) to UVR1611, by uploading the file from  computer onto the SD card inserted into the device and then dragging the file to the control unit’s icon in this web interface. This is an improvement over BL-NET which required an additional software application called Memory Manager to transfer functional data to the logger first. Existing functional data can be downloaded and inspected with the recent versions of TAPPS.

2) Accessing the control unit as if you would use the scroll wheel and buttons, replicating its physical interfaces to a virtual version. The layout and menu shown here is defined by programming (functional data).

This is a web view of the configurable items exposed by the control unit UVR1611, as seen via CMI’s web interface.  The language of the web interface itself can be changed but the menu of the control unit depends on the operating system of the device (DE).

UVR1611 welcome screen, as seen via CMI.

The custom welcome screen (also shown in the photo above), as ‘forwarded’ via CMI’s web interface. The highlighted ‘DE’ indicates German firmware.

BL-NET basically does the same: it also ‘forwards the hardware interface to a web page.

Managing UVR1611 via BL-NET – same ‘MENUE’ as available on the physical device or via CMI. Here also the navigation of the web interface itself (left pane) is language-dependent, as tied to the device’s firmware.

Reader’s question: When you click BL-NET’s icon on the CMI website, you just see an error – why? It is expected as BL-NET operates at the same logical level as CMI, and thus cannot be managed via CMI (and BL-NET’s firmware predates the release of CMI).

Result of clicking the BL-NET icon in CMI’s display of the devices on the CAN bus.

3) Storing the logged data. In contrast to BL-NET and its scarce storage CMI’s storage card often does not need to be cleared often. We log data every 1,5 minutes, in total a few MB every month. An SD card with up to 32 GB capacity could be used, capable of holding several years of logging data.

Log files can be downloaded / ‘dragged’ from the SD card – but these files are not readable text files. To get CSV text files you would use Technische Alternative’s software Winsol, a Windows software, not a web application.The Winsol PC can communicate with CMI on the local network and having installed most recent firmware, also with other users’ devices via the cloud portal. But the software can also interpret data gathered from other loggers, e.g. files sent by clients.

Winsol: Heat pump operations, temperatures of heating water and brine

Screenshot of Winsol’s display of logged data: a custom view of temperatures of the heating water (curves in the middle 30°C to 50°C, and temperature of brine at different points (bottom curves, below 0°C. Zooming in on an interesting part of the curves is done by selecting a rectangular area anywhere in the plot with the mouse.

We use Winsol for digging into the data to spot glitches and evaluate heating systems’ performance – for optimization. Using Winsol and logfiles ‘sent’ by whatever transmission method will always work, no matter which logger a client uses, how their firewall is configured, or if they use the cloud portal.

The ‘logging architecture’ was the same with BL-NET, but from checking the networking traffic between the Winsol PC and the logger I conclude that the communication protocol was different. CMI now seems to use more straight-forward HTTP calls.

4) Providing visualization of the data measured right now. In contrast to BL-NET, you cannot show your system to anonymous visitors on the internet. Viewers need to register with Technische Alternative’s online (‘cloud’) portal and be given Guest access. With BL-NET system owners forwarded port 80 at their local firewalls and kept the Guest User’s password blank. Perhaps not always on purpose as the same was often true for the Expert’s password. Theoretically, you can still do this with CMI but I would not recommend it as the port for web access is now the same port as for fetching the log files.

Hydraulic Schema, as displayed by CMI

Hydraulic schema with dynamic values, as displayed by CMI to 1) local network users and 2) cloud users given Guest access. The green numbers are the current sensor values – a subset of all columns in the log file. CMI’s web server allows for creating different pages, and versions for different languages.

The software TA Designer creates the web view based on an image file of the hydraulic layout, and on a list of sensors and controlled devices read from functional data:

Hydraulic schematic for CMI in TA Designer

First steps when creating a dynamic visualization: You need to provide a  drawing of your hydraulic layout. Status and readings of sensors, valves,, pumps etc. can be dragged in the right place from the ‘tree’ in the left pane – which has been created from the imported functional data file.

What web developers like to add or improve is related to the last two points: Logging data into a database directly, and providing a custom web interface – with the option to give anonymous users view-schematic-only access.

Recent questions:

  • Is there a (standardized, XML-based) web service I can use to poll the data?
  • (Why) do I need an additional box like BL-NET?
  • You stated you log and analyze your data on your local network – how do you do it?

No, there is no web service. But I have been pointed to this open source web application: UVR1611 Data Logger Pro. Data Logger Pro uses the same port as Winsol to talk to the BL-NET (40000), so the same protocol. Data are polled and stored in a MySQL database – working around BL-NET’s limited storage capacity. You still need the logger hardware, as data gathered from communicating over the CAN bus have to be converted. In this case BL-NET operates as a CAN-Ethernet gateway only.

If you google for UVR1611 Data Logger Pro, you will find lots of websites on the internet: They all use nice domain names, like heater.surname.com, so I suppose these are accessible on purpose.

This solution does not yet work with CMI due to the different communication protocol. But somebody might work on this already, so this information might be outdated soon.

Update, autumn 2015: CMI and UVR16x2 are now supported by UVR Data Logger Pro.

We also use our own database (Microsoft SQL Server), but we create it from the CSV files exported with Winsol.

SQL scripts import data from the CSV export files created with Winsol to a database. Custom views are used to consolidate data (daily, monthly, per season), and to merge them with data measured manually about every day.

Since 2012, we have added sensors, and we calculated new parameters from these sensors’ readings. Sometimes you need to exclude non-meaningful sensor values from calculations, e.g. when the tank is drained or changes are made to the collector.  The custom SQL application keeps track of different calculations to be applied to different periods.

Recently I have also developed an Excel application – to calculate the most important performance parameters only, directly from a bunch of CSV files. The latter is surprisingly performant if you resist the temptation to mix VBA and those really huge spreadsheet formulas.

All the plots I had inserted into blog posts or into our PDF summary of key data had been created with Excel – as a frontend to SQL Server. For the Ice Storage Challenge plot, we picked the columns with daily averages of temperatures and the volume of ice as calculated from the increase in water level:

Volume of ice in the water tank over time, 2015-03-06

A plot created from our database of measured sensor values. Excel connects to SQL server – to a view with daily averages and lots of calculated values, such as the volume of ice.

___________________________

Re UVR1611’s successor, UVR16x2: We have it installed, but we are waiting for the firmware update that will allow logging via CMI.

9 thoughts on “Data Logging with UVR1611 – FAQ

  1. Pingback: Hacking My Heat Pump – Part 1: CAN Bus Testing with UVR1611 | elkemental Force

  2. Pingback: Watching TV Is Dangerous | Theory and Practice of Trying to Combine Just Anything

  3. Fascinating. Until now I hadn’t realized–though I should have–the extent to which you have utilized sensing and control in the execution of your plans. It’s amazing just the extent to which this can be utilized. Perhaps, for example, instead of responding to existing conditions, say from the temperature sensor, you could access predictive data, say from the federal weather service and even think of ways you could enact changes in advance. Mind is temporarily boggled 🙂

    • Yes, the control concept and programming the control unit is one of our major deliveries in a heat pump project! Since the combination of heat sources in our system is ‘non-standard’ we need to control it using a programmable system – so we recommend the most stupid commercial heat pump, one without its own extensive control unit which is actually a blackbox that can only be supported by the vendor. Our clients have full access to the control logic we provide. In addition we can integrate additing existing systems (such as flat plate solar collectors).

      Taking into account weather forecast is a feature management systems for photovoltaic generators have. Battery systems are still not economical today so you either have a rather small battery or none and you might want to optimize the amount of energy you can consume yourself by shifting loads (as the infamous washing machine although I don’t like that example). In Germany grid operators can also limit the power fed in by solar panels at noon to 70% to avoid instabilities for the same reasons as why they had been concerned about the eclipse yesterday. If your battery is full at noon and you cannot use all your power yourself that energy you cannot feed it into the grid either. The logic based on weather data (and on your typical usage pattern in the past) would try to make sure in advance that peak power would be consumed by appliances whose loads can be shifted (in the ‘worst case’ you might even use that power for heating a hot water tank) or that you battery is empty again at noon.

      But in contrast to electrical power generated by PV we can store the heat from ambient air easily, so we don’t take into account weather data but: heating demands, the ambient temperature, the tank temperature. If the collector is too cold, brine only circulates through the tank. If the ambient air is high enough, brine pump will be switched on to harvest energy, also when the heat pump is not on. Then the tank must not get too hot for heat pump operations. For cooling in summer, the collector can be switched in in cold nights to cool the tank. … The figure of the programming interface shows only part of the control logic – so even without weather data it is quite complex…. and often done wrong as we conclude from reviewing other malconfigured systems 😉

      • we had a graphical interface with logical gates and blackboxes to be filled in …
        I used turbopascal in the late eighties for I don’t remember what …too busy with photography these days to get my hands dirty on microcontrollers .. . But I still read hackaday.com … 🙂

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s