Half a Year of Solar Power and Smart Metering

Our PV generator and new metering setup is now operational for half a year; this is my next wall of figures. For the first time I am combining data from all our loggers (PV inverter, smart meter for consumption, and heat pump system’s monitoring), and I give a summary on our scrutinizing the building’s electrical power base load.

For comparison: These are data for Eastern Austria (in sunny Burgenland). Our PV generator has 4.77kWp, 10 panels oriented south-east and 8 south-west. Typical yearly energy production in our place, about 48° latitude: ~ 5.300 kWh. In the first 6 months – May to November 2015 – we harvested about 4.000kWh.
Our house (private home and office) meets the statistical average of an Austrian private home, that is about 3.500 kWh/year for appliances (excl. heating, and cooling is negligible here). We heat with a heat pump and need about 7.200kWh electrical energy per year in total.

In the following plots daily and monthly energy balances are presented in three ways:

  1. Total consumption of the building as the sum of the PV energy used immediately, and the energy from the utility.
  2. The same total consumption as the sum of the heat pump compressor’s input energy and the remaining energy for appliances, computers, control etc.
  3. Total energy generated by PV panels as the sum of energy used (same amount as contributions to 1) and the energy sold to the utility.

Monthly energy balances: PV generation and consumption, May-Nov 2015

Monthly electrical energy consumption - heat pump and appliances, May-Nov 2015

In summer there is more PV  energy available than needed and – even with a battery – the rest would needed to be fed into the grid. In October, heating season starts and more energy is needed by the heat pump that can be provided by solar energy.

This is maybe demonstrated best by comparing the self-sufficiency quota (ratio of PV energy and energy consumed) and the self-consumption quota (ratio of PV energy consumed and PV production). Numbers ‘flip’ in October:

pv-self-sufficiency-self-consumption-may-nov-2015

In November we had some unusually hot record-breaking days while the weather became more typical at the end of the month:

air-temperature-max-min-avg-nov-2015

This is reflected in energy consumption: November 10 was nearly like a summer day, when the heat pump only had to heat hot water, but on the colder day it needed about 20kWh (resulting in 80-100kWh heating energy).

Daily energy balances: PV generation and consumption, Nov 2015

Daily electrical energy consumption - heat pump and appliances, Nov 2015

In July, we had the chance to measure what the building without life-forms needs per day – the absolute minimum baseline. On July 10, 11, and 12 we were away and about 4kWh were consumed per day160W on average.

Daily energy balances: PV generation and consumption, July 2015

Note that the 4kWh baseline is 2-3 times the energy the heat pump’s compressor needs for hot water heating every day:

Daily electrical energy consumption - heat pump and appliances, July 2015

We catalogued all devices, googled for data sheets and measured power consumption, flipped a lot of switches, and watched the smart meter tracking the current consumption of each device.

smart-meter-office-evening

Consumption minus production: Current values when I started to write this post, the sun was about to set. In order to measure the consumption of individual devices they have been switched an of off one after the other, after sunset.

We abandoned some gadgets and re-considered usage. But in this post I want to focus on the base load only – on all devices that contribute to the 160W baseline.

As we know from quantum physics, the observing changes the result of the measurement. It was not a surprise that the devices used for measuring, monitoring and metering plus required IT infrastructure make up the main part of the base load.

Control & IT base load – 79W

  • Network infrastructure, telephone, and data loggers – 35W: Internet provider’s DSL modem / router, our router + WLAN access point, switch, ISDN phone network termination, data loggers / ethernet gateways for our control unit, Uninterruptible Power Supply (UPS).
  • Control and monitoring unit for the heat pump system, controlling various valves and pumps: 12W.
  • The heat pump’s internal control: 10W
  • Three different power meters: 22W: 1) Siemens smart meter of the utility, 2) our own smart meter with data logger and WLAN, 3) dumb meter for overall electrical input energy of the heat pump (compressor plus auxiliary energy). The latter needs 8W despite its dumbness.

Other household base load – 39W

Electrical toothbrush, at least no bluetooth.

  • Unobtrusive small gadgets – 12W: Electrical toothbrush, motion detectors, door bell, water softener, that obnoxious clock at the stove which is always wrong and can’t be turned off either, standby energy of microwave oven and of the PV generator’s inverter.
  • Refrigerator – 27W: 0,65 kWh per day.

Non-essential IT networking infrastructure – 10W

  • WLAN access point and router for the base floor – for connecting the PV inverter and the smart meter and providing WLAN to all rooms.

These are not required 24/7; you don’t lose data by turning them off. Remembering to turn off daily might be a challenge:

Boldly going where no one has gone before!

Non-24/7 office devices – 21W. Now turned off with a flip switch every evening, and only turned on when needed.

  • Phones and headsets: 9W.
  • Scanner/Printer/Fax: 8W. Surprisingly, there was no difference between ‘standby’ and ‘turned off’ using the soft button – it always needs 8W unless you really disconnect it.
  • Server in hibernated state 4W. Note that it took a small hack of the operating system already to hibernate the server operating system at all. Years ago the server was on 24/7 and its energy consumption amounted to 500kWh a year.

Stuff retired after this ‘project’ – 16W.

  • Radio alarm clock – 5W. Most useless consumption of energy ever. But this post is not meant as bragging about the smartest use of energy ever, but about providing a realistic account backed up by data.
  • Test and backup devices – 7W. Backup notebooks, charging all the time, backup router for playground subnet not really required 24/7, timer switch most likely needing more energy than it saved by switching something else.
  • Second old Uninterruptable Power Supply – 4W. used for one connected device only, in addition to the main one. It was purchased in the last century when peculiarities of the local power grid had rebooted  computers every day at 4:00 PM.

Historical UPS from last century

In total, we were able to reduce the base load by about 40W, 25% of the original value. This does not sound much – it is equivalent to a small light bulb. But on the other hand, it amounts to 350kWh / year, that is 10% of the yearly energy consumption!

___________________________

Logging setup:

  • Temperature / compressor’s electrical power: Universal control UVR1611 and C.M.I. as data logger, logging interval 90 seconds. Temperature sensor: PT1000. Power meter:  CAN Energy Meter. Log files are exported daily to CSV files using Winsol. Logging interval: 90 seconds.
  • PV output power: Datamanager 2.0 included with PV inverter Fronius Symo 4.5-3-M, logging interval 5 minutes.
  • Consumed energy: Smart meter EM-210, logging interval 15 minutes.
  • CSV log files are imported into Microsoft SQL Server 2014 for analysis and consolidation. Plots are created with Microsoft Excel as front end to SQL Server, from daily and monthly views on joined UVR1611 / Fronius Symo / EM-210 tables.

The Impact of Ambient Temperature on the Output Power of Solar Panels

I have noticed the impact of traversing clouds on solar power output: Immediately after a cloud has passed, power surges to a record value. This can be attributed to the focusing effect of the surrounding clouds and/or cooling of the panels. Comparing data for cloudless days in May and June, I noticed a degradation of power – most likely due to higher ambient temperatures in June.

We had a record-breaking summer here; so I wondered if I could prove this effect, using data taken at extremely hot days. There is no sensor on the roof to measure temperature and radiation directly at the panels, but we take data taken every 90 seconds for:

  • Ambient air temperature
  • Global radiation on a vertical plane, at the position of the solar thermal collector used with the heat pump system.

I was looking for the following:

  • Two (nearly) cloudless days, in order to rule out the impact of shadowing at different times of the days.
  • These days should not be separated by too many other days, to rule out the effect of the changing daily path of the sun.
  • Ideally, air temperature should be very different on these days but global radiation should be the the same.

I found such days: August 1 and August 12 2015:

Daily PV ouput energies and ambient temperatures in August 2015

Daily output of the photovoltaics generator (4,77 kW peak), compared to average and maximum air temperatures and to the global radiation on a vertical plane. Dotted vertical lines indicate three days nearly without clouds.

August 12 was  a record-breaking day with a maximum temperature of 39,5°C. August 1 was one of the ‘cool’ but still perfectly sunny days in August. The ‘cold day’ resulted in a much higher PV output, despite similar input in terms of radiation. For cross-checking I have also included August 30: Still fairly hot, but showing a rather high PV output, at a slightly higher input energy.

August 2015 in detail:

Daily PV ouput energies and ambient temperatures in August 2015 - details

Same data as previous plot, zoomed in on August. Dotted lines indicate the days compared in more detail.

Overlaying the detailed curves for temperature and power output over time for the three interesting days:

PV power and ambient temperature over time

Detailed logging of ambient air temperature and output power of the photovoltaic generator on three nearly cloudless days in August 2015.

The three curves are stacked ‘in reverse order’:

The higher the ambient air temperature, the lower the output power.

Note that the effect of temperature can more than compensate for the actually higher radiation for the middle curve (August 30).

I have used global radiation on a vertical plane as an indicator of radiation, not claiming that it is related to the radiation that would be measured on the roof – or on a horizontal plane, as it is usually done – in a simple way. We measure radiation at the position of our ribbed pipe collector that serves as a heat source for the heat pump; it is oriented vertically so that it resembles the orientation of that collector and allows us for using these data as input for our simulations of the performance of the heat pump system.

Our house casts a shadow on the solar collector and this sensor on the afternoon; therefore data show a cut-off in the afternoon:

Global radiation on solar collector, vertical plane, August 2015

Global radiation in W per square meter on a vertical plane, measured at the position of the solar collector. The collector is installed on the ground, fence-like, behind the house, about north-east of it.

Yet, if you compare two cloudless days where the sun traversed about the same path (thus days close in the calendar) you can conclude that solar radiation everywhere – including the position on the roof – was the same if these oddly shaped curves are alike.

This plot shows that the curves for these two days that differed a lot in output and temperature, August 1 and 12, were really similar. Actually, the cooler day with higher PV output, August 1, even showed the lower solar radiation due to some spikes. Since the PV inverter only logs every 5 minutes whereas our system’s monitoring logs every 1,5 minutes those spikes might have been averaged out in the PV power curves. August 30 clearly showed higher radiation which can account for the higher output energy. But – as shown above – the higher solar power could not compensate for the higher ambient temperature.

___________________________

Logging setup:

Heat Pump System Data: Three Seasons 2012 – 2015

We have updated the documentation of monthly and seasonal measurement data – now including also the full season September 2014 to August 2015.

The overall Seasonal Performance Factor was 4,4 – despite the slightly lower numbers in February and March, when was the solar collector was off during the Ice Storage Challenge.

Edit: I have learned from a question that the SPF is also calculated in BTU/Wh. ‘Our’ SPF uses the same units in nominator and denominator, so 4,4 is in Wh/Wh. The conversion factor is about 3,4 (note that I use a decimal comma BTW), so our SPF [kWh/kWh] is equivalent to an SPF [BTU/Wh] ~ 15.

Monthly Performance Factor, Heat Pump System

Monthly heating energy provided by the heat pump – total of both space heating and hot water water, related electrical input energy, and the ratio = monthly performance factor. The SPF is in kWh/kWh.

The SPF determines economics of heating with a heat pump.

It’s time to compare costs again, based on current minimum prices of electricity and natural gas in our region in Austria (published by regulator e-control):

  • We need about 20.000 kWh (*) of heating energy per year.
  • Assuming a nearly perfect gas boiler with an efficiency of 95%, we would need about 21.050 kWh of gas.
  • Cost of natural gas incl. taxes, grid fees: ~ 0,0600 € / kWh
  • Yearly energy costs for heating with gas would be: € 1.260
  • Given an SPF of 4,4 for the heat pump, 20.000 kWh heating energy demands translate to 4.545 kWh of electrical energy.
  • Costs of electricity incl. taxes, grid: ~ 0,167 € / kWh
  • Yearly energy costs for heating with the heat pump: € 760
  • Yearly savings with the heat pump: € 500 or 40% of the costs of gas.

(*) As indicated in the PDF, In the past year only the ground floor was heated by the heat pump. So we needed only 13.300 kWh. In the first floor we got rid of the remainders of the old roof truss. The season 2012/2013 was more typical, requiring about 19.700 kWh.

The last winter was not too extreme – we needed 100 kWh maximum heating energy per day. The collector was capable of harvesting about 50 kWh / day:

Daily energy balances, heat pump system, season 2014-2015

Daily energies: 1) Heating energy delivered by the heat pump. Heating energy = electrical energy + ambient energy from the tank. 2) Energy supplied by the collector to the water tank, turned off during the Ice Storage Challenge. Negative collector energies indicate cooling of the water tank by the collector during summer nights. 200 kWh peak in January: due to the warm winter storm ‘Felix’.

Ice formation in this season was mainly triggered by turning off the solar collector deliberately. As soon as we turn the collector on again in March the ice was melted quickly, and the temperature increased to the set value of 8°C – a value picked deliberately to prepare for cooling in summer:

Temperatures and ice formation, heat pump system, season 2014-2015

Daily averages of the air temperature and the temperature in the water tank plus volume of ice created by extracting heat from the heat source (water tank).

 

Having Survived the Hottest July Ever (Thanks, Natural Cooling!)

July 2015 was the hottest July ever since meteorological data had been recorded in Austria (since 248 years). We had more than 38°C ambient air temperature at some days; so finally a chance to stress-test our heat pump system’s cooling option.

Heating versus cooling mode

In space heating ‘winter’ mode, the heat pump extracts heat from the heat source – a combination of underground water / ice tank and unglazed solar collector – and heats the bulk volume of the buffer storage tank. We have two heating circuits exchanging heat with this tank – one for the classical old radiators in ground floor, and one for the floor heating loops in the first floor – our repurposed attic.

Space heating with solar collector on, heat pump system punktwissen,

Space heating mode: The heat pump (1) heats the buffer tank (7), which in turn heats the heating circuits (only one circuit shown, each has its circuit pump and mixer control). Heat source: Solar/air collector (4) and water / ice storage (3) connected in a single brine circuit. The heat exchanger in the tank is built from the same ribbed pipes as the solar collector. If the ambient temperature is too low too allow for harvesting of energy the 3-way valve (5) makes the brine flow bypass the collector.

The heat pump either heats the buffer tank for space heating, or the hygienic tank for hot tap water. (This posting has a plot with heating power versus time for both modes).

We heat hot tap water indirectly, using a hygienic storage tank with a large internal heat exchanger. Therefore we don’t need to fight legionella by heating to high temperatures, and we only need to heat the bulk volume of the tank to 50°C – which keeps the Coefficient of Performance high.

Heating hot water, solar collector off, heat pump system punktwissen

Hot tap water heating mode: The flow of water heated by the heat pump is diverted to the hygienic storage tank (6). Otherwise, the heat source is used in the same way as for space heating. In this picture, the collector is ‘turned off’ – corresponding to heating water on e.g. a very cold winter evening.

In summer, the still rather cold underground water tank can be used for cooling. Our floor heating loops become cooling loops and we simply use the cool water or ice in the underground tank for natural (‘passive’) cooling. So the heat pump can keep heating water – this is different from systems that turn an air-air heat pump into an air conditioner by reverting the cycle of the refrigerant.

Heating hot water in parallel to cooling is beneficial as the heat pump extracts heat from the underground tank and cools it further!

Space cooling while heating hot water, heat pump system punktwissen

Cooling mode: Via automated 3-way valve (9) brine is diverted to flow through the heat exchanger in the buffer tank (7). Water in the buffer tank is cooled down so water in the floor ‘heating’ / cooling loops. If the heat pump operates in parallel to heat hot tap water, it cools the brine.

How we optimize cooling power this summer

Water tank temperature. You could tweak the control to keep the large ice cube as long as possible, but there is a the trade-off: The cooler the tank,  the lower the heat pump’s performance factor in heating mode. This year we kept the tank at 8°C after ‘ice season’ as long as possible. To achieve this, the solar collector is bypassed if ambient temperature is ‘too high’. The temperature in the tank rose quickly in April – so our ice is long melted:

Temperatures and performance factors, July 2015

The red arrow indicates the end of the ice period; then the set temperature of the tank was 8°C (‘Ice storage tank’ is rather a common term denoting this type of heat source than indicating that it really contains ice all the time.) Green arrows indicate three spells of hot weather. The tank’s temperature increased gradually, being heating by the surrounding ground and by space cooling. At the beginning of August its temperature is close to 20°C, so cooling energy has nearly be used up completely.

At the beginning of July the minimum inlet temperature in the floor loops was 17°C, determined by the dew point (monitored by our control system that controls the mixer accordingly); at the end of the month maximum daily ambient air temperatures were greater than 35°C, and the cooling water had about 21°C.

Room temperature. Cooling was activated only if the room temperature in the 1st floor was higher than 24°C – this allows for keeping as much cooling energy as possible for the really hot periods. We feel that 25°C in the office is absolutely OK as temperatures outside are more then 10°C higher.

Scheduling hot water heating. After the installation of our PV panels we set the hot water heating time slots to periods with high solar radiation – when you have more than 2 kW output power on cloudless days. So we utilized the solar energy generator in the most economic way and the heat pump supports cooling exactly when cooling is needed.

Using the collector for cooling in the night. If the ambient temperature drops to a value lower than the tank temperature, the solar collector can actually cool the tank!

Ventilation. I have been asked if we have forced ventilation, ductwork, and automated awnings etc. No, we haven’t – we just open all the windows during the night and ‘manually operated’ shades attached to the outside of the windows. We call them the Deflector Shields:

Ventilation: Night

Manually operated ventilation – to be shut off at sunrise. We had already 30°C air temperature at 08:00 AM on some days.

Deflector shields: Day

South-east deflector shields down. We feel there is still enough light in the (single large) room as we only activate the subset of shields facing the sun directly.

These are details for two typical hot days in July:

Temperature and cooling power for two days in July

The blue line exhibits the cooling power measured for the brine ‘cooling’ circuit. If the heat pump is off, cooling power is about 1 kW; during heat pump operations (blue arrows) 4 kW cooling power can be obtained. Night-time ventilation is crucial to keep room temperatures at reasonable levels.

The cooling power is lower than so-called standard cooling load as defined in AC standards – the power required to keep the temperature at about 24°C in steady-state conditions, when ambient temperature would be 30°C and no shades are used. For our attic-office this standard cooling power would amount to more than 10 kW which is higher than the standard (worst case) heating load in winter.

Overall electrical energy balance

I have been asked for a comparison of the energy needed in the house, the heat pump in particular, and the energy delivered by the PV panels and fed in to the grid.

PV numbers in July were not much different from June’s – here is the overview on June and July, maximum PV power on cloudless days has decreased further due to the higher temperatures:

Daily energy balance, PV generation and self-consumption-2015-06-and-2015-07In July, our daily consumption slightly decreased to 9-10 kWh per day, the heat pump needs 1-2 kWh of that. The generator provides for 23 kWh per day,

Currently the weather forecast says, we will have more than 35°C each noon and 20-25° minimum in the night until end of this week. We might experience the utter depletion of our cooling energy storage before it will be replenished again on a rainy next weekend.

Solar Power: Some Data for the First Month.

On May 4, 2015, we started up our photovoltaic generator. Here are some numbers and plots for the first month – and what I plan to do next.

Our generator has a rated power of 4,77 kWp (kilowatt peak), one module has 265 Wp. The generator would deliver 4,77 kW of electrical power under so-called standard testing conditions: An irradiance of 1000 W/m2 of light from the sun, a module temperature of 25°C, and a standard spectrum of wavelengths determined by the thickness of the atmosphere light has to traverse (Air mass – AM 1,5, equivalent to sunlight hitting the earth at an angle of about 48° from the zenith).

Our 18 panels are mounted on two different roof areas, 10 of them (2,65 kWp) oriented south-east and 8 modules (2,12 kWp) south-west. The inclination relative to the surface of the earth is 30°, the optimum angle for PV at our latitude:

Plan of our house with PV modules.

Positions of our PV panels on the roof.

We aimed at using our 30° upper roof spaces most efficiently while staying below the ‘legal threshold’ of 5 kW, avoiding a more complicated procedure for obtaining a permit to install them.

The standard conditions are typically met in spring here – not in summer – as the efficiency of solar panels gets worse with increasing temperature: for our panels -0,44% of rated power per °C in temperature difference. If the temperature is 60°C, peak power (for otherwise same irradiance and spectrum) would drop by 15% . We can already see this effect, when comparing two nearly cloudless days in May and in June. The peak power is lower in the first days of June when maximum daily air temperatures were already about 30°C:

PV power over time, for a day in May versus a day in June

Total output power (AC) of the PV generator and input power (DC) for each string as a function of time for two days. 1) May 11 – maximum ambient air temperature 23°C. 2) June 5 – maximum ambient air temperature 30,5°C.

The temperature-dependence of performance might in part explain impressive spikes in power you see after clouds have passed: The modules have a chance to cool off, and immediately after the cloud has gone away the output power is then much higher than in case of constant irradiance. Here is a typical example of very volatile output:

PV power over time, data points taken every few seconds.

Output power of our PV generator when clouds are passing. The spikes (clear sky) show a peak power much higher than the constant value on a cloudless day in May; the troughs correspond to clouds shadowing the panels. The data logger included with the inverters only logs a data point every 5 minutes, so I parsed the inverter’s website instead to grab the current power displayed there every second (Using the inverter’s Modbus TCP interface would be the more professional solution, but parsing HTTP after reverse engineering the HTML structure is usually a quick and dirty ‘universal logging interface’.)

The maximum intermittent power here was about 4,4 kW!

Another explanation for the difference is local ‘focussing’ of radiation by specific configuration of clouds reflecting more radiation into one direction: Consider a cloudless region surrounded by clouds – a hole in the clouds so to speak. Then radiation from above might be reflected at the edges of that hole at a very shallow angle, so that at some place in the sunny spot below the power might be higher than if there were no clouds at all. Here is another article about this phenomenon.

A PV expert told me that awareness of this effect changed best practices for sizing the inverter: From using one with a maximum power about 20% lower than the generator’s power a few years ago (as you hardly ever reach the rated power level with constant radiation) to one with matches the PV peak output better.

The figures from May 11 and June 5  also show that the total power is distributed more evenly throughout the day as if we would have had a ‘perfect’ roof oriented to the south. In the latter case the total energy output in a year would be higher, but we would not be able to consume as much power directly. But every kWh we can use immediately is worth 3 times a kWh we have to sell to the utility.

The next step is to monitor the power we consume in the house with the same time resolution, in order to shift more loads to the sunny hours or to identify some suckers for energy. We use more than 7000 kWh per year; more than half of that is the heat pump’s input energy. Our remaining usage is below the statistical average in Austria (3700 kWh per 2-person household) as we already did some detective work to discover energy-sucking devices.

Smart meters are to be rolled out in Austria in the next years, by 2020 95% of utilities’ clients should be equipped with them. These devices measure energy consumption in 15-minute intervals; they send the data to the utility daily (which runs a web portal where clients can access their data) but must also have a local interface for real-time logging given to clients on request. As a freshly minted owner of a PV generator I got a new ‘smart’ meter by the utility; but this device is just a temporary solution, not connected to the utility’s central system. It will be replaced by a meter from another vendor in a few years. Actually, in the past years we could read off the old analogue Ferraris meter and submit the number at the utility’s website. This new dumb smart meter, in contrast, requires somebody to visit us and read off the stored data once a year again, using its infrared interface.

I did some research on all possible options we have to measure the power we consume, the winner was another smart meter plus integrated data logger and WLAN and LAN interfaces. It has been installed yesterday ‘behind’ the official meter:

Our power meters in the distribution cabinet

Our power distribution cabinet. The official (Siemens) smart meter is the rather large box to the left; our own smart meter with integrated data logger is is the small black one above it – the one with the wireless LAN antenna.

We will combine its data with the logging of ‘PV energy harvested’ provided by the inverter of the PV panels – an inverter we picked also because of the wealth of options and protocols for accessing it [*]

For the first month we can just have a look at daily energy balances from two perspectives (reading off the display of the dumb smart meter manually every day):

  1. The energy needed by appliances in the house and for hot water heating by the heat pump – 11 kWh per day: On average 56,5% in the first month come from the solar panels (self-sufficiency quota), and the rest was provided by the grid.
  2. The daily energy output of the solar generator was 23 kWh per day on average – either consumed in the house – this is the same cyan bar as in (1) – or fed into the grid. In this month we consumed 27% of the PV power directly (self-consumption quota).
Daily energy balance: 1) The energy we consume in the house - partly from PV, partly from the grid and 2) The energy harvested by the PV generator - party used directly, partly fed into the grid.

Daily energy balance: 1) The energy we consume in the house – partly from PV, partly from the grid (left axis) and 2) The energy harvested by the PV generator – party used directly, partly fed into the grid (right axis).

___________________________________

[*] For German-speaking readers: I wrote a summary about different solutions for metering and logging in this case in this German article called ‘The Art of Metering’ – options are to use the official meter’s IR interface with yet another monitoring ‘server’, your own unrelated meter (as we did), a smart meter integrated with the inverter and using the inverter’s own data logging capabilities), or building and programming your own smart meter from scratch.

Watching TV Is Dangerous

I am not talking about humans.

But TV-sets might threaten other devices in the smart home; this was a recent puzzle submitted by a blog reader.

Two unrelated devices / services met on the user’s local computer network:

  • IP-TV provided by a large German telco.
  • a data logger for monitoring the heating system.

This user had one of the solutions in place that I mentioned in my previous post on data logging: The logger BL-NET connects to the controller UVR1611 via CAN bus, and to the computer network via ethernet, and it acts as a ‘CAN-ethernet gateway’ to allow for logging data to database server on the network, hosting the application UVR Data Logger Pro.

Data Logger BL-NET, with ethernet, CAN, and USB interfaces (My attempt at 'organic tech product photography.)

Data Logger BL-NET, with ethernet, field bus, and USB interfaces (My attempt at ‘organic tech product photography’.)

The issue: Every time the user turned on the TV, BL-NET suddenly refused to work – communicating its predicament via a red LED. The IP-TV did not use up all the band width; so my suspicion was that the TV (or TV service) sends a network packet that the logger does not like; perhaps a special – sci-fi-like – unicast or broadcast message. Or any of the parties involved does not strictly comply with standards. Or standards might be ambiguous.

It would have been interesting to do network analysis and trace the network traffic and spot that packet of death.The BL-NET product had been superseded by its successor – called C.M.I. – Control and Monitoring Interface – which has better out-of-the box logging, cloud support etc.. The open source UVR Data Logger Pro does not yet speak CMI’s protocol so I understand that BL-NET users do not want to change their solution immediately. But it is unlikely that BL-NET will get firmware updates, and it is very unlikely that a large internet services provider will change its IP-TV communications protocol.

My suggestion was to shield the logger from packets sent by the TV – by tucking BL-NET away in its private subnet – using a spare internet router or the sniffer-router PC I had described in Network Sniffing for Everyone:

The ‘spare’ internet router was placed behind the main internet router, connecting its ‘WAN’ port to the main LAN, and BL-NET was connected to a LAN port of this second router.  If the router is a PC with sniffer software this configuration would also allow for researching the evil packet.

This did the trick – BL-NET did not die of TV’s packets anymore!

In order to avoid running yet another box consuming electrical power, one might also…

  • add another network interface to the the UVR Data Logger Pro database server and use this one as that router.
  • replace the internet router by one that can be configured for more than one virtual LAN (in case the current one does not have this option).
Bundesarchiv Bild 183-H0812-0031-001, Werbung, RFT Color 20, Fernseher

At that time, TV-sets did not yet jeopardize the integrity of other devices in the dumb home. Ad for color TV set, 1969. (Bundesarchiv – German national archive, Wikimedia)

Data Logging with UVR1611 – FAQ

I have received several questions related to my article on data logging on this blog, or to my postings on monitoring and control on our German blog.

Thus I have decided to write the article I would have wanted to read when I once made myself familiar with this. The target audience for this article are IT guys / web developers / ambitious DIY enthusiasts trying to make sense of the interfaces provided by the freely programmable controller UVR1611.

We use this device as the main controller of heat pump systems we design, and for monitoring and optimizing heating systems in general.

UVR1611, customized welcome screen

This control unit receives data from sensors (temperature, flow, irradiation,…) and controls pumps and valves accordingly.

You interact with the unit via programming it directly – using a scroll wheel and buttons, but this should only be used for changing parameters such as a temperature set point. The control logic should rather be developed with a graphical programming application, called TAPPS. This software creates the functional data which have to be uploaded to the controller.

Snapshot of a part of the control logic of our heat pump system, as designed with TAPPS.

Sensors and manageable devices as valves talk to the control unit via traditional field buses, such as CAN bus (e.g. also used in cars’ internal networks) and so-called DL Bus. In order to access UVR1611 via a standard TCP/IP computer network you need a kind of gateway. This device does not only convert the field bus communication, but also serves as a repository for the logged measurement data. 

In our control network there are two different kinds of loggers, for ‘research purposes’. According to the vendor Technische Alternative GmbH, using two loggers in parallel is not supported and discouraged at is might cause issues. For us it works fine, but only try at your own risk:

Data loggers CMI and BL-NET

Data loggers by Technische Alternative: CMI (Control and Monitoring Interface) to the left, BL-NET (Boot Loader)  in the middle, standard ethernet switch to the right. The CAN bus cable is connected to both of them via the blue connector.

Two loggers – CMI and BL-NET – are connected to UVR1611 via CAN bus – a linear bus that needs to be terminated on both ends. Each of the devices is connected to the local computer network via standard ethernet wiring.

CMI is BL-NET’s successor although there might be no immediate reason to upgrade. Starting from scratch now, I would recommend CMI though.

This is how our local CAN bus looks like now, as displayed in CMI’s web interface:

Devices on CAN bus, displayed by CMI

Devices on CAN bus, displayed by CMI: Loggers CMI and BL-NET, plus and energy counter (CAN-EZ) and an extension of inputs/outputs (CAN-IO), UVR1611’s successor UVR16x2.

So there is a web server on CMI, which can be accessed locally. As described in the previous article, you can also access it via a ‘cloud-based’ portal.

In summary, this logger / gateway allows for the following:

1) Uploading functional data (programming logic) to UVR1611, by uploading the file from  computer onto the SD card inserted into the device and then dragging the file to the control unit’s icon in this web interface. This is an improvement over BL-NET which required an additional software application called Memory Manager to transfer functional data to the logger first. Existing functional data can be downloaded and inspected in the recent versions of TAPPS.

2) Accessing the control unit as if you would use the scroll wheel and buttons, replicating its physical interfaces to a virtual version. The layout and menu is defined by programming (functional data).

This is a web view of the configurable items exposed by the control unit UVR1611, as seen via CMI’s web interface.  The language of the web interface itself can be changed but the menu of the control unit depends on the operating system of the device (DE).

UVR1611 welcome screen, as seen via CMI.

The custom welcome screen (also shown in the photo above), as ‘forwarded’ via CMI’s web interface. The highlighted ‘DE’ indicates German firmware.

BL-NET basically does the same: it also ‘forwards the hardware interface to a web page.

Managing UVR1611 via BL-NET – same ‘MENUE’ as available on the physical device or via CMI. Here also the navigation of the web interface itself (left pane) is language-dependent, as tied to the device’s firmware.

Reader’s question: When you click BL-NET’s icon on the CMI website, you just see an error – why? It is expected as BL-NET operates at the same logical level as CMI, and thus cannot be managed via CMI (and BL-NET’s firmware predates the release of CMI).

Result of clicking the BL-NET icon in CMI’s display of the devices on the CAN bus.

3) Storing the logged data. In contrast to BL-NET and its scarce storage CMI’s storage card often does not need to be cleared often. We log data every 1,5 minutes, in total a few MB every month. An SD card with up to 32 GB capacity could be used, capable of holding several years of logging data.

Log files can be downloaded / ‘dragged’ from the SD card – but these files are not readable text files. To get CSV text files you would use Technische Alternative’s software Winsol, a Windows software, not a web application.The Winsol PC can communicate with CMI on the local network and having installed most recent firmware, also with other users’ devices via the cloud portal. But the software can also interpret data gathered from other loggers, e.g. files sent by clients.

Winsol: Heat pump operations, temperatures of heating water and brine

Screenshot of Winsol’s display of logged data: a custom view of temperatures of the heating water (curves in the middle 30°C to 50°C, and temperature of brine at different points (bottom curves, below 0°C. Zooming in on an interesting part of the curves is done by selecting a rectangular area anywhere in the plot with the mouse.

We use Winsol for digging into the data to spot glitches and evaluate heating systems’ performance – for optimization. Using Winsol and logfiles ‘sent’ by whatever transmission method will always work, no matter which logger a client uses, how their firewall is configured, or if they use the cloud portal.

The ‘logging architecture’ was the same for BL-NET, but from checking the networking traffic between the Winsol PC and the logger I conclude that the communication protocol was different. CMI now seems to use more straight-forward HTTP calls.

4) Providing visualization of the data measured right now. In contrast to BL-NET, you cannot show your system to anonymous visitors on the internet. Viewers need to register with Technische Alternative’s online (‘cloud’) portal and be given Guest access. With BL-NET system owners forwarded port 80 at their local firewalls and kept the Guest User’s password blank. Perhaps not always on purpose as the same was often true for the Expert’s password. Theoretically, you can still do this with CMI but I would not recommend it as the port for web access is now the same port as for fetching the log files.

Hydraulic Schema, as displayed by CMI

Hydraulic schema with dynamic values, as displayed by CMI to 1) local network users and 2) cloud users given Guest access. The green numbers are the current sensor values – a subset of all columns in the log file. CMI’s web server allows for creating different pages, and versions for different languages.

The software TA Designer creates the web view based on an image file of the hydraulic layout, and on a list of sensors and controlled devices read from functional data:

Hydraulic schematic for CMI in TA Designer

First steps when creating a dynamic visualization: You need to provide a  drawing of your hydraulic layout. Status and readings of sensors, valves,, pumps etc. can be dragged in the right place from the ‘tree’ in the left pane – which has been created from the imported functional data file.

What web developers like to add or improve is related to the last two points: Logging data into a database directly, and providing a custom web interface – with the option to give anonymous users view-schematic-only access.

Recent questions:

  • Is there a (standardized, XML-based) web service I can use to poll the data?
  • (Why) do I need an additional box like BL-NET?
  • You stated you log and analyze your data on your local network – how do you do it?

No, there is no web service. But I have been pointed to this open source web application: UVR1611 Data Logger Pro. Data Logger Pro uses the same port as Winsol to talk to the BL-NET (40000), so the same protocol. Data are polled and stored in a MySQL database – working around BL-NET’s limited storage capacity. You still need the logger hardware, as data gathered from communicating over the CAN bus have to be converted. In this case BL-NET operates as a CAN-Ethernet gateway only.

If you google for UVR1611 Data Logger Pro, you will find lots of websites on the internet: They all use nice domain names, like heater.surname.com, so I suppose these are accessible on purpose.

This solution does not yet work with CMI due to the different communication protocol. But somebody might work on this already, so this information might be outdated soon.

Update, autumn 2015: CMI and UVR16x2 are now supported by UVR Data Logger Pro.

We also use our own database (Microsoft SQL Server), but we create it from the CSV files exported with Winsol.

SQL scripts import data from the CSV export files created with Winsol to a database. Custom views are used to consolidate data (daily, monthly, per season), and to merge them with data measured manually about every day.

Since 2012, we have added sensors, and we calculate new key parameters from these sensors’ readings. Sometimes you need to exclude non-meaningful sensor values from calculations, e.g. when the tank is drained or changes are made to the collector.  The custom SQL application keeps track of different calculations to be applied to different periods.

Recently I have also developed an Excel application – to calculate the most important performance parameters only, directly from a bunch of CSV files. The latter is surprisingly performant if you resist the temptation to mix VBA and those really huge spreadsheet formulas.

All the plots I had inserted into blog posts or into our PDF summary of key data had been created with Excel – as a frontend to SQL Server. For the Ice Storage Challenge plot, we picked the columns with daily averages of temperatures and the volume of ice as calculated from the increase in water level:

Volume of ice in the water tank over time, 2015-03-06

A plot created from our database of measured sensor values. Excel connects to SQL server – to a view with daily averages and lots of calculated values, such as the volume of ice.

___________________________

Re UVR1611’s successor, UVR16x2: We have it installed, but we are waiting for the firmware update that will allow logging via CMI.

Google and Heating Systems (2)

I googled our company name. Then I found this:

What should not be online

Auftrag means order and the obfuscated parts contain our full company name, the Chief Engineer’s name, the URL of a vendor we ordered material from recently, invoice total, and a comment like The client said we should…

The now inaccessible URL had pointed to a comma-separated text related to statistics for orders. Obviously they had put company-internal data on an internet-facing system without knowing it. If you are familiar with the details of the URL and keywords you can actively search for such systems on the internet.

This is in essence what Google Hacking is about – here is a detailed manual, a presentation from a security conference. The infamous list of orders is used as a prime example on p.10.

If you wonder why this is called Google and Heating (2). This was on Google and heating, too, though there is not much relation to the topics covered.

Search engine Shodan takes this a step further: It allows for searching specifically for devices who are listening for incoming connections on the internet. Analyzing the standardized headers of the responses tells you if this is a traffic light, web cam, an internet router … or some home owner’s heating system.

These are search results for ADSL modems used by a large telco.

shodan-search-resultThose devices have a web server listening on HTTP. Not necessarily an issue if passwords have been set, there are no known vulnerabilities, and in case there is those systems are updated. As an end user you would not have a chance to interfere here as the modems are managed by the provider.

But it definitely should not look like this.

This is the passwords page of of data logger (BL-NET by Technische Alternative) for a heater accessible via the internet, showing that none of the passwords for guests, normal and expert user had been set. You could maliciously change control parameters or set passwords and lock the owner out.

But in contrast to a provider’s modem you need to take action to make such loggers and their web interfaces available on the internet. Vulnerabilities aside, any typical internet router (a device doing Network Address Translation) does not allow unsolicited incoming connections from the the internet to a device on the local network, that is behind the provider’s access device and/or your router. Only traffic recognized as the response to an outgoing request, such as browsing a public web pages, will be relayed by the router. In order to show off your heater’s performance to your friend you need to open up your router’s firewall and configure a rule for so-called port forwarding.

The problem with this approach is that some people don’t know exactly what they are doing (see inquiries via forums along the lines: I have no idea at all what VPN, TCP/IP, ports, DNS etc. means – but could you explain me briefly how to access my heater from the internet?), and there might be lots of running systems never touched again, once configured by the computer-savvy friend.

Then there might be hidden risks related to undetected vulnerabilities in the embedded web servers used. A German vendor of heating systems had caused a stir last year: Their clients’ systems had been accessible from the internet via port-forwarding. Their naming conventions for the dyndns names of such hosts could easily be guess – so attackers could find the systems. Passwords have been set; but sending a specifically crafted URL to the device you could force the web server to respond with the list of all passwords in clear text. The vendor reacted quickly and referred the issue to the supplier of the underlying control software – which was used with larger and more critical systems and residential heating. It turned out that the software vendor had never recommended to use the system in that way – only protected by passwords, but a VPN tunnel should be provided instead – wrapping the insecure traffic within a channel equipped with stronger protection. Adding a VPN is a major change and required the installation of a new physical module at clients’ site.

Apart from opening up your network up to the internet or VPNs there is another class of solutions to the Internet of Things issue: Things may actively connect to a server on the internet, and this server will relay or mediate the connection. I have written about Things  ‘phoning home’ and how to sniff the traffic before, and I add some more links at the end of this post. If the owner of the thing is given some control over the communication I still think it is the best option.

We now use such a Thing as our latest data logger for our heat pump system.

That’s the Thing – C.M.I., Control and Monitoring Interface – here is my failed attempt at innovative tech product photography:

(The usual disclaimer: I don’t make money from reselling or recommending products, I just like them. Vendors beware, I might change my mind anytime.)

It does not get better if I try to capture The Things in their natural habitats – CMI to the left, BL-NET in the middle, and a simple ethernet switch to the right.

CMI and BL-NEZ data loggers, by Technische Alternative

This is the ‘data center’. The control system (UVR1611) is in the ‘boiler room’, connected via CAN bus (blue connectors) to both loggers. We operate them in parallel, on the same CAN bus – for ‘research purposes’ and fun, though this is discouraged by Technische Alternative. Both loggers are connected to the local network.

We haven’t opened our firewall for BL-NET but CMI is allowed to make an outbound connection to the vendor’s portal https://cmi.ta.co.at/. You are required to create a user at this portal (that is running on amazon’s cloud BTW), and associate your CMI’s unique serial number and key with your user online. Other portal users may be given permission to view or manage your device – this is how we do online support of clients’ devices. It is not possible to allow anonymous users to view your current data and hydraulic layout.

The CMI is keeping a permanent outbound connection to the portal server who relays ‘incoming’ requests that technically aren’t incoming.

What I find important is:

You can access the device locally and directly, too. All your logged data are stored on an SD card – the slot and the blue card are visible in the photos. You can turn off the device’s connection to the portal and perhaps only turn it on if you required support.

The networking settings are similar to that of any computer on the local network. Turning off the portal is equivalent to not running Teamviewer, VNC, or similar remote support tools.

CMI settings, turn off connection to online portal.Unfortunately this cannot be said for any appliance that sends data to a portal. Actually, this article had in part been triggered by my researching the data logging capabilities of inverters of photovoltaic generators. Some of those send data to their clouds while giving the user no local access to the data at all.

Ambitious users build tools (e.g. running on Raspberry Pi) that intercept and store the traffic that was intended for the portal. A user reported that his battery did not work for weeks after the inverter vendor had upgraded the firmware. The new firmware used different temperature thresholds when determining if the battery was operating normally – and decided that the battery was much too cold. It took some time to persuade the vendor to restore the previous version of the firmware.

Remote firmware upgrade is subject to heated discussions, and can cause legal issues. Vendors of smart meters have to to separate the software that is required for ‘features’ – to be upgraded later, following ever changing standards and advances in technology – and the software associated with the data used in billing – subject to official calibration.

In case the vendor of the modems shown in the Shodan screenshot detects a vulnerability we would probably happy if they patch it immediately. Our favorite Things can be updated automatically and it went well so far.

____________________________________________________

Further reading:

Security Statement for Teamviewer – which also happens to be the software I am using for remote connections to clients’ computer systems and for remote meetings.

The Internet of Things, and how those Things phone home. An accessible and brief explanation of the different ways things allow for connections leveraged by a server on the internet.

Peer to Peer – Hole Punching – more detailed explanations.

Peer-to-Peer Communication Across Network Address Translators – even more detailed explanations, similar to this RFC by the same authors.