Spheres in a Space with Trillions of Dimensions

I don’t venture into speculative science writing – this is just about classical statistical mechanics; actually about a special mathematical aspect. It was one of the things I found particularly intriguing in my first encounters with statistical mechanics and thermodynamics a long time ago – a curious feature of volumes.

I was mulling upon how to ‘motivate’ the calculation below in a really comprehensible way, but I am afraid I can’t. When introducing the ‘kinetic theory’ in thermodynamics often the pressure of an ideal gas is calculated first, by considering averages over momenta transferred from particles hitting the wall of a container. This is rather easy to understand but still sort of an intermediate view – between phenomenological thermodynamics that does not explain the microscopic origin of properties like energy, and ‘true’ statistical mechanics. The latter makes use of a phase space with with dimensions the number of particles. One cubic meter of gas contains ~1025 molecules. Each possible state of the system is depicted as a point in so-called phase space: A point in this abstract space represents one possible system state. For each (point-like) particle 6 numbers are added to a gigantic vector – 3 for its position and 3 for its momentum (mass times velocity), so the space has ~6 x 1025 dimensions. Years ago I tried to explain this with illustrations and metaphors (Here and here). Thermodynamic properties are averages taken over the state of one system watched for a long time or over a lot of ‘comparable’ systems starting from different initial conditions. At the heart of statistical mechanics are distributions functions that describe how a set of systems described by such gigantic vectors evolves. This function is like a density of an incompressible fluid in hydrodynamics. I resorted to using the metaphor of a jelly in hyperspace before.

Taking averages means to multiply the ‘mechanical’ property by the density function and integrate it over the space where these functions live. The volume of interest is a  generalized N-ball defined as the volume within a generalized sphere. A ‘sphere’ is the surface of all points in a certain distance (‘radius’ R) from an origin

x_1^2 + x_2^2 + ... + x_ {N}^2 = R^2

(x_n being the co-ordinates in phase space and assuming that all co-ordinates of the origin are zero). Why a sphere? Because states are ordered or defined by energy, and larger energy means a greater ‘radius’ in phase space. It’s all about rounded surfaces enclosing each other. The simplest example for this is the ellipse of the phase diagram of the harmonic oscillator – more energy means a larger amplitude and a larger maximum velocity.

And here is finally the curious fact I actually want to talk about: Nearly all the volume of an N-ball with so many dimensions is concentrated in an extremely thin shell beneath its surface. Then an integral over a thin shell can be extended over the full volume of the sphere without adding much, while making integration simpler.

This can be seen immediately from plotting the volume of a sphere over radius: The volume of an N-ball is always equal to some numerical factor, times the radius to the power of the number of dimensions. In three dimensions the volume is the traditional, honest volume proportional to r3, in two dimensions the ‘ball’ is a circle, and its ‘volume’ is its area. In a realistic thermodynamic system, the volume is then proportional to rN with a very large N.

The power function rN turn more and more into an L-shaped function with increasing exponent N. The volume increases enormously just by adding a small additional layer to the ball. In order to compare the function for different exponents, both ‘radius’ and ‘volume’ are shown in relation to the respective maximum value, R and RN.

The interesting layer ‘with all the volume’ is certainly much smaller than the radius R, but of course it must not be too small to contain something. How thick the substantial shell has to be can be found by investigating the volume in more detail – using a ‘trick’ that is needed often in statistical mechanics: Taylor expanding in the exponent.

A function can be replaced by its tangent if it is sufficiently ‘straight’ at this point. Mathematically it means: If dx is added to the argument x, then the function at the new target is f(x + dx), which can be approximated by f(x) + [the slope df/dx] * dx. The next – higher-order term would be proportional to the curvature, the second derivation – then the function is replaced by a 2nd order polynomial. Joseph Nebus has recently published a more comprehensible and detailed post about how this works.

So the first terms of this so-called Taylor expansion are:

f(x + dx) = f(x) + dx{\frac{df}{dx}} + {\frac{dx^2}{2}}{\frac{d^2f}{dx^2}} + ...

If dx is small higher-order terms can be neglected.

In the curious case of the ball in hyperspace we are interested in the ‘remaining volume’ V(r – dr). This should be small compared to V(r) = arN (a being the uninteresting constant numerical factor) after we remove a layer of thickness dr with the substantial ‘bulk of the volume’.

However, trying to expand the volume V(r – dr) = a(r – dr)N, we get:

V(r - dr) = V(r) - adrNr^{N-1} + a{\frac{dr^2}{2}}N(N-1)r^{N-2} + ...
= ar^N(1 - N{\frac{dr}{r}} + {\frac{N(N-1)}{2}}({\frac{dr}{r}})^2) + ...

But this is not exactly what we want: It is finally not an expansion, a polynomial, in (the small) ratio of dr/r, but in Ndr/r, and N is enormous.

So here’s the trick: 1) Apply the definition of the natural logarithm ln:

V(r - dr) = ae^{N\ln(r - dr)} = ae^{N\ln(r(1 - {\frac{dr}{r}}))}
= ae^{N(\ln(r) + ln(1 - {\frac{dr}{r}}))}
= ar^Ne^{\ln(1 - {\frac{dr}{r}}))} = V(r)e^{N(\ln(1 - {\frac{dr}{r}}))}

2) Spot a function that can be safely expanded in the exponent: The natural logarithm of 1 plus something small, dr/r. So we can expand near 1: The derivative of ln(x) is 1/x (thus equal to 1/1 near x=1) and ln(1) = 0. So ln(1 – x) is about -x for small x:

V(r - dr) = V(r)e^{N(0 - 1{\frac{dr}{r})}} \simeq V(r)e^{-N{\frac{dr}{r}}}

3) Re-arrange fractions …

V(r - dr) = V(r)e^{-\frac{dr}{(\frac{r}{N})}}

This is now the remaining volume, after the thin layer dr has been removed. It is small in comparison with V(r) if the exponential function is small, thus if {\frac{dr}{(\frac{r}{N})}} is large or if:

dr \gg \frac{r}{N}

Summarizing: The volume of the N-dimensional hyperball is contained mainly in a shell dr below the surface if the following inequalities hold:

{\frac{r}{N}} \ll dr \ll r

The second one is needed to state that the shell is thin – and allow for expansion in the exponent, the first one is needed to make the shell thick enough so that it contains something.

This might help to ‘visualize’ a closely related non-intuitive fact about large numbers, like eN: If you multiply such a number by a factor ‘it does not get that much bigger’ in a sense – even if the factor is itself a large number:

Assuming N is about 1025  then its natural logarithm is about 58 and…

Ne^N = e^{\ln(N)+N} = e^{58+10^{25}}

… 58 can be neglected compared to N itself. So a multiplicative factor becomes something to be neglected in a sum!

I used a plain number – base e – deliberately as I am obsessed with units. ‘r’ in phase space would be associated with a unit incorporating lots of lengths and momenta. Note that I use the term ‘dimensions’ in two slightly different, but related ways here: One is the mathematical dimension of (an abstract) space, the other is about cross-checking the physical units in case a ‘number’ is something that can be measured – like meters. The co-ordinate  numbers in the vector refer to measurable physical quantities. Applying the definition of the logarithm just to rN would result in dimensionless number N side-by-side with something that has dimensions of a logarithm of the unit.

Using r – a number with dimensions of length – as base, it has to be expressed as a plain number, a multiple of the unit length R_0 (like ‘1 meter’). So comparing the original volume of the ball a{(\frac{r}{R_0})}^N to one a factor of N bigger …

aN{(\frac{r}{R_0})}^N = ae^{\ln{(N)} + N\ln{(\frac{r}{R_0})}}

… then ln(N) can be neglected as long as \frac{r}{R_0} is not extreeeemely tiny. Using the same argument as for base e above, we are on the safe side (and can neglect factors) if r is of about the same order of magnitude as the ‘unit length’ R_0 . The argument about negligible factors is an argument about plain numbers – and those ‘don’t exist’ in the real world as one could always decide to measure the ‘radius’ in a units of, say, 10-30 ‘meters’, which would make the original absolute number small and thus the additional factor non-negligible. One might save the argument by saying that we would always use units that sort of match the typical dimensions (size) of a system.

Saying everything in another way: If the volume of a hyperball ~rN is multiplied by a factor, this corresponds to multiplying the radius r by a factor very, very close to 1 – the Nth root of the factor for the volume. Only because the number of dimensions is so large, the volume is increased so much by such a small increase in radius.

As the ‘bulk of the volume’ is contained in a thin shell, the total volume is about the product of the surface area and the thickness of the shell dr. The N-ball is bounded by a ‘sphere’ with one dimension less than the ball. Increasing the volume by a factor means that the surface area and/or the thickness have to be increased by factors so that the product of these factors yield the volume increase factor. dr scales with r, and does thus not change much – the two inequalities derived above do still hold. Most of the volume factor ‘goes into’ the factor for increasing the surface. ‘The surface becomes the volume’.

This was long-winded. My excuse: Also Richard Feynman took great pleasure in explaining the same phenomenon in different ways. In his lectures you can hear him speak to himself when he says something along the lines of: Now let’s see if we really understood this – let’s try to derive it in another way…

And above all, he says (in a lecture that is more about math than about physics)

Now you may ask, “What is mathematics doing in a physics lecture?” We have several possible excuses: first, of course, mathematics is an important tool, but that would only excuse us for giving the formula in two minutes. On the other hand, in theoretical physics we discover that all our laws can be written in mathematical form; and that this has a certain simplicity and beauty about it. So, ultimately, in order to understand nature it may be necessary to have a deeper understanding of mathematical relationships. But the real reason is that the subject is enjoyable, and although we humans cut nature up in different ways, and we have different courses in different departments, such compartmentalization is really artificial, and we should take our intellectual pleasures where we find them.

___________________________________

Further reading / sources: Any theoretical physics textbook on classical thermodynamics / statistical mechanics. I am just re-reading mine.

Other People Have Lives – I Have Domains

These are just some boring update notifications from the elkemental Webiverse.

The elkement blog has recently celebrated its fifth anniversary, and the punktwissen blog will turn five in December. Time to celebrate this – with new domain names that says exactly what these sites are – the ‘elkement.blog‘ and the ‘punktwissen.blog‘.

Actually, I wanted to get rid of the ads on both blogs, and with the upgrade came a free domain. WordPress has a detailed cookie policy – and I am showing it dutifully using the respective widget, but they have to defer to their partners when it comes to third-party cookies. I only want to worry about research cookies set by Twitter and Facebook, but not by ad providers, and I am also considering to remove social media sharing buttons and the embedded tweets. (Yes, I am thinking about this!)

On the websites under my control I went full dinosaur, and the server sends only non-interactive HTML pages sent to the client, not requiring any client-side activity. I now got rid of the last half-hearted usage of a session object and the respective cookie, and I have never used any social media buttons or other tracking.

So there are no login data or cookies to protect, but yet I finally migrated all sites to HTTPS.

It is a matter of principle: I of all website owners should use https. Since 15 years I have been planning and building Public Key Infrastructures and troubleshooting X.509 certificates.

But of course I fear Google’s verdict: They have announced long ago to HTTPS is considered a positive ranking by its search engine. Pages not using HTTPS will be tagged as insecure using more and more terrifying icons – e.g. http-only pages with login buttons already display a striked-through padlock in Firefox. In the past years I migrated a lot of PKIs from SHA1 to SHA256 to fight the first wave of Insecure icons.

Finally Let’s Encrypt has started a revolution: Free SSL certificates, based on domain validation only. My hosting provider uses a solution based on Let’s Encrypt – using a reverse proxy that does the actual HTTPS. I only had to re-target all my DNS records to the reverse proxy – it would have been very easy would it not have been for all my already existing URL rewriting and tweaking and redirecting. I also wanted to keep the option of still using HTTP in the future for tests and special scenario (like hosting a revocation list), so I decided on redirecting myself in the application(s) instead of using the offered automated redirect. But a code review and clean-up now and then can never hurt 🙂 For large complex sites the migration to HTTPS is anything but easy.

In case I ever forget which domains and host names I use, I just need to check out this list of Subject Alternative Names again:

(And I have another certificate for the ‘test’ host names that I need for testing the sites themselves and also for testing various redirects ;-))

WordPress.com also uses Let’s Encrypt (Automattic is a sponsor), and the SAN elkement.blog is lumped together with several other blog names, allegedly the ones which needed new certificates at about the same time.

It will be interesting what the consequences for phishing websites will be. Malicious websites will look trusted as being issued certificates automatically, but revoking a certificate might provide another method for invalidating a malicious website.

Anyway, special thanks to the WordPress.com Happiness Engineers and support staff at my hosting provider Puaschitz IT. Despite all the nerdiness displayed on this blog I prefer hosted / ‘shared’ solutions when it comes to my own websites because I totally like it when somebody else has to patch the server and deal with attacks. I am an annoying client – with all kinds of special needs and questions – thanks for the great support! 🙂

You Never Know

… when obscure knowledge comes in handy!

You can dismantle an old gutter without efforts, and without any special tools:

Just by gently setting it into twisted motion, effectively applying ~1Hz torsion waves that would lead to fatigue break within a few minutes.

I knew my stint in steel research in the 1990s would finally be good for something.

If you want to create a meme from this and tag it with Work Smart Not Harder, don’t forget to give me proper credits.

Ploughing Through Theoretical Physics Textbooks Is Therapeutic

And finally science confirms it, in a sense.

Again and again, I’ve harped on this pet theory of mine – on this blog and elsewhere on the web: At the peak of my immersion in the so-called corporate world, as a super-busy bonus miles-collecting consultant, I turned to the only solace: Getting up (even) earlier, and starting to re-read all my old mathematics and physics textbooks and lecture notes.

The effect was two-fold: It made me more detached, perhaps more Stoic when facing the seemingly urgent challenges of the accelerated world. Maybe it already prepared me for a long and gradual withdrawal from that biosphere. But surprisingly, I felt it also made my work results (even ;-)) better: I clearly remember compiling documentation I wrote after setting up some security infrastructure with a client. Writing precise documentation was again more like casting scientific research results into stone, carefully picking each term and trying to be as succinct as possible.

As anybody else I enjoy reading about psychological research that confirms my biases one-datapoint-based research – and here it finally is. Thanks to Professor Gary for sharing it. Science says that Corporate-Speak Makes You Stupid. Haven’t we – Dilbert fans – always felt that this has to be true?

… I’ve met otherwise intelligent people, after working with management consultant, are convinced that infinitely-malleable concepts like “disruptive innovation,” “business ecosystem,” and “collaborative culture” have objective value.

In my post In Praise of Textbooks with Tons of Formulas I focused on possible positive explanations, like speeding up your rational System 2 ((c) Daniel Kahneman) – by getting accustomed to mathematics again. By training yourself to recognize patterns and to think out of the box when trying to find the clever twist to solve a physics problem. Re-reading this, I cringe though: Thinking out of the box has entered the corporate vocabulary already. Disclaimer: I am talking about ways to pick a mathematical approach, by drawing on other, slightly related problems intuitively – in the way Kahneman explains the so-called intuition of experts as pattern recognition.

But perhaps the explanation is really as simple as that we just need to shield ourselves from negative effects of certain ecosystems and cultures that are particularly intrusive and mind-bending. So this is my advice to physics and math graduates: Do not rely on your infamous analytical skills forever. First, using that phrase in a job application sounds like phony hollow BS (as unfortunately any self-advertising of social skills does). Second, these skills are real, but they will decay exponentially if you don’t hone them.

6 volumes on all of Theoretical Physics - 1960s self-consistent series by my late professor Wilhelm Macke

Simulating Peak Ice

This year ice in the tank was finally melted between March 5 to March 10 – as ‘visual inspection’ showed. Level sensor Mr. Bubble was confused during the melting phase; thus it was an interesting exercise to compare simulations to measurements.

Simulations use the measured ambient temperature and solar radiation as an input, data points are taken every minute. Air temperature determines the heating energy needed by the house: Simulated heat load is increasing linearly until a maximum ‘cut off’ temperature.

The control logic of the real controller (UVR1611 / UVR16x2) is mirrored in the simulation: The controller’s heating curve determines the set temperature for the heating water, and it switches the virtual 3-way valves: Diverting heating water either to the hygienic storage or the buffer tank for space heating, and including the collector in the brine circuit if air temperature is high enough compared to brine temperature. In the brine circuit, three heat exchangers are connected in series: Three temperatures at different points are determined self-consistently from three equations that use underground tank temperature, air temperature, and the heat pump evaporator’s power as input parameters.

The hydraulic schematic for reference, as displayed in the controller’s visualization (See this article for details on operations.)

The Coefficient of Performance of the heat pump, its heating power, and its electrical input power are determined by heating water temperature and brine temperature – from polynomial fit curves to vendors’ data sheet.

So for every minute, the temperatures of tanks – hot and cold – and the volume of ice can be calculated from energy balances. The heating circuits and tap water consume energy, the heat pump delivers energy. The heat exchanger in the tank releases energy or harvests energy, and the collector exchanges energy with the environment. The heat flow between tank and ground is calculated by numerically solving the Heat Equation, using the nearly constant temperature in about 10 meters depth as a boundary condition.

For validating the simulation and for fine-tuning input parameters – like the thermal properties of ground or the building – I cross-check calculated versus measured daily / monthly energies and average temperatures.

Measurements for this winter show the artificial oscillations during the melting phase because Mr. Bubble faces the cliff of ice:

Simulations show growing of ice and the evolution of the tank temperature in agreement with measurements. The melting of ice is in line with observations. The ‘plateau’ shows the oscillations that Mr. Bubble notices, but the true amplitude is smaller:

2016-09 - 2017-03: Temperatures and ice formation - simulations.

Simulated peak ice is about 0,7m3 greater than the measured value. This can be explained by my neglecting temperature gradients within water or ice in the tank:

When there is only a bit of ice yet (small peak in December), tank temperature is underestimated: In reality, the density anomaly of water causes a zone of 4°C at the bottom, below the ice.

When the ice block is more massive (end of January), I overestimate brine temperature as ice has less than 0°C, at least intermittently when the heat pump is turned on. Thus the temperature difference between ambient air and brine is underestimated, and so is the simulated energy harvested from the collector – and more energy needs to be provided by freezing water.

However, a difference in volume of less than 10% is uncritical for system’s sizing, especially if you err on the size of caution. Temperature gradients in ice and convection in water should be less critical if heat exchanger tubes traverse the volume of tank evenly – our prime design principle.

I have got questions about the efficiency of immersed heat exchangers in the tank – will heat transfer deteriorate if the layer of ice becomes too thick? No, according also to this very detailed research report on simulations of ‘ice storage heat pump systems’ (p.5). We grow so-called ‘ice on coil’ which is compared to flat-plate heat exchangers:

… for the coil, the total heat transfer (UA), accounting for the growing ice surface, shows only a small decrease with growing ice thickness. The heat transfer resistance of the growing ice layer is partially compensated by the increased heat transfer area around the coil. In the case of the flat plate, on the contrary, also the UA-value decreases rapidly with growing ice thickness.

__________________________________

For system’s configuration data see the last chapter of this documentation.

Mr. Bubble Was Confused. A Cliffhanger.

This year we experienced a record-breaking January in Austria – the coldest since 30 years. Our heat pump system produced 14m3 of ice in the underground tank.

The volume of ice is measured by Mr. Bubble, the winner of The Ultimate Level Sensor Casting Show run by the Chief Engineer last year:

The classic, analog level sensor was very robust and simple, but required continuous human intervention:

Level sensor: The old way

So a multitude of prototypes had been evaluated …

Level sensors: The precursors

The challenge was to measure small changes in level as 1 mm corresponds to about 0,15 m3 of ice.

Mr. Bubble uses a flow of bubbling air in a tube; the measured pressure increases linearly with the distance of the liquid level from the nozzle:

blubber-messrohr-3

Mr. Bubble is fine and sane, as long as ice is growing monotonously: Ice grows from the heat exchanger tubes into the water, and the heat exchanger does not float due to buoyancy, as it is attached to the supporting construction. The design makes sure that not-yet-frozen water can always ‘escape’ to higher levels to make room for growing ice. Finally Mr. Bubble lives inside a hollow cylinder of water inside a block of ice. As long as all the ice is covered by water, Mr. Bubble’s calculation is correct.

But when ambient temperature rises and the collector harvests more energy then needed by the heat pump, melting starts at the heat exchanger tubes. The density of ice is smaller than that of water, so the water level in Mr. Bubble’s hollow cylinder is below the surface level of ice:

Mr. Bubble is utterly confused and literally driven over the edge – having to deal with this cliff of ice:

When ice is melted, the surface level inside the hollow cylinder drops quickly as the diameter of the cylinder is much smaller than the width of the tank. So the alleged volume of ice perceived by Mr. Bubble seems to drop extremely fast and out of proportion: 1m3 of ice is equivalent to 93kWh of energy – the energy our heat pump would need on an extremely cold day. On an ice melting day, the heat pump needs much less, so a drop of more than 1m3 per day is an artefact.

As long as there are ice castles on the surface, Mr. Bubble keeps underestimating the volume of ice. When it gets colder, ice grows again, and its growth is then overestimated via the same effect. Mr. Bubble amplifies the oscillations in growing and shrinking of ice.

In the final stages of melting a slab-with-a-hole-like structure ‘mounted’ above the water surface remains. The actual level of water is lower than it was before the ice period. This is reflected in the raw data – the distance measured. The volume of ice output is calibrated not to show negative values, but the underlying measurement data do:

Only when finally all ice has been melted – slowly and via thermal contact with air – then the water level is back to normal.

In the final stages of melting parts of the suspended slab of ice may break off and then floating small icebergs can confuse Mr. Bubble, too:

So how can we picture the true evolution of ice during melting? I am simulating the volume of ice, based on our measurements of air temperature. To be detailed in a future post – this is my cliffhanger!

>> Next episode.

Where to Find What?

I have confessed on this blog that I have Mr. Monk DVDs for a reason. We like to categorize, tag, painstakingly re-organize, and re-use. This is reflected in our Innovations in Agriculture …

The Seedbank: Left-over squared timber met the chopsaw.

The Nursery: Rebirth of copper tubes and newspapers.

… as well as in my periodical Raking The Virtual Zen Garden: Updating collections of web resources, especially those related to the heat pump system.

Here is a list of lists, sorted by increasing order of compactification:

But thanks to algorithms, we get helpful advice on presentation from social media platforms: Facebook, for example, encouraged me to tag products in the following photo, so here we go:

“Hand-crafted, artisanal, mobile nursery from recycled metal and wood, for holding biodegradable nursery pots.” Produced without crowd-funding and not submitted to contests concerned with The Intersection of Science, Art, and Innovation.