Computers, Science, and History Thereof

I am reading three online resources in parallel – on the history and the basics of computing, computer science, software engineering, and the related culture and ‘philosophy’. An accidental combination I find most enjoyable.

Joel on Software: Joel Spolsky’s blog – a collection of classic essays. What every developer needs to know about Unicode. New terms like Astronaut Architects and Leaky Abstractions. How to start a self-funded software company, how to figure out the price of software, how to write functional specifications. Bringing back memories of my first encounters with Microsoft VBA. He has the best examples – Martian Headsets to explain web standards.

The blog started in 1999 – rather shortly after I had entered the IT industry. So it is an interesting time capsule, capturing technologies and trends I was sort of part of – including the relationship with one large well-known software company.

Somewhere deep in Joel’s blog I found references to another classic; it was in an advice on how to show passion as an applicant for a software developer job. Tell them how reading this moved you to tears:

Structure and Interpretation of Computer Programs. I think I have found the equivalent to Feynman’s Physics Lectures in computer science! I have hardly ever read a textbook or attended a class that was both so philosophically insightful and useful in a hands-on, practical way. Using Scheme (Lisp) as an example, important concepts are introduced step-by-step, via examples, viewed from different perspectives.

It was amazing how far you can get with purely Functional Programming. I did not even notice that they had not used a single assignment (Data Mutation) until far into the course.

The quality of the resources made available for free is incredible – which holds for all the content I am praising in this post: Full textbook, video lectures with transcripts, slides with detailed comments. It is also good to know and reassuring that despite the allegedly fast paced changes of technology, basic concepts have not changed that much since decades.

But if you are already indulging in nostalgic thoughts why not catch up on the full history of computing?

Creatures of Thought. A sublime book-like blog on the history of computing – starting from with the history of telephone networks and telegraphs, covering computing machines – electro-mechanical or electronic, related and maybe unappreciated hardware components like the relay, and including biographic vignettes of the heroes involved.

The author’s PhD thesis (available for download on the About page) covers the ‘information utility’ vision that was ultimately superseded by the personal computer. This is an interesting time capsule for me as well, as this story ends about where my personal journey started – touching personal PCs in the late 1980s, but having been taught the basics of programming via sending my batch jobs to an ancient mainframe.

From such diligently done history of engineering I can only learn not to rush to any conclusions. There are no simple causes and effects, or unambiguous stories about who invented what and who was first. It’s all subtle evolution and meandering narratives, randomness and serendipity. Quoting from the post that indicates the beginning of the journey, on the origins of the electric telegraph:

Our physics textbooks have packaged up the messy past into a tidy collection of concepts and equations, eliding centuries of development and conflict between competing schools of thought. Ohm never wrote the formula V = IR, nor did Maxwell create Maxwell’s equations.

Though I will not attempt to explore all the twists and turns of the intellectual history of electricity, I will do my best to present ideas as they existed at the time, not as we retrospectively fit them into our modern categories.

~

Phone, 1970s, Austria

The kind of phone I used at the time when the video lectures for Structure and Interpretation of Computer Programs had been recorded and when I submitted my batch jobs of Fortran code to be compiled. I have revived the phone now and then.

 

Gödel, Escher, Bach, and Strange Loops: Nostalgia and Random Thoughts

I am curious – who read the book, too? Did you like it?

I read it nearly 30 years ago and I would also tag it one of the most influential books I read as a teenager.

[This might grow into a meandering and lengthy post with different (meta-)levels – given the subject of the post I think this is OK.]

GEB QR preview featured

A modern variant of the ambigram presented at GEB’s cover. The shadows created by this cube represent the QR codes of Wikipedia articles about Gödel, Escher, Bach, respectively.

In 1995 author Douglas Hofstadter said the following in an interview by Wired – and this also resembles similar statements in his book I am a Strange Loop published 2007. He utters frustration with the  effect of GEB on readers and on his reputation – although he won a Pulitzer Prize for his unusual debut book (published 1979).

From the Wired interview:

What Gödel, Escher, Bach was really about – and I thought I said it over and over again – was the word I. Consciousness. It was about how thinking emerges from well-hidden mechanisms, way down, that we hardly understand. How not just thinking, but our sense of self and our awareness of consciousness, sets us apart from other complicated things. How understanding self-reference could help explain consciousness so that someday we might recognize it inside very complicated structures such as computing machinery. I was trying to understand what makes for a self, and what makes for a soul. What makes consciousness come out of mere electrons coursing through wires.

There is nothing metaphysical in the way the term soul is used here. Having re-read GEB now I marvel at the level Hofstadter was able to provide an interpretation devoid of metaphysics – yet elegant and even poetic. Hofstadter is quoting Zen koans but he does not force “spirituality”  upon the subject – he calls Zen intellectual quicksand.

GEB is about the machinery of mind without catering to the AI enthusiasm shared by transhumanists. It has once been called a Bible of AI but maybe today it would not be considered optimistic enough in the nerdy sense. It is not about how new technology might exploit our (alleged) understanding of the mind – it is only about said understanding.

When I read the book nearly 30 years ago I enjoyed it for two main reasons: the allusions and references to language, metaphors and translation – especially as implemented in the whimsical Lewis-Carroll-style dialogues of Achilles, Mr. Tortoise and friends…

And yet many people treated the book as just some sort of big interdisciplinary romp whose point was simply to have fun. In fact, the fun was merely icing on the cake.

… and,  above all, that popular but yet mathy introduction to Gödel’s Incompleteness Theorem(s). Gödel’s theorem is presented as the analogue of oxymoronic statements such as I am a liar or This statement is false – translated to math. More precisely there are true statements about integers in sufficiently powerful formal systems that yet cannot be proven within those systems.

Originally, the book was purely about the way the proof of Gödel’s theorem kept cropping up in the middle of a fortress – Principia Mathematica by Bertrand Russell and Alfred North Whitehead – that was designed to keep it out. I thought, Here’s a structure that attempts to keep out self-knowledge, but when things get sufficiently complex and sufficiently tangled, all of a sudden – whammo! – it’s got self-representation in it. That to me was the trick that underlies consciousness.

I had considered Gödel the main part of the trio and I think I was sort of “right” due to this:

So, at first, there were no dialogs, no jokes, no wordplay, and no references to Escher or Bach. But as I typed the manuscript up in ’74, I decided it was written in an immature style. I decided to insert the dialogs and the Escher so that the playfulness became a kind of a secondary – but extremely important – part of the book. Many people focused on those things and treated the book as a big game-playing thing.

I am afraid, I did. I read the chapters dealing with a gradual introduction of the theorem more often than the parts about consciousness. Blending something abstract – that only hardcore nerds might appreciate – with wordplay, Escher drawings and musings on musical theory (pun not intended but obviously this is contagious) was a master piece of science writing. It seems this has widened the audience but not in an intended way.

But isn’t that the fate of nearly every real well-written science book transcending the boundaries of disciplines? Is there any philosopher-physicist writing about quantum mechanics who had not been quoted out-of-context by those who prefer to cook up metaphysical / emotionally appealing statements using scientifically sounding phrases as ingredients?

Anyway, focusing on the theorem: The gist of Hofstadter’s argument is that inherent contradictions were introduced directly to the very epitome of pristine rationality, Russell’s and Whitehead’s attempted to create. So we should not be surprised to find self-reference and emergent symbols in other systems built from boring little machine-like components. In a dialogue central to the idea of GEB his main protagonists discuss about holism and reductionism with a conscious ant hill – made up from dumb ants.

The meticulously expounded version of Gödel’s theorem is the heart and the pinnacle of the storyline of GEB in my point of view, and it is interesting to compare Hofstadter’s approach to the crisp explanation Scott Aaronson gives in Quantum Computing since Democritus. Scott Aaronson calls Gödel’s way to have formal statement talking about themselves an elaborate hack to program without programming. Aaronson makes the very convincing case that you could avoid all that talk about grand difficult math and numbering statements by starting from the notion of a computer, a Universal Turing machine.

Model of a Turing machine

Model of a Turing machine (Wikimedia, http://aturingmachine.com): an idealized computer working on a tape. It can move the read forward or back, (over-)write symbols on the tape or halt. Its actions are determined by the instructions on the tape and its internal state. Given a program you cannot decide if it will ever halt.

Gödel’s Proof then turns into a triviality as a formal system envisaged by Russell would be equivalent to having found a solution to the halting problem. The philosophical implications are preserved but it sounds more down-to-earth and it takes about two orders of magnitude less pages.

As Hofstadter says implicitly and explicitly: Metaphors and context are essential. Starting from a proof involving a program that is fed its own code probably avoids unwanted metaphysical-mystical connotations – compared to cooking up a scheme for turning statements of propositional logic into numbers, framed with Zen Buddhism, molecular biology, and art. But no matter in which way I might prefer to think about Gödel’s proof I guess I missed the mark:

(From the Wired interview – continued)

I had been aiming to have the book reach philosophers, people who thought about the mind and consciousness, and a small number actually saw what I was getting at, but most people just saw the glitter. At the time, I felt I’d lost a great deal by writing a book like that so early in my career, because I was no longer taken seriously by anybody.

If you did not get the message either you are in good company. David Deutsch, says in his review of I am a Strange Loop:

Hofstadter … expresses disappointment that his 1979 masterpiece Gödel, Escher, Bach (one of my favourite books) was not recognized as explaining the true nature of consciousness, or “I”-ness. I have to confess that it never occurred to me that it was intended to do so. I thought it merely explained the problem, highlighting stark flaws in common-sense ideas about minds. It also surveyed the infinite depth and meaning that can exist in “mere” computer programs. One could only emerge from the book (or so I thought) concluding that brains must in essence be computers, and consciousness an attribute of certain programs – and that discovering exactly what attribute is an urgent problem for philosophy and computer science. Hofstadter agrees with the first two conclusions but not the third; he considers that problem solved.

I can’t comment on the problem of consciousness being a yet-to-clarified attribute / by-product of computing but I find the loopy part about brains that must in essence be computers convincing.

Accidentally I have now read three different refutations of the so-called Chinese Room argument against strong AI – by Hofstadter, Aaronson and Ray Kurzweil. A human being in an hypothetical room pretends to exchange messages (on paper) in Chinese with interrogators. They might believe the guy speaks Chinese though he does only lookup rules in a book and mindlessly shift papers.

But how could you not associate the whole room, the rule book, the (high-speed!) paper-shuffling process with what goes on the system of the brains’ neurons? The person does not speak Chinese but “speaking Chinese” is an emergent phenomenon of the whole setup. Mental images invoked by “rule book” and “paper” are called intuition pumps by Hofstadter (a term coined by his friend Daniel Dennett) – examples picked deliberately to invoke that sudden “self-evident” insight along the lines of: Of course the human mind does not follow a mere rulebook!.

[Pushing to the level of self-referential  navel-gazing now]

Re-reading the blurb of my old version of the book I am able to connect some dots: I had forgotten that Hofstadter actually has a PhD in physics – theoretical condensed matter physics – and not in computer science or cognitive science. So the fact that a PhD in physics could prepare you for a career / life of making connections between all kinds of hard sciences, arts and literature was certainly something that might have shaped my worldview. All the authors heroes who have written those books that have influenced my the most as a teenager were scientist-philosophers, such as Albert Einstein and Viktor Frankl.

If I go on like this, talking about the science books and classics I read as a child I might get the same feedback as Hofstadter (see amazon.com reviews for example): This is elitist and only about showing off his education etc.

I am not sure what Hofstadter should have been done to avoid this. Not writing the books at all? Focusing on a narrower niche in order to comply with common belief that talents in seemingly diverse fields have to be mutually exclusive?

Usually a healthy dose of self-irony mitigates the smarty effect. Throw in jokes about how your stereotype absent-mindedness prevents you from exchanging that clichéd light bulb. But Hofstadter’s audience is rather diverse – so zooming in on the right kind of humor could be tricky.

[Pop]

And now I do what is explained so virtuoso in GEB – having pushed and popped through various meta-levels I will not resolve the tension and return to the tonic of the story … a music pun, pathetically used out of context.

[Coda]

You might wonder why I did not include any Escher drawings. There are all copyrighted still since  less than 70 years have passed since Escher’s death. But there are some interesting DIY Projects on Youtube, bringing to life Escher’s structures – such as this one:

_____________________________

Further reading The Man Who Would Teach Machines to Think (The Atlantic)

Douglas Hofstadter, the Pulitzer Prize–winning author of Gödel, Escher, Bach, thinks we’ve lost sight of what artificial intelligence really means. His stubborn quest to replicate the human mind.

Scott Aaronson’s website, blog and papers  – a treasure trove! His book is not an easy read and probably unlike every so called science book you have ever read. It has been created from lecture notes. His tone is conversational and the book is incredibly witty – but nonetheless it is quite compressed information containing more than one course in math, quantum physics and computer science. And yet – this is exactly the kind of science book I want to read when trying to make myself familiar with a new field. One “warning”: it is about theory, not about how to build a quantum computer. Thanks to wavewatching.net for the pointer.