Generalization
Regarding the topic of our ability and inability to generalize, the following is some notes and remarks based on the books
We will start with a summary of these books and then follow up with thoughts and observations.
Artificial Intelligence is a high-level look on the current progress on AI in games, logic, computervision and speech recognition. Its conclusion is, that deep learning based AI approaches have come very far in solving specific problems (to no small degree because of the exponential curve of hardware advancements). Their shortcomings are a lack of task generalization, which we humans do continuously. According to Melanie Mitchell, we are as far away from a general artificial intelligence as we where 50 years ago, as virtually no progress could be made to find an appropriate design. A hindrance is our lack of understanding regarding the design of our own brain. Convolutional Neural Nets are apparently similarly modeled to the way the neural paths from our eye are constructed into increasingly denser clusters of information. This, considering how spectacular CNNs have been performing in the past years, is a sign that modeling our networks based on the newest neurological research might be a way forward.
The book gives a few key differences between the structure of deep neural networks and the network design of the human brain. While deep neural networks generally are designed to move information forward, the brain has a lot more loop-backs and interconnections between different neuron layers. Neurons that fire in the ‘wrong direction’ actually outnumber the perceptron styled neurons 10:1. Also, humans have far more neurons than are currently used inside any artificial neural network - which is mainly due to hardware limitations I would presume. Finally, humans have - throughout our life - access to a substantial amount of data and possibilities to test the current neuron configuration.
This book summarizes a range of studies done, to show not only the irrationality of people but also an inherent pattern in this irrational thinking. The topics range from how we deal with a price change from 1 cent to 0 cents, how social cost and economic cost differ, and our necessity of keeping our options open.
A large section of the book deals with our irrationality in the context of social norms and expected behavior within groups. And while these topics were interesting, the ones that fascinated me the most are the ones, in which we are irrational, even when outside of the complex interactions with society.
Note, that the thesis of this book is to show, that the homo economicus is not a valid model of humans. Nevertheless, I believe there is some value here within the context of generalization. Our evaluation of worth (resources of any kind) is not sufficiently generalized, to allow for any form of objective reasoning. Even when dealing with time or money, which are exceptionally simple to quantify, we are still not able to overcome irrational behavior.
This book mainly supplied me with the mental model mentioned in the title. The main thesis of this book is removing oneself from thinking in absolutes, towards thinking in probabilities. I’m not sure if that’s just what is already happening in my environment, but it seems to me like most people already do that. It nevertheless certainly helped me in acknowledging this behavior as a positive one, and adding a feedback loop, where I check if the way I am approaching a question is in ‘probabilistic’ terms.
Though, what is a tool that has reframed some questions for me, is posing the question as a bet. For some reason, putting (virtual) money on the line changes my way of preceiving the problem. It helps remove hopes and desires and moves my mindset further into rational economics - at least that’s how it feels.
The authors of this book want to show, that the broader a person’s knowledge is, the greater their potential impact in society. They argue, that abstract thinking is a skill which humans have improved upon constantly and the past few hundred years have seen the greatest improvements. A study, done during and after the industrial revolution in Russia examines different towns - ones already affected by the revolution and others not. The industrialized towns had additional schooling and the need to understand a new, more complex work environment. The study tried to gauge the level of abstract thinking in the population, by giving them simple three-part logic puzzles, eq. Rice grows only in hot and dry climates; London is cold and humid. Does rice grow in London? Similar the people were asked to group a filled circle and an empty circle, a big triangle and a small triangle into two logical groups.
The people in the pre-industrial town were not able to do either of these examples. Apparently, so the authors note, the ability to use abstract logic to combine the information given, was not developed in these humans. The two circles were either a coin or a moon and did not bear any resemblance with each other. Shapes, colors and forms are skills, which we pick up so early in life, that it seems natural to us - not something we needed to learn. This shows how much we can learn with our brains, and how far away we already are from our initial ancestors.
The main points regarding generalization are:
Generalization seems to be learned behavior. The more diverse the dataset, the better. Edge cases (free food) need to be explicitly added to the dataset because it seems our brain might not be able to extrapolate easily - similar to the long tail problem in AI research. A similar event is our hight reliance on initial anchors described in predictably irrational. A single initial event primes our expectations and any further data modifies this. We can parallel this to when we teach artificial neural networks and initialize the network either with random weights or some pre-trained neurons.
A quick note on economic concepts changing the way we think of decisions: The similarity between behaving irrational, when something is ‘free’ and behaving more rational when something suddenly is ’not free’ - i.e. when thinking in bets.