Relating Animal Behavior to Learning Algorithms Through Bayesian Inferences
Introduction:
Bayesian inference is at the center of many sophisticated machine learning algorithms. In this paper, we try to understand how the outcome of reinforcement learning algorithms is improved over time by analyzing animal behavior and decision making which is based upon the probabilistic models. Animals rely on these models in order to make decisions based on incomplete information and uncertainties in their surroundings. Animals learn from their experience similar to how the reinforcement learning algorithms which interact with an environment, and are improved upon receiving feedback in form of rewards and penalties.
Following is the Bayes Theorem: P(A∣B)=P(B∣A)⋅P(A)/P(B) where
P(A∣B) represents the posterior probability,
P(B∣A) is the likelihood of the observed data given the hypothesis,
P(A) is the prior probability of the hypothesis,
P(B) is the probability of the observed data.
In simplest terms, Bayes theorem involves combining prior knowledge or beliefs (prior probability) with new data (likelihood) to arrive at an updated or posterior probability.
Animals construct a posterior opinion grounded in sampled data, such as the quality of food patches or the average qualities of potential mates. Their prior knowledge stems from personal experience or the adaptive wisdom passed down through generations.
Consider a squirrel that has acquired knowledge about the likelihood of finding nuts in different areas of the forest based on past experiences or ancestral wisdom. While foraging, the squirrel keenly observes nut availability in various forest regions, combining this information with its prior knowledge to estimate the probability of finding nuts in a particular area. This estimation guides the squirrel in deciding where to concentrate its foraging efforts. Additionally, in response to seasonal changes affecting nut availability, the squirrel adapts its strategy accordingly. With each foraging experience, the squirrel continuously refines its understanding of where nuts are most likely to be found.
Above principles used by animals underlie reinforcement learning principles. In reinforcement learning, continuous learning is also based on integration of prior knowledge with new experiences that allows for adaptability over time as conditions change, hence leading to improved decision-making.
Source: Youssef, Yasmin. (2022). Bayes Theorem and Real-life Applications.