Predictive Coding Approximates Backprop along Arbitrary Computation Graphs

In this meetup, we will discuss this paper: https://arxiv.org/abs/2006.04182

Why? Feedforward-backprop usually has a loss function that involves all the parameters. Backprop means we need this huge global loss $\mathcal L({w_{ij}})$. However, it is hard to imaging such global loss calculations in our brain. One of the alternatives is predictive coding, which only utilizes local connection information.

In this paper (2006.04182), the author proves the equivalence of backprop and predictive coding on arbitary graph.