NeuronStarhttps://neuronstar.github.io/Recent content on NeuronStarHugo -- gohugo.ioen-USThu, 24 Mar 2016 00:00:00 +0000Conditional Probability and Bayeshttps://neuronstar.github.io/cpe/01.conditional-probability-and-bayes/Wed, 18 Nov 2020 00:00:00 +0000https://neuronstar.github.io/cpe/01.conditional-probability-and-bayes/The Bayesian view of probability is quite objective and also more general than the frequentist’s view. It doesn’t rely on repeatition of events.01.Neuron Biological Propertieshttps://neuronstar.github.io/snm/01.single_neuron_model/Fri, 18 Mar 2016 00:00:00 +0000https://neuronstar.github.io/snm/01.single_neuron_model/Neuron biological propertiesLeast Squares, Bootstrap, Maximum Likelihood, and Bayesianhttps://neuronstar.github.io/cpe/02.least-squares-bootstrap-maximum-likelihood-and-bayesian/Sat, 12 Dec 2020 00:00:00 +0000https://neuronstar.github.io/cpe/02.least-squares-bootstrap-maximum-likelihood-and-bayesian/Least squares, bootstrap, maximum likelihood, and maximum posterior leads to the same results in many cases.02.Review of Last Week's Readinghttps://neuronstar.github.io/snm/02.limitations_srm_contd_and_coding/Fri, 18 Mar 2016 00:00:00 +0000https://neuronstar.github.io/snm/02.limitations_srm_contd_and_coding/Review of Last Week’s ReadingEM Methodshttps://neuronstar.github.io/cpe/03.em-methods/Sat, 02 Jan 2021 00:00:00 +0000https://neuronstar.github.io/cpe/03.em-methods/Topics EM for Gaussian mixtures General EM algorithm Why does it work? Decomposition of log-likelihood into KL divergence and Relation between EM and Gibbs sampling03.Equilibrium Potential and Hodgkin-Huxley Modelhttps://neuronstar.github.io/snm/03.equilibrium_potential_and_hodgkin-huxley_model/Sun, 13 Mar 2016 00:00:00 +0000https://neuronstar.github.io/snm/03.equilibrium_potential_and_hodgkin-huxley_model/Equilibrium Potential and Hodgkin-Huxley ModelVariantional Inference Normalizing Flowhttps://neuronstar.github.io/cpe/04.variational-inference-normalizing-flow/Sat, 16 Jan 2021 00:00:00 +0000https://neuronstar.github.io/cpe/04.variational-inference-normalizing-flow/Topics Variational Inference Normalizing Flow Variational Inference with Normalizing Flows04.The Zoo of ion channelshttps://neuronstar.github.io/snm/04.the_zoo_of_ion_channels/Wed, 23 Mar 2016 00:00:00 +0000https://neuronstar.github.io/snm/04.the_zoo_of_ion_channels/The Zoo of ion channelsReview of Normalizing Flowhttps://neuronstar.github.io/cpe/05.normalizing-flow-review/Sat, 30 Jan 2021 00:00:00 +0000https://neuronstar.github.io/cpe/05.normalizing-flow-review/Topics Normalizing flow Applications of normalizing flow Methods of normalizing flow Problems of normalizing flow05.Synapse and receptorhttps://neuronstar.github.io/snm/05.synapse_and_receptors/Mon, 28 Mar 2016 00:00:00 +0000https://neuronstar.github.io/snm/05.synapse_and_receptors/Synapse and receptorDeep AutoRegressive Networkshttps://neuronstar.github.io/cpe/06.deep-autoregressive-networks/Sat, 13 Feb 2021 00:00:00 +0000https://neuronstar.github.io/cpe/06.deep-autoregressive-networks/Topics Refer to references.
Notes 1310.8499_notes.pdf06.Cable Equation and Its Solutionshttps://neuronstar.github.io/snm/06.cable_equation_and_its_solutions/Sat, 02 Apr 2016 00:00:00 +0000https://neuronstar.github.io/snm/06.cable_equation_and_its_solutions/Cable Equation and Its SolutionsMADE: Masked Autoencoder for Distribution Estimationhttps://neuronstar.github.io/cpe/07.made/Sat, 27 Feb 2021 00:00:00 +0000https://neuronstar.github.io/cpe/07.made/Topics Refer to references.
Notes 1310.8499_notes.pdf07.Two dimensional neuron modelshttps://neuronstar.github.io/snm/07.reduction_to_two_dimensions_and_phase_plane_analysis/Thu, 21 Apr 2016 00:00:00 +0000https://neuronstar.github.io/snm/07.reduction_to_two_dimensions_and_phase_plane_analysis/Two dimensional neuron modelsMAF: how is MADE being usedhttps://neuronstar.github.io/cpe/08.maf/Sat, 27 Feb 2021 00:00:00 +0000https://neuronstar.github.io/cpe/08.maf/We discussed MAF (arXiv:1705.07057v4) last time: The paper did not explain how exactly is MADE being used to update the shift and logscale.
We will use the tensorflow implementation of MAF to probe the above question. Here is the link to the relevant documentation: https://www.tensorflow.org/probability/api_docs/python/tfp/bijectors/MaskedAutoregressiveFlow
Topics Refer to references.
Notes 1310.8499_notes.pdf08.Integrate and Fire Models Part 1https://neuronstar.github.io/snm/08.integrate-and-fire-models-1/Sat, 21 May 2016 00:00:00 +0000https://neuronstar.github.io/snm/08.integrate-and-fire-models-1/Integrate and Fire Model Part 1Summary of Generative Modelshttps://neuronstar.github.io/cpe/09.summary-of-generative-models/Sat, 27 Feb 2021 00:00:00 +0000https://neuronstar.github.io/cpe/09.summary-of-generative-models/09.Reduction of the Hodgkin-Huxley model type IIhttps://neuronstar.github.io/snm/09.from_detailed_models_to_formal_spiking_neurons/Sat, 25 Jun 2016 00:00:00 +0000https://neuronstar.github.io/snm/09.from_detailed_models_to_formal_spiking_neurons/Reduction of the Hodgkin-Huxley model ‘type II’Energy-based Modelshttps://neuronstar.github.io/cpe/10.energy-based-learning/Sat, 27 Feb 2021 00:00:00 +0000https://neuronstar.github.io/cpe/10.energy-based-learning/We will discuss energy-based learning in this session.
References:
Lecture notes: https://atcold.github.io/pytorch-Deep-Learning/[cid:90ae645c-415b-4c9f-8ea6-c78839a8e8d4] https://atcold.github.io/pytorch-Deep-Learning/en/week07/07-1/ https://atcold.github.io/pytorch-Deep-Learning/en/week07/07-2/ https://drive.google.com/file/d/1z8Dz1YtkOEJpU-gh5RIjORs3GGqkYJQa/view : if you can not access Google Drive, this file (007-ebm-01) has been attached to this calendar event too. Supplementary:
https://arxiv.org/pdf/1803.08823.pdf10.Information Codinghttps://neuronstar.github.io/snm/10.noise_and_renewal_process/Sat, 25 Jun 2016 00:00:00 +0000https://neuronstar.github.io/snm/10.noise_and_renewal_process/Information codingEnergy-based Models 2https://neuronstar.github.io/cpe/11.energy-based-learning-2/Sat, 27 Feb 2021 00:00:00 +0000https://neuronstar.github.io/cpe/11.energy-based-learning-2/We will discuss energy-based learning in this session.
References:
Lecture notes: https://atcold.github.io/pytorch-Deep-Learning/[cid:90ae645c-415b-4c9f-8ea6-c78839a8e8d4] https://atcold.github.io/pytorch-Deep-Learning/en/week07/07-1/ https://atcold.github.io/pytorch-Deep-Learning/en/week07/07-2/ https://drive.google.com/file/d/1z8Dz1YtkOEJpU-gh5RIjORs3GGqkYJQa/view : if you can not access Google Drive, this file (007-ebm-01) has been attached to this calendar event too. Supplementary:
https://arxiv.org/pdf/1803.08823.pdf11.Renewal Theoryhttps://neuronstar.github.io/snm/11.stationary_renewal_theory/Tue, 05 Jul 2016 00:00:00 +0000https://neuronstar.github.io/snm/11.stationary_renewal_theory/Renewal TheoryEnergy-based Models 3https://neuronstar.github.io/cpe/12.energy-based-learning-3/Sat, 24 Apr 2021 00:00:00 +0000https://neuronstar.github.io/cpe/12.energy-based-learning-3/In the past two meetups, we have been discussing EBM from a computer scientist’s perspective.
In this discussion, we will discuss chapter XV of Mehta P, Bukov M, Wang C-HH, Day AGRR, Richardson C, Fisher CK, et al. A high-bias, low-variance introduction to Machine Learning for physicists. Phys Rep. 2018;810: 122. doi:10.1016/j.physrep.2019.03.00109.Escape Noisehttps://neuronstar.github.io/snm/12.escape_noise/Fri, 29 Jul 2016 00:00:00 +0000https://neuronstar.github.io/snm/12.escape_noise/Escape NoiseEnergy-based Models 4https://neuronstar.github.io/cpe/13.energy-based-learning-4/Wed, 26 May 2021 00:00:00 +0000https://neuronstar.github.io/cpe/13.energy-based-learning-4/In this discussion, we will discuss the Pytorch Deep Learning Lectures by LeCun.13.Comparison Between Neuron Modelshttps://neuronstar.github.io/snm/13.all_neuron_models/Tue, 05 Jul 2016 00:00:00 +0000https://neuronstar.github.io/snm/13.all_neuron_models/Comparison Between Neuron ModelsEnergy-based Models 5https://neuronstar.github.io/cpe/14.energy-based-learning-5/Wed, 02 Jun 2021 00:00:00 +0000https://neuronstar.github.io/cpe/14.energy-based-learning-5/In this meetup, we will discuss Restricted Boltzmann Machine (RBM). We will cover the reason for introducing RBM and the training. At the end of the discussion, we will also cover some topics of Deep Boltzmann Machines.14.Noise in Refractory Kernel and Diffusive Noisehttps://neuronstar.github.io/snm/14.slow-noise/Fri, 22 Jul 2016 00:00:00 +0000https://neuronstar.github.io/snm/14.slow-noise/Slow Noise in parameters and diffusive noise (Part 1)Predictive Coding Approximates Backprop along Arbitrary Computation Graphshttps://neuronstar.github.io/cpe/15.predictive-coding/Mon, 21 Jun 2021 00:00:00 +0000https://neuronstar.github.io/cpe/15.predictive-coding/In this meetup, we will discuss this paper: https://arxiv.org/abs/2006.04182
Why? Feedforward-backprop usually has a loss function that involves all the parameters. Backprop means we need this huge global loss $\mathcal L({w_{ij}})$. However, it is hard to imaging such global loss calculations in our brain. One of the alternatives is predictive coding, which only utilizes local connection information.
In this paper (2006.04182), the author proves the equivalence of backprop and predictive coding on arbitary graph.15.Diffusive Noise and The Subthreshold Regimehttps://neuronstar.github.io/snm/15.diffusive_noise_and_the_subthreshold_regime/Fri, 29 Jul 2016 00:00:00 +0000https://neuronstar.github.io/snm/15.diffusive_noise_and_the_subthreshold_regime/Diffusive Noise and The subthreshold RegimeLTD/LTPhttps://neuronstar.github.io/cpe/16.ltd-ltp/Mon, 21 Jun 2021 00:00:00 +0000https://neuronstar.github.io/cpe/16.ltd-ltp/In this meetup, we will discuss some key ideas related to biological neural network: LTP and LTD.16.stochastic processhttps://neuronstar.github.io/snm/16.stochastic_process/Fri, 05 Aug 2016 00:00:00 +0000https://neuronstar.github.io/snm/16.stochastic_process/stochastic processSelf-supervised Learning: Generative or Contrastivehttps://neuronstar.github.io/cpe/17.self-supervised-learning/Sat, 03 Jul 2021 00:00:00 +0000https://neuronstar.github.io/cpe/17.self-supervised-learning/Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218
We have discussed many topics of generative models. This paper serves as a summary of the current season of the discussions.17.Homogeneous Networkhttps://neuronstar.github.io/snm/17.homogeneous-network/Fri, 02 Sep 2016 00:00:00 +0000https://neuronstar.github.io/snm/17.homogeneous-network/Review of population activity; Homogeneous network.Self-supervised Learning: GANhttps://neuronstar.github.io/cpe/18.self-supervised-learning-gan/Sun, 01 Aug 2021 00:00:00 +0000https://neuronstar.github.io/cpe/18.self-supervised-learning-gan/We will discuss the reset of the paper arXiv:2006.08218.18.SRM with Escape Noisehttps://neuronstar.github.io/snm/18.population-srm-with-escape-noise/Fri, 23 Sep 2016 00:00:00 +0000https://neuronstar.github.io/snm/18.population-srm-with-escape-noise/SRM neurons with escape noiseSelf-supervised Learning: Theories (Part 1)https://neuronstar.github.io/cpe/19.self-supervised-learning-theories-1/Thu, 26 Aug 2021 00:00:00 +0000https://neuronstar.github.io/cpe/19.self-supervised-learning-theories-1/We will discuss Section 6 of the paper arXiv:2006.08218.19.Population Activityhttps://neuronstar.github.io/snm/19.population-activity/Fri, 30 Sep 2016 00:00:00 +0000https://neuronstar.github.io/snm/19.population-activity/Integral equations for population activitySelf-supervised Learning: Theories (Part 2)https://neuronstar.github.io/cpe/20.self-supervised-learning-theories-2/Thu, 26 Aug 2021 00:00:00 +0000https://neuronstar.github.io/cpe/20.self-supervised-learning-theories-2/We will dive deep into Section 6 of the paper arXiv:2006.08218. Here are a few topics to be explored.
InfoGAN objective; Positive and negative samples in loss function (InfoNCE); Uniformity in constrastive loss; JS-divergence.20.Basics of Renewal Theoryhttps://neuronstar.github.io/snm/20.basics-of-renewal-theory/Sat, 08 Oct 2016 00:00:00 +0000https://neuronstar.github.io/snm/20.basics-of-renewal-theory/from math to neuroscienceGraph Neural Networks: Basicshttps://neuronstar.github.io/cpe/21.gnn-basics/Sat, 11 Sep 2021 00:00:00 +0000https://neuronstar.github.io/cpe/21.gnn-basics/This will be the beginning of a new topic: Graph Neural Networks. In this new series, we will use the textbook by Hamilton1. For the first episode, we will discuss some basics about graphs to make sure we are all on the same page.
@Steven will lead the discussion.
Hamilton2020 Hamilton WL. Graph representation learning. Synth lect artif intell mach learn. 2020;14: 1–159. doi:10.2200/s01045ed1v01y202009aim046 ↩︎21.Asynchronous Firinghttps://neuronstar.github.io/snm/21.asynchronous-firing/Fri, 14 Oct 2016 00:00:00 +0000https://neuronstar.github.io/snm/21.asynchronous-firing/Asynchronous firing of homogeneous network22.interacting populations and continuum modelshttps://neuronstar.github.io/snm/22.interacting-populations-and-continuum-models/Sat, 29 Oct 2016 00:00:00 +0000https://neuronstar.github.io/snm/22.interacting-populations-and-continuum-models/network of networks and continuum network23.Linearized Population Equation and Transientshttps://neuronstar.github.io/snm/23.linearized-population-equation/Fri, 11 Nov 2016 00:00:00 +0000https://neuronstar.github.io/snm/23.linearized-population-equation/The population equation is quite complicated to solve, hence we linearize it and inspect the perturbation theory.24. From individual neurons to collective burstinghttps://neuronstar.github.io/snm/24.from_individual_neurons_to_collective_bursting/Wed, 16 Nov 2016 00:00:00 +0000https://neuronstar.github.io/snm/24.from_individual_neurons_to_collective_bursting/Predicting collective dynamics from individual neuron properties.25. The Significance of Single Spikehttps://neuronstar.github.io/snm/25.significance-of-single-spike/Fri, 09 Dec 2016 00:00:00 +0000https://neuronstar.github.io/snm/25.significance-of-single-spike/Single spike can have dramatic consequences on population activity.27. Synchronized Oscillations and Lockinghttps://neuronstar.github.io/snm/27.synchronized-oscillations-and-locking/Fri, 03 Feb 2017 00:00:00 +0000https://neuronstar.github.io/snm/27.synchronized-oscillations-and-locking/Locking28. Oscillations in Reverberating Loopshttps://neuronstar.github.io/snm/28.oscillations-in-reverberating-loops/Fri, 17 Feb 2017 00:00:00 +0000https://neuronstar.github.io/snm/28.oscillations-in-reverberating-loops/Oscillations in reverberating loops can be simplified and researched.29. Hebbian Learninghttps://neuronstar.github.io/snm/29.hebbian-learning/Sat, 06 May 2017 00:00:00 +0000https://neuronstar.github.io/snm/29.hebbian-learning/Simplest learning rule, aka, correlation based learning30. Learning Equationshttps://neuronstar.github.io/snm/30.learning-equations/Sat, 03 Jun 2017 00:00:00 +0000https://neuronstar.github.io/snm/30.learning-equations/Unsupervised learning31. Plasticity and Codinghttps://neuronstar.github.io/snm/31.plasticity-and-coding/Sat, 10 Jun 2017 00:00:00 +0000https://neuronstar.github.io/snm/31.plasticity-and-coding/How is plasticity related to neuronal codingReferences for Probability Estimation Clubhttps://neuronstar.github.io/cpe/00.references/Sat, 12 Dec 2020 00:00:00 +0000https://neuronstar.github.io/cpe/00.references/A list of references for our online discussions.Conditional Probability Estimationhttps://neuronstar.github.io/projects/conditional-probability-estimation/Tue, 03 Nov 2020 00:00:00 +0000https://neuronstar.github.io/projects/conditional-probability-estimation/Understand models to estimate conditional probabilitiesFoundations of Machine Learninghttps://neuronstar.github.io/projects/ml-foundations/Sun, 03 May 2020 00:00:00 +0000https://neuronstar.github.io/projects/ml-foundations/Dive deep into the foundations of machine learning.Spiking Neuron Modelshttps://neuronstar.github.io/projects/snm/Mon, 27 Apr 2020 13:22:46 +0200https://neuronstar.github.io/projects/snm/Reading club for the book Spiking Neuron ModelsThe Elements of Statistical Learning Reading Clubhttps://neuronstar.github.io/projects/esl/Mon, 27 Apr 2020 13:22:46 +0200https://neuronstar.github.io/projects/esl/Read the book05.Least Angle Regressionhttps://neuronstar.github.io/esl/05.least-angle-regression/Thu, 29 Sep 2016 00:00:00 +0000https://neuronstar.github.io/esl/05.least-angle-regression/Least angle regression, aka, LAR04.Shrinkage Methodshttps://neuronstar.github.io/esl/04.shrinkage-methods/Fri, 23 Sep 2016 00:00:00 +0000https://neuronstar.github.io/esl/04.shrinkage-methods/Shrinkage methods03.Guass-Markov Theorem and Multiple Regressionhttps://neuronstar.github.io/esl/03.gauss-markov-theorem/Thu, 01 Sep 2016 00:00:00 +0000https://neuronstar.github.io/esl/03.gauss-markov-theorem/Gauss-Markov Theorem02.Linear Methods for Regressionhttps://neuronstar.github.io/esl/02.linear-methods-for-regresssion/Thu, 18 Aug 2016 00:00:00 +0000https://neuronstar.github.io/esl/02.linear-methods-for-regresssion/Linear regression, least squares01.Introductions (Review) and Several Preliminary Statistical Methodshttps://neuronstar.github.io/esl/01.statistical-learning-theory/Wed, 06 Jul 2016 00:00:00 +0000https://neuronstar.github.io/esl/01.statistical-learning-theory/Some basics of statistical learning; least squares and k nearest neighbors; statistical decision theory; local methods in high dimensionsABOUThttps://neuronstar.github.io/about/Thu, 24 Mar 2016 00:00:00 +0000https://neuronstar.github.io/about/We host reading clubs and seminars on neuroscience, machine learning, complex networks and intelligence.
License & Source Articles on this website are published under CC BY-NC-SA license if no specific license is designated.
This website is hosted on GitHub and generated by GitHub Pages (hugo). Computational Neuroscience Map is written in TiddlyMap and hosted statically on GitHub Pages.00.Spiking Neuron Models Reading Clubhttps://neuronstar.github.io/snm/00.spiking_neuron_models_club/Fri, 18 Mar 2016 00:00:00 +0000https://neuronstar.github.io/snm/00.spiking_neuron_models_club/Introduction to reading club of spiking neuron models, schedule, and notice00.The Elements of Statistical Learning Reading Clubhttps://neuronstar.github.io/esl/00.the-elements-of-statistical-learning/Fri, 18 Mar 2016 00:00:00 +0000https://neuronstar.github.io/esl/00.the-elements-of-statistical-learning/Introduction to reading club of The Elements of Statistical Learning, schedule, and notice<link>https://neuronstar.github.io/esl/readme/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://neuronstar.github.io/esl/readme/</guid><description>elements-of-statistical-learning Reading club: The Elements of Statistical Learning
Online Course: StatLearning@Standford
An Introductory Book: http://www-bcf.usc.edu/~gareth/ISL/ISLR%20Seventh%20Printing.pdf http://www.dataschool.io/15-hours-of-expert-machine-learning-videos/ The easiest way of creating notes is to duplicate one of the previous .md files and make changes to it.
Code of conduct:
Create a markdown file with extension .md; Any file name works, however, file names begins with two-digit number would be a nice convention. The markdown file has to include a header session that specifies the meta data.</description></item><item><title/><link>https://neuronstar.github.io/typography/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://neuronstar.github.io/typography/</guid><description>ToC {:toc} We use kramdown This website uses kramdown as the basic syntax. However, a lot of html/css/js has been applied to generate some certain contents or styles.
Math also follows the kramdown syntax.
Footnote Syntax for footnotes is elaborated more on the website of kramdown.
{% highlight text %} Some text here some other text here.1
Table of Contents {% highlight text %}
ToC {:toc} {% endhighlight %} is used to generate table of contents.</description></item></channel></rss>