PSYCH-GA.2211 / NEURL-GA.2201 Mathematical Tools for Neural and Cognitive Science

Instructors: Mike Landy & Eero Simoncelli Teaching Assistants: TBD

Time: Lectures: Tuesday/Thursday, 10:00-12:00 Labs: selected Fridays, 9:30-12:00

Location: Meyer 636

Description: A graduate lecture course covering fundamental mathematical methods for visualization, analysis, and modeling of neural and cognitive data and systems. The course was introduced in Spring of 1999, became a requirement for Neural Science doctoral students in 2000, and for Psychology doctoral students in the Cognition and Perception track in 2008. The course covers a foundational set of mathematical and statistical tools, providing assumptions, motivation, logical and geometric intuition, and simple derivations for each. Concepts are reinforced with extensive computational exercises. The goal is for students to be able to understand, use and interpret these tools.

Topics include: Linear algebra, least-squares and total-least-squares regression, eigen-analysis and PCA, linear shift-invariant systems, convolution, Fourier transforms, Nyquist sampling, basics of probability and statistics, hypothesis testing, model comparison, bootstrapping, estimation and decision theory, signal detection theory, linear discriminants, classification, clustering, simple models of neural spike generation, white noise (reverse-correlation) analysis.

Prerequisites: Algebra, trigonometry, and calculus. Some experience with matrix algebra and/or computer programming is helpful, but not required. The real

prerequisites are an aptitude for logical and geometric reasoning, and a willingness to work hard!

 

Announcements: We use Piazza for online class announcements, questions, and discussion: https://piazza.com/nyu/fall2020/psychga2211neurlga2201/home . Rather than emailing the instructors or TAs, we encourage you to post your questions/comments there, where they can be discussed and/or answered by any of us or your fellow classmates.

 

	Topic
	Introduction to the course Linear algebra I: vectors, vector spaces, inner products
	Matlab I: Environment, basic data types and operations, plotting
	Linear algebra II: projection, coordinate systems, linear systems
	Linear algbra III: linear systems, orthogonal/diagonal matrices, geometry
	Matlab II: scripts, conditionals, iteration, functions
	Lab: review of linear algebra concepts
	Linear algebra IV: singular value decomposition
	Linear algebra V: SVD, trichromacy example
	Regression I: regression, multiple regression via linear algebra
	Regression II: choosing regressors, weighting, outliers, overfitting
	Regression III: incorporating linear/quadratic constraints
	Regression IV: TLS regression, PCA, eigenvalues/ eigenvectors
	Lab: Regression/PCA
	LinSys I: Linear shift-invariant systems, convolution
	LinSys II: Sinusoids and LSI systems, Discrete Fourier transform
	Lab: Convolution in 1 and 2 dimensions
	LSI systems: review and examples

 

	LSI Systems: sampling and aliasing.
	Lab: Fourier Transform
	LSI Systems: Extended example: Sound, filtering, and the cochlea.
	Stats intro: summary stats, central tendency, disperson, multi-D.
	Stats: Multi-D, correlation, regression. Intro probability.
	Probability: expectations, moments, cumulatives, transformations, drawing samples
	Lab: Probability/samplinling
	Probability: Gaussians, marginals, conditionals, dependency, correlation mis-interpretations
	Inference I: Averages: 1/N convergence to mean, CLT, significance tests,p-values, z-test, t-test, permutation tests
	Lab: Bayes
	Inference II: estimation, the MLE, examples
	Inference III: confidence, MAP, sequential updating, bias-variance tradeoff, Bayes estimators
	Inference IV: Bayes. Signal Detection theory: ML/MAP/Bayes, d', ROC
	Inference V: decision theory, Multi-D decisions: prototype classifier, FLD.
	Lab: Simulations, bootstrapping, cross-validation
	Inference VI: Multi-D decisions: FLD, SVM, QDA.
	[No class: Happy Thanksgiving!]
	Inference VII: Fisher information, regularization
	modelFitting I: Regularization, model comparison
	Lab: Classification, regularization, clustering
	Fitting LNP models to data
	Spike-triggered covariance, population decoding

 

Resources (Electrons):

 

Online matlab help at The MathWorks | Tutorial at the MathWorks v | Intro video at MIT | Antonia Hamilton's Tutorial | at U. Utah | on reddit Richard Johnson's Matlab Style Handbook (lots of helpful tips, if a bit idiosyncratic)

Linear algebra Appendix from PDP series, by Michael Jordan. (pdf) Online lecture videos from Gilbert Strang's course at MIT

Online YouTube BlueBrown Linear Algebra videos, YouTube Essence of Linear Algebra videos Todd Will's Interactive Intro to the SVD

The Elements of Statistical Learning, Hastie, Tibshirani and Friedman - Excellent textbook on regression, decision/classification, clustering, and many advanced topics in data fitting and analysis. Available online (pdf)

Convex Optimization, Boyd and Vandenberg - Excellent textbook on the formulation of, and algorithms for, optimization of convex functions. Available online (pdf)

Thomas Minka's On-line Glossary of Statistical Pattern Recognition Wolfram Research World of Mathematics

History of various topics in mathematics

 

Resources (Dead Trees): Matlab:

Getting Started with MATLAB; A Quick Introduction for Scientists and Engineers, R. Pratap, Oxford U. Press, 2009.

Matlab for Neuroscientists. An introduction to scientific computing in Matlab, P. Wallisch, M. Lusignan, M. Benayoun, T. Baker, A. Dickey & N. Hatsopoulos, Elsevier Press, 2008.

Mastering MATLAB, B. L. Littlefield & D. C. Hanselman, Prentice-Hall, 2011.

 

Linear algebra / Least squares:

Linear Algebra and Its Applications, Gilbert Strang. Academic Press, 1980.

Introduction to Applied Linear Algebra, Stephen Boyd & Lieven Vandenberghe. Cambridge U. Press, 2018.

 

Linear (shift-invariant) Systems:

Discrete-time Signal Processing, A. Oppenheim & R. Schafer. Prentice Hall, 1989. The Fourier transform and its applications, R. Bracewell, McGraw Hill Science, 1999. Fast Fourier transform and its applications, E. Brigham, Prentice Hall, 1988.

 

Probability/Statistics:

 

Statistics, Freedman, Pisani, Purves, Norton, 2007 (4th ed.) Mathematical statistics, J. E. Freund, Prentice Hall, 1992. Decision Theory:

Biology: Elementary Signal Detection Theory, by Thomas D. Wickens. Oxford University Press, 2001. Signal Detection Theory and Psychophysics, by David Green & John Swets. Peninsula Publishing, 1988. Math: Statistical Decision Theory, by James O. Berger. Springer-Verlag, 1980.

Chapter 2 of Pattern Classification, by Duda, Hart and Storck. Wiley, 2001.

 

Bootstrap/Resampling:

An Intoduction to the Bootstrap, by Bradley Efron and Robert Tibshirani. Chapman & Hall, 1998.

Resampling Methods: A practical guide to data analysis, by Phillip Good. Birkhäuser, 1999.

 

Spikes, Neural Coding, Reverse Correlation:

Spikes: Exploring the Neural Code, by Fred Rieke, David Warland, Rob De Ruyter, & Bill Bialek. MIT Press, 1997.

 

Computational/Theoretical Neuroscience:

Theoretical Neuroscience , by Peter Dayan and Larry Abbott. MIT Press, 2001.