Show The Graduate Center Menu

Neural Networks and Deep Learning


With the recent boom in artificial intelligence, more specifically, Deep Learning and its underlying Neural Networks, are essential part of systems that must perform recognition, make decisions and operate machinery. 

Deep Learning in Artificial Neural Networks (ANN) is relevant for Supervised, Unsupervised, and Reinforcement Learning. This course will provide a thorough examination of the state-of-the-art and will present the mathematical and algorithmic foundations of Deep Learning in ANN. 


Deep Learning concerns multilevel data representation with every level providing hierarchical explanations of data. These algorithms have proven effective in discovering the structure of data and the features that most differentiate among classes. 

Artificial Neural Networks are created to mimic the human brain. In the process, various architectures of neurons and connections have been created. Generally, the artificial neurons are activated through input signals and the connections among them are learned through various algorithms. 

The course will cover the underlying theory, the range of applications, and learning from very large data sets. Connectionist architectures commonly associated with deep learning, e.g., basic neural networks, convolutional neural networks and recurrent neural networks will also be discussed. Methods to train and optimize the architectures and methods to perform effective inference with them, will be the main focus. 

Topic List 

Topics may include but are not limited to: 

  Deep Learning 

  o Depth of deep learning 

  o Overview of methods 

  o Linear algebra and numerical methods 

  Neural Networks 

  o Feed-forward architecture 

  o Gradient-based learning 

  Deep Neural Networks 

  o Architecture design 

  o Computational graphs 

  o Backpropagation 

  Regularization 

  o Overview 

  o Multitask learning 

  o Parameter sharing 

  Convolutional Neural Networks 

  o Operation and algorithm 

  o Pooling 

  Recurrent and Recursive Networks 

  o Unfolding graphs 

  o Recurrent neural networks 

  Applications and Research 

  o Large scale systems 

  o Computer vision 

  o Natural language processing 

  o Boltzmann machines 

Learning Goals 

The student must be able to demonstrate a working knowledge of the foundations of deep learning in neural networks represented by: 

   Deep learning backpropagation 

   Convolutional neural networks 

   Recurrent networks 

   Computer vision large scale system processing 


Written exams and projects will be assigned to test the students’ abilities in the design and development of deep learning systems based on neural networks. The students should be able to experiment and discuss the performance of such systems and evaluate the applicability of the algorithms and neural network architectures. Exams – 40%, projects – 60%.