Deep Learning Overview#
Deep learning is a category of machine learning. Machine learning is a category of artificial intelligence. Deep learning is the use of neural networks to do machine learning, like classify and regress data. This chapter provides an overview and we will dive further into these topics in later chapters.
There are many good resources on deep learning to supplement these chapters.
The introduction the from Ian Goodfellow’s book to be a good intro.
short video series specifically about neural networks that give an applied introduction to the topic.
DeepMind has a high-level video showing what can be accomplished with deep learning & AI.
When people write “deep learning is a powerful tool” in their research papers, they typically cite this Nature paper by Yann LeCun, Yoshua Bengio, and Geoffery Hinton.
A practical and example-driven online book that gives each example in Tensorflow, PyTorch, and MXNet.
Many chemistry-specific examples and information about deep learning in chemistry via the excellent DeepChem project.
The main advice I would give to beginners in deep learning are to focus less on the neurological inspired language (i.e., connections between neurons), and instead view deep learning as a series of linear algebra operations where many of the matrices are filled with adjustable parameters. Of course nonlinear functions (activations) are used to join the linear algebra operations, but deep learning is essentially linear algebra operations specified via a “computation network” (aka computation graph) that vaguely looks like neurons connected in a brain.
nonlinearity
A function \(f(\vec{x})\) is linear if two conditions hold:
for all \(\vec{x}\) and \(\vec{y}\). And
where \(s\) is a scalar. A function is nonlinear if these conditions do not hold for some \(\vec{x}\).