📝 PyTorch

Description

< What is it? >

PyTorch is a Python framework for building and training neural networks. It provides GPU-friendly tensors, automatic differentiation, neural-network modules, data loading, and optimization tools.

< Core training loop >

prediction = model(x)
loss = loss_fn(prediction, y)

optimizer.zero_grad()
loss.backward()
optimizer.step()

• A Tensor stores data and parameters, optionally on an accelerator such as a GPU.
• nn.Module defines a model from reusable layers.
• autograd records operations and computes gradients when loss.backward() runs.
• torch.optim updates parameters using those gradients.

Key points

• Eager execution makes model code ordinary Python, which is convenient to inspect and debug.
• CUDA and MPS support allow the same model code to run on accelerators.
• Its ecosystem supports data pipelines, distributed training, vision, audio, and deployment.

Related ideas

• Backpropagation explains what loss.backward() computes.
• Gradient Descent explains how an optimizer uses those gradients.

Favorites

🏀

Magic

Crash course

• computation graph — the concept, and why reverse-mode is cheap, is on Backpropagation

Pytorch by example: computation graph

Reference

• PyTorch documentation
• PyTorch Tutorials
• Introduction To Neural Networks (geeksforgeeks.org)
• Activation Functions in Neural Networks (geeksforgeeks.org)