Deep Learning Fundamentals

Overview

This hub tracks neural network basics, optimization, architectures, representation learning, regularization, expressivity, generalization, and the core ingredients that feed later research notes.

Core Questions

• Which neural-network concepts show up repeatedly and deserve evergreen notes?
• How do architecture choices, optimization dynamics, representations, and scaling assumptions shape results?
• What do expressivity theorems guarantee, and what do they leave open about training and generalization?
• Where do learned representations matter for interpretability, continual adaptation, robotics, or sequential decision-making?
• How do actor-critic and policy-gradient updates behave when other agents are also learning?
• When do replay buffers, critics, or learned representations become stale because the policy population that generated the data has changed?
• When does learned communication encode task-successful but non-shared representations?
• When can normalization and regularization replace target networks or large replay buffers in deep TD learning?
• Which tabular RL guarantees are inherited by neural value-learning methods, and which are lost under function approximation?
• Which cooperative-MARL baselines are strong enough that safety or coordination claims should be compared against them?
• When do representation choices, such as frozen image encoders, change apparent MARL algorithm performance?
• How much of an ad hoc teamwork result comes from learned representation, teammate modelling, or task-specific benchmark structure?

Current Research Touchpoints

• Opponent Shaping
• Emergent Communication
• Successful Misunderstandings
• Multi-Agent Non-Stationarity
• Multi-Agent Coordination
• Ad Hoc Teamwork
• Cooperative MARL Benchmarking
• Q-Learning
• Q-Learning Convergence
• Temporal Difference Learning
• Regularized TD Stability and PQN
• Cooperative Multi-Agent Reinforcement Learning
• Value Decomposition Networks
• Multi-Agent PPO
• Continual Learning
• Explainable AI
• Safe Multi-Agent Reinforcement Learning
• AI Evaluation and Benchmarking

Key Concepts

• Continual Learning
• Emergent Communication
• Successful Misunderstandings
• Multi-Agent Non-Stationarity
• Multi-Agent Coordination
• Ad Hoc Teamwork
• Cooperative MARL Benchmarking
• Q-Learning
• Q-Learning Convergence
• Temporal Difference Learning
• Regularized TD Stability and PQN
• Cooperative Multi-Agent Reinforcement Learning
• Value Decomposition Networks
• Multi-Agent PPO
• Explainable AI
• Sigmoid Functions
• Universal Approximation Theorem
• Double Descent
• S-Curves and Saturating Growth