AI Termcirca 1951· Added Jun 1, 2026
Gradient Descent
A numerical optimization technique used to minimize the cost function in machine learning models.
Gradient Descent is an iterative optimization algorithm for finding the minimum of a function. In machine learning, it is used to minimize a loss function by iteratively adjusting parameters along the negative direction of the gradient of the function. The size of these adjustments is determined by the learning rate—a crucial hyperparameter that defines how quickly or slowly models converge to a solution.
Examples
- Training neural networks where weights are adjusted using backpropagation and gradient descent.
- Optimizing logistic regression models by finding the best-fit line or decision boundary.
Common misconceptions
- It always finds the global minimum. Most times it finds a local minimum, especially in non-convex spaces.
- The learning rate should always be small. In practice, too small a learning rate can lead to slow convergence.
Related terms
Want more like this?
Open the full library
Fresh AI mastery content every 2 hours.