Particle Swarm Optimization (PSO) is a population-based metaheuristic proposed by James Kennedy and Russell Eberhart in 1995. Inspired by the social behavior of bird flocks and fish schools, PSO maintains a swarm of candidate solutions (“particles”) that move through the search space. Each particle adjusts its trajectory based on its own best-known position and the best position found by the entire swarm (or its local neighborhood), creating an emergent collective search behavior.
PSO occupies a distinctive niche in the metaheuristic landscape: it is designed primarily for continuous optimization problems — unlike genetic algorithms and tabu search, which are more natural for combinatorial/discrete problems. Its velocity-position update mechanism works directly in real-valued parameter spaces without needing crossover, mutation, or neighborhood operators. This makes PSO a natural fit for engineering design optimization, neural network training, and hyperparameter tuning over continuous variables.
The algorithm is remarkably simple to implement (the core loop is ~15 lines of code), converges quickly to good solutions, and requires only two or three parameters. These properties have made PSO one of the most widely used swarm intelligence methods, with thousands of variants published since its introduction. Python libraries like PySwarms and pymoo provide production-ready implementations.
Each particle i has two vectors in a D-dimensional search space:
x_i: the current candidate solution (a point in ℝD).v_i: the direction and speed of movement (also a vector in ℝD).Each particle also remembers:
p_i — its personal best (pbest): the best position this particle has ever visited.g — the global best (gbest): the best position any particle in the swarm has ever visited.At each iteration, every particle updates its velocity and then its position:
# PSO velocity and position update (per dimension j)
v_ij(t+1) = w * v_ij(t) # inertia
+ c1 * r1 * (p_ij(t) - x_ij(t)) # cognitive (pull toward pbest)
+ c2 * r2 * (g_j(t) - x_ij(t)) # social (pull toward gbest)
x_ij(t+1) = x_ij(t) + v_ij(t+1) # moveWhere:
w — inertia weight: controls how much of the previous velocity is retained. High w → more exploration; low w → more exploitation.c1 — cognitive coefficient: attraction toward the particle’s personal best.c2 — social coefficient: attraction toward the global (or neighborhood) best.r1, r2 — independent random numbers in [0, 1], drawn fresh each dimension, each iteration.Each velocity update is the sum of three forces: inertia (momentum from the current trajectory — exploration), cognitive pull (attraction to the particle’s own best — individual learning), and social pull (attraction to the swarm’s best — collective knowledge). The balance between these forces determines whether the swarm explores broadly or converges tightly. The random factors r1 and r2 add stochasticity that prevents deterministic convergence to suboptimal points.
import numpy as np
def pso(objective, bounds, n_particles=30, max_iter=200,
w=0.7, c1=1.5, c2=1.5):
dim = len(bounds)
lo = np.array([b[0] for b in bounds])
hi = np.array([b[1] for b in bounds])
# Initialize positions and velocities
X = np.random.uniform(lo, hi, (n_particles, dim))
V = np.random.uniform(-(hi-lo), (hi-lo), (n_particles, dim)) * 0.1
# Evaluate initial fitness
fitness = np.array([objective(x) for x in X])
pbest = X.copy()
pbest_fit = fitness.copy()
gbest_idx = np.argmin(fitness)
gbest = X[gbest_idx].copy()
gbest_fit = fitness[gbest_idx]
for t in range(max_iter):
r1 = np.random.rand(n_particles, dim)
r2 = np.random.rand(n_particles, dim)
# Velocity update
V = (w * V
+ c1 * r1 * (pbest - X)
+ c2 * r2 * (gbest - X))
# Position update
X = X + V
X = np.clip(X, lo, hi) # enforce bounds
# Evaluate
fitness = np.array([objective(x) for x in X])
# Update personal bests
improved = fitness < pbest_fit
pbest[improved] = X[improved]
pbest_fit[improved] = fitness[improved]
# Update global best
if fitness.min() < gbest_fit:
gbest_idx = np.argmin(fitness)
gbest = X[gbest_idx].copy()
gbest_fit = fitness[gbest_idx]
return gbest, gbest_fitThe inertia weight w is the most impactful parameter. Common strategies:
w = 0.9 (exploration) and linearly decrease to w = 0.4 (exploitation) over the run. This is the most widely used default, proposed by Shi and Eberhart (1998).w = 0.729 with c1 = c2 = 1.49445 — Clerc and Kennedy’s constriction coefficient, which provides theoretical convergence guarantees.w when the swarm is converging too fast (low diversity); decrease when particles are scattered (high diversity). This eliminates the need to set the decay schedule manually.Clerc and Kennedy (2002) proved that without velocity damping, PSO can diverge — particles accelerate unboundedly. They derived a constriction coefficient χ ≈ 0.7298 that guarantees convergence to a stable fixed point when c1 + c2 > 4. The standard setting χ = 0.7298, c1 = c2 = 2.05 (giving effective coefficients of c1 * χ = c2 * χ ≈ 1.4962) is mathematically equivalent to linear inertia decay but provides a theoretical foundation.
In gbest PSO, every particle is attracted to the single best position in the entire swarm. This converges fast but is prone to premature convergence on multimodal problems. In lbest PSO, each particle only knows about its k nearest neighbors in a ring or graph topology. Information propagates more slowly through the swarm, preserving diversity longer:
Without limits, velocities can grow large, causing particles to overshoot the search space. Two standard fixes:
|v_ij| to V_max, typically set to a fraction of the search range (e.g., V_max = 0.2 * (hi - lo)).PSO is heavily used in electrical engineering, mechanical design, and antenna optimization. It can optimize continuous parameters (dimensions, voltages, impedances) with complex, non-differentiable, or simulation-based fitness functions. For example, PSO has been used to optimize wind turbine blade profiles, where each evaluation requires a CFD simulation — the gradient-free nature of PSO makes it ideal when the objective is a black-box simulator.
PSO can train neural networks by treating all weights as a continuous vector and minimizing the loss function. While gradient-based methods (SGD, Adam) are far more efficient for standard architectures, PSO is used when gradients are unavailable — non-differentiable activation functions, spiking neural networks, or networks with discrete structural choices. PSO has also been applied to optimize network architectures (number of layers, units per layer) as a continuous relaxation.
For tuning continuous hyperparameters (learning rate, regularization strength, dropout rate), PSO provides a parallel search that is more structured than random search but doesn’t require the surrogate modeling of Bayesian optimization. Libraries like Optuna support PSO-based samplers. The swarm naturally balances exploration (early iterations with high inertia) and exploitation (late iterations as the swarm converges).
Binary PSO (BPSO) adapts the algorithm for discrete problems by applying a sigmoid to the velocity and using it as the probability of setting a binary variable to 1 or 0. This is commonly used for feature selection in high-dimensional datasets (genomics, text), where each dimension of the particle represents whether a feature is included.
PySwarms provides a clean API for PSO with built-in topologies, boundary handling, and visualization:
import pyswarms as ps
import numpy as np
# Define objective: Rosenbrock function
def rosenbrock(x):
return np.sum(100 * (x[:, 1:] - x[:, :-1]**2)**2
+ (1 - x[:, :-1])**2, axis=1)
# Set bounds
bounds = (np.full(10, -5.0), np.full(10, 10.0))
# Configure optimizer
options = {'c1': 1.5, 'c2': 1.5, 'w': 0.7}
optimizer = ps.single.GlobalBestPSO(
n_particles=50, dimensions=10,
options=options, bounds=bounds
)
# Run optimization
best_cost, best_pos = optimizer.optimize(rosenbrock, iters=300)
print(f"Best cost: {best_cost:.6f}")
print(f"Best position: {best_pos}")w, c1, and c2.0.9 to 0.4 is the most common default; Clerc’s constriction coefficient χ ≈ 0.73 provides theoretical convergence guarantees.