MathLabs

Applied and computational mathematics

Discrete optimization

Optimization over a finite or countable set of choices, such as scheduling or routing problems.

IntuitionWhat Is Discrete Optimization?

Imagine you must pick a route for a delivery truck, an assignment of workers to shifts, or a subset of projects to fund. Unlike smooth curves, the choices here come in whole, indivisible units: you either send the truck down a street or you do not, a worker is either on shift or not. Discrete optimization is the mathematics of finding the best choice among a finite (or countably infinite) collection of nn discrete possibilities, instead of sliding continuously along a curve.

Network diagram with a highlighted route among several nodes and edges.
A delivery network: nodes are stops and edges are roads with a cost. The highlighted path shows one discrete choice out of finitely many possible routes.

SchoolFrom Continuous to Discrete: Choosing Whole-Number Solutions

Definition: Integer Linear Program (ILP)

An integer linear program maximizes a linear objective cTxc^T x subject to linear constraints Ax≤bAx \le b, where every decision variable must be a non-negative integer: x∈Z≥0nx \in \mathbb{Z}^n_{\ge 0}. Many real problems — how many trucks to buy, which projects to select — are naturally integer, so rounding a continuous solution is not good enough.

max⁡ cTx s.t. Ax≤b, x∈Z≥0n\max\ c^T x \ \text{s.t.}\ Ax \le b,\ x \in \mathbb{Z}^n_{\ge 0}

Here cTxc^T x is the objective value (profit, cost, coverage) to maximize, Ax≤bAx \le b packs all the resource limits (budget, capacity, time) into one matrix inequality, and x∈Z≥0nx \in \mathbb{Z}^n_{\ge 0} says every coordinate of the solution vector must be a non-negative whole number. Dropping the integrality condition gives the LP relaxation below.

max⁡ cTx s.t. Ax≤b, x∈R≥0n\max\ c^T x \ \text{s.t.}\ Ax \le b,\ x \in \mathbb{R}^n_{\ge 0}
ILP, its LP relaxation, and branch-and-bound compared
MethodFeasible setComplexity
Integer programx∈Z≥0nx \in \mathbb{Z}^n_{\ge 0}NP-hard in general
LP relaxationx∈R≥0nx \in \mathbb{R}^n_{\ge 0}Polynomial (simplex/interior-point)
Branch-and-boundSplits the LP relaxation into subproblemsWorst-case exponential, fast in practice

UndergraduateExact Guarantees: Max-Flow Min-Cut and the Branch-and-Bound Bound

In any flow network with source ss and sink tt and non-negative edge capacities, the maximum value of an ss-tt flow equals the minimum capacity over all ss-tt cuts.

Why is it true?

It turns a maximization problem (find the biggest flow) into an equivalent minimization problem (find the smallest bottleneck), so a single cut gives an instant, checkable certificate that a flow is already optimal — the same primal-dual idea that powers LP duality and branch-and-bound pruning.

Proof

Weak duality first. Take any ss-tt cut SS,TT with ss in SS and tt in TT, and any feasible flow ∣f∣|f|. Every unit of flow leaving ss must eventually cross from SS to TT to reach tt, and by conservation the net flow across the cut equals the total flow value. Since edges from TT back to SS can only reduce this net amount, the flow value can never exceed the sum of capacities of edges leaving SS, i.e. ∣f∣|f| is at most cap(S,T)\mathrm{cap}(S,T) for every cut.

Now run the Ford-Fulkerson procedure to completion: repeatedly find a path from ss to tt in the residual graph (remaining capacity plus reversible flow already sent) and push as much flow as the tightest edge on that path allows. Because capacities are assumed integer or rational and strictly decrease along the path used, this process terminates after finitely many augmentations.

At termination no augmenting path exists. Let SS be the set of nodes reachable from ss in the final residual graph, and TT the rest (so tt is in TT since no path reaches it). Every edge from SS to TT in the original network must be fully saturated (otherwise it would still have residual capacity, extending SS), and every edge from TT back to SS must carry zero flow (otherwise its reverse residual edge would extend SS).

Summing flow conservation over all nodes in SS shows the total flow value equals exactly the capacity of the saturated forward edges minus the zero backward flow, which is precisely cap(S,T)\mathrm{cap}(S,T) for this particular cut. Combined with the upper bound from the first paragraph, this cut achieves the minimum possible capacity, so the maximum flow value equals the minimum cut capacity.

For a maximization integer linear program, the optimal value of its LP relaxation is an upper bound on the optimal value of the integer program; consequently, branch-and-bound may discard a subproblem whenever its LP relaxation bound is no better than the best integer solution found so far, without ever discarding the true optimum.

Why is it true?

Solving the LP relaxation is fast (polynomial time), so it gives a cheap upper bound to compare against the best integer solution found so far; if a branch cannot possibly beat that bound, exploring it further would be wasted work, which is exactly what makes branch-and-bound practical on large integer programs.

Proof

Let x∈Z≥0nx \in \mathbb{Z}^n_{\ge 0} denote the integer feasible set and x∈R≥0nx \in \mathbb{R}^n_{\ge 0} the (larger) LP-relaxed feasible set of the same constraints Ax≤bAx \le b. Because dropping the integrality requirement only removes constraints, every integer-feasible point is also LP-feasible, so the integer feasible set is a subset of the LP-relaxed feasible set.

Since both problems maximize the same linear objective cTxc^T x and the integer program's feasible set is contained in the LP relaxation's feasible set, the best value achievable over the smaller set cannot exceed the best value achievable over the larger set. Hence the LP relaxation optimum is an upper bound on the ILP optimum.

Branch-and-bound builds a search tree by picking a variable with a fractional value in the current LP-relaxed solution and creating two child subproblems that add x1≤1x_1 \le 1 or x1≥2x_1 \ge 2 respectively; every integer-feasible point satisfies exactly one of these two conditions, so no integer solution is ever lost by this split, and repeating the split keeps partitioning the space without gaps.

At each node, solving the LP relaxation of that subproblem gives an upper bound for every integer solution still reachable below it, by the same argument as above applied to the smaller constraint set. If this bound is no larger than the value of the best integer solution found anywhere in the tree so far, no descendant of this node can improve on the current best, so the whole subtree may be pruned while still guaranteeing the true optimum is found elsewhere in the tree.

UndergraduateReal-World Applications and Worked Examples

Discrete optimization runs the logistics of the modern world: airlines solve crew-scheduling ILPs every day, chip manufacturers use graph-coloring and max-flow ideas to route wires, and delivery companies solve vehicle-routing problems that are, at their core, branch-and-bound searches over an integer program. Two worked examples below show the LP-relaxation-plus-branching pattern and the max-flow min-cut pattern in action.

Example: Branch-and-bound on a small ILP

Maximize x1x_1 + x2x_2 subject to 2x1+x2≤52x_1 + x_2 \le 5 and x1+2x2≤5x_1 + 2x_2 \le 5, with x1x_1 and x2x_2 non-negative integers.

Solution

Step 1: Solve the LP relaxation. The two constraints are tightest at their intersection: solving 2x1+x2=52x_1 + x_2 = 5 and x1+2x2=5x_1 + 2x_2 = 5 together gives x1=x2=53x_1 = x_2 = \frac{5}{3}, with objective value 103≈3.33\frac{10}{3} \approx 3.33.

Step 2: Since the LP-optimal x1x_1 is fractional, branch on it: one child adds x1≤1x_1 \le 1, the other adds x1≥2x_1 \ge 2.

Step 3: On the branch with x1≤1x_1 \le 1, the constraints force x2x_2 up to 22 at x1x_1 = 11, giving the integer point x1=1, x2=2x_1 = 1,\ x_2 = 2 with x1+x2=3x_1 + x_2 = 3.

Step 4: On the branch with x1≥2x_1 \ge 2, the constraints force x2x_2 down to 11 at x1x_1 = 22, giving the integer point x1=2, x2=1x_1 = 2,\ x_2 = 1, again with x1+x2=3x_1 + x_2 = 3.

Step 5: Both branches already produce integer solutions with the same objective value, so no further branching is needed; the ILP optimum is 33, strictly below the LP relaxation bound of 103≈3.33\frac{10}{3} \approx 3.33 as the theorem predicts.

Example: Maximum flow in a small shipping network

A network has nodes ss, a, b, tt with edge capacities ss→a: 1010, ss→b: 55, a→b: 44, a→tt: 88, b→tt: 99. Find the maximum flow from ss to tt.

Solution

Step 1: Push flow along ss→a→tt. The bottleneck is min⁡(10,8)=8\min(10,8)=8, so send 88 units; the ss→a edge has 22 units of capacity left.

Step 2: Push flow along ss→b→tt. The bottleneck is min⁡(5,9)=5\min(5,9)=5, so send 55 units; the b→tt edge has 44 units left.

Step 3: Push flow along ss→a→b→tt using the leftover capacity: min⁡(2,4,4)=2\min(2,4,4)=2, so send 22 more units. Now both edges out of ss are fully used.

Step 4: Total flow sent is 8+5+2=158+5+2=15.

Step 5: No augmenting path remains since both edges leaving ss are saturated. The cut that isolates ss alone has capacity 10+5=1510+5=15, which matches the flow found — by the max-flow min-cut theorem this confirms 1515 is optimal.

If the LP relaxation of a maximization ILP has optimal value 42.342.3, what can we conclude about the ILP's optimal value?

A courier company must decide, for each of several candidate routes, whether to use it or not, to minimize total cost subject to coverage constraints. Which formulation fits best?

In a flow network, the minimum ss-tt cut has capacity 1515. What is the maximum flow value?

In branch-and-bound for a maximization ILP, when should a subtree be pruned?

References

  1. Alexander Schrijver (2003). Combinatorial Optimization: Polyhedra and Efficiency
  2. Anna R. Karlin, Nathan Klein, Shayan Oveis Gharan (2021). A (Slightly) Improved Approximation Algorithm for Metric TSP · arXiv:2007.01409