2.4 Constraints and Lagrange multipliers

Single constraint

In some cases, the problem of finding a stationary point does not generalize to any virtual displacement, as constraints between coordinates limit the choice of direction for \(\vec{\alpha}\). For example, in the case finding the minimum of the potential energy of a marble in a bowl, we would have a constraint equation in the form:

\begin{align} x^2+y^2+z^2=(R-r)^2 \tag{2.17}\end{align}

where \((x,y,z)\) are the position of the center of mass of the ball, \(R\) is the radius of the bowl, and \(r\) is the radius of the ball. The ball is constrained to be in the bowl, so we cannot just consider any point in space as the minimum of the potential energy.

In general then, we seek to minimize the function \(F(q_1, q_2, \dots, q_n)\), subject to some constraint equation(s) between the coordinates:

\begin{align} f(q_1, q_2, \dots, q_n)=0 \tag{2.18}\end{align}

The obvious (and valid) way to handle this is to use the constraint equation to eliminate one of the variables from \(F\) and work with n-1 independent (generalized) coordinates. However, it may not be convenient to eliminate a coordinate, and there might not be an obvious choice of which coordinate should be the “dependent” one. Lagrange’s method of multipliers allows one to preserve all the coordinates while including the constraints (or “auxiliary conditions”). We start by finding the stationary point subject to a single auxiliary condition. We have:

\begin{align} \delta f=\frac{\partial f}{\partial q_1}\delta q_1 + \frac{\partial f}{\partial q_2}\delta q_2 +\dots+\frac{\partial f}{\partial q_n}\delta q_n=0 \tag{2.19}\end{align}

In order to have a stationary point of \(F\), we also have:

\begin{align} \delta F=\frac{\partial F}{\partial q_1}\delta q_1 + \frac{\partial F}{\partial q_2}\delta q_2 +\dots+\frac{\partial F}{\partial q_n}\delta q_n=0 \tag{2.20}\end{align}

however, unlike in the un-constrained case, the \(\delta q_i\), are not independent, thus this no longer implies that each term (and hence each partial derivative) is zero. Since the two above equations are zero, we can create a linear combination of them, which will still equal zero:

\begin{align} &\delta F+\lambda \delta f=\frac{\partial F}{\partial q_1}\delta q_1 + \frac{\partial F}{\partial q_2}\delta q_2 +\dots+\frac{\partial F}{\partial q_n}\delta q_n + \lambda (\frac{\partial f}{\partial q_1}\delta q_1 + \frac{\partial f}{\partial q_2}\delta q_2 +\dots+\frac{\partial f}{\partial q_n}\delta q_n)=0\\ &\sum_{i=1}^{n} \left(\frac{\partial F}{\partial q_i}+ \lambda \frac{\partial f}{\partial q_i}\right)\delta q_i=0 \tag{2.21}\end{align}

where \(\lambda\) is called a “Lagrange multiplier”, and is not known a priori. Note again that only the sum is zero, not the individual terms. We now choose \(\lambda\) so that the n-th term is zero:

\begin{align} \frac{\partial F}{\partial q_n}+ \lambda \frac{\partial f}{\partial q_n}=0 \tag{2.22}\end{align}

Given the chosen value of \(\lambda\), the sum now contains one less term:

\begin{align} \sum_{i=1}^{n-1} \left(\frac{\partial F}{\partial q_i}+ \lambda \frac{\partial f}{\partial q_i}\right)\delta q_i&=0 \tag{2.23}\end{align}

At this point however, we effectively imposed the constraint to eliminate \(\delta q_n\); the remaining coordinates can now be varied independently (“freely”), so that the remaining terms (given \(\lambda\)) are individually zero:

\begin{align} \frac{\partial F}{\partial q_i}+ \lambda \frac{\partial f}{\partial q_i}=0 \text{ (i=1, 2, …n-1)} \tag{2.24}\end{align}

We thus have \(n-1\) equations as above, and 1 equation to determine \(\lambda\), resulting in \(n\) equations to find the point where \(F\) is stationary given the auxiliary condition \(f=0\). By inspection, those equations are the same and the distinction between dependent and independent variables vanishes!

This method can thus be written a little bit more generally by considering all the \(\delta q_i\) as independent instead of (artificially) eliminating \(\delta q_n\), and considering the variation:

\begin{align} \delta F + \lambda \delta f \tag{2.25}\end{align}

We have:

\begin{align} \delta (F+\lambda f) &= \delta F+ (\delta \lambda) f + \lambda (\delta f)\\ &=\delta F + \lambda \delta f \tag{2.26}\end{align}

since \(f=0\). We can then construct a new function, \(\bar{F}\):

\begin{align} \bar{F}=F + \lambda f \tag{2.27}\end{align}

and consider the stationary points of \(\bar{F}\) without having to worry about auxiliary conditions, thus treating all variations of the coordinates as unconstrained (“free variations”). In this situation, we have n equations by setting the partial derivatives of \(\bar{F}\) to zero and 1 equation from the constraint equation \(f=0\). This allows us to solve for the \(n\) values of \(q\) at the minimum and the unknown \(\lambda\).

Example 2-3

Calculate the dimensions, \(x,y,z\) of a box that maximizes the volume for a fixed surface area of 2A.

We wish to maximize the volume of a box, given by:

\begin{align*} F(x,y,z)=xyz \end{align*}

subject to the constraint that the area is equal to 2A:

\begin{align*} 2xy+2xz+2yz=2A \end{align*}

Therefore, the constraint can be written as:

\begin{align*} f(x,y,z)=xy+xz+yz-A=0 \end{align*}

We thus introduce a Lagrange multiplier, \(\lambda\), and minimize

\begin{align*} \bar{F}=F+\lambda f= xyz+\lambda(xy+xz+yz-A) \end{align*}

This gives us the 3 equations (one for each coordinate):

\begin{align*} \frac{\partial \bar{F}}{\partial x}&=yz+\lambda(y+z)=0\\ \frac{\partial \bar{F}}{\partial y}&=xz+\lambda(x+z)=0\\ \frac{\partial \bar{F}}{\partial z}&=yx+\lambda(y+x)=0 \end{align*}

Together with the constraint equation, this gives 4 equations and 4 unknowns (\(x,y,z,\lambda\)). We can multiply each of the 3 equations by \(x,y,z\) respectively to obtain:

\begin{align*} xyz+x\lambda(y+z)=0\\ yxz+y\lambda(x+z)=0\\ zyx+z\lambda(y+x)=0 \end{align*}

From the first two equations:

\begin{align*} \therefore x\lambda(y+z)=y\lambda(x+z)\\ \lambda x= \lambda y \end{align*}

giving us the possibility that either \(\lambda=0\) or \(x=y\). If \(\lambda=0\) anyone of the above equations imply that \(xyz=0\), which corresponds to a volume of zero. In principle, our method will only give us a stationary value, and a volume of zero is indeed a minimum of the volume. However, that is not the choice of interest for us, so we choose \(x=y\). Combining the other two equations gives us that \(x=z\), showing us that the result is a cube. We can use the constraint equations to find the dimensions of the cube:

\begin{align*} f(x,y,z)=xy+xz+yz-A=3x^2-A=0\nonumber\\ \therefore x=\sqrt{\frac{A}{3}} \end{align*}

We found our solution and did not need to explicitly solve for the Lagrangian multiplier. Also note that our volume function itself has no maximum, however, the auxiliary condition results in a stationary value that is other than the minimum.

Multiple constraints

The treatment is similar when we have multiple auxiliary conditions (say \(k\) such constraints):

\begin{align} f_1(q_1, q_2, \dots, q_n)&=0\\ f_2(q_1, q_2, \dots, q_n)&=0\\ \dots\\ f_k(q_1, q_2, \dots, q_n)&=0 \tag{2.28}\end{align}

where we now have \(n-k\) degrees of freedom. The variation of the auxiliary conditions are:

\begin{align} \delta f_i=\frac{\partial f_i}{\partial q_1}\delta q_1 + \frac{\partial f_i}{\partial q_2}\delta q_2 +\dots+\frac{\partial f_i}{\partial q_n}\delta q_n=0 \tag{2.29}\end{align}

Again, we wish to find the stationary point of \(F\), leading to:

\begin{align} \delta F = \sum_{i=1}^{n}\frac{\partial F}{\partial q_i}\delta q_i=0 \tag{2.30}\end{align}

We now introduce a Lagrange multiplier \(\lambda_i\) for each of the \(k\) auxiliary conditions \(f_i\), so that the following linear combination is still zero:

\begin{align} \delta F + \lambda_1 \delta f_1 +\lambda_2 \delta f_2 +\dots+\lambda_k \delta f_k&=0\\ \sum_{i=1}^{n}\left(\frac{\partial F}{\partial q_i}+\lambda_1 \frac{\partial f_1}{\partial q_i}+\dots +\lambda_k \frac{\partial f_k}{\partial q_i}\right)\delta q_i=0 \tag{2.31}\end{align}

We now choose to solve for the \(k\) Lagrange multipliers by eliminating the last \(k\) terms in the sum, leading to \(k\) equations for the \(k\) unknown \(\lambda\):

\begin{align} &\frac{\partial F}{\partial q_n}+\lambda_1 \frac{\partial f_1}{\partial q_n}+\dots +\lambda_k \frac{\partial f_k}{\partial q_n}=0\\ &\frac{\partial F}{\partial q_{n-1}}+\lambda_1 \frac{\partial f_1}{\partial q_{n-1}}+\dots +\lambda_k \frac{\partial f_k}{\partial q_{n-1}}=0\\ &\frac{\partial F}{\partial q_{n-2}}+\lambda_1 \frac{\partial f_1}{\partial q_{n-2}}+\dots +\lambda_k \frac{\partial f_k}{\partial q_{n-2}}=0\\ &\dots \\ &\frac{\partial F}{\partial q_{n-k+1}}+\lambda_1 \frac{\partial f_1}{\partial q_{n-k+1}}+\dots +\lambda_k \frac{\partial f_k}{\partial q_{n-k+1}}=0 \tag{2.32}\end{align}

The rest of the \(n-k\) coordinates in equation ? can now be varied freely, so that each corresponding terms in the sum must vanish:

\begin{align} \sum_{i=1}^{n-k}\left(\frac{\partial F}{\partial q_i}+\lambda_1 \frac{\partial f_1}{\partial q_i}+\dots +\lambda_k \frac{\partial f_k}{\partial q_i}\right)\delta q_i=0 \tag{2.33}\end{align}

thus giving us \(n-k\) equations:

\begin{align} &\frac{\partial F}{\partial q_1}+\lambda_1 \frac{\partial f_1}{\partial q_1}+\dots +\lambda_k \frac{\partial f_k}{\partial q_1}=0\\ &\frac{\partial F}{\partial q_2}+\lambda_1 \frac{\partial f_1}{\partial q_2}+\dots +\lambda_k \frac{\partial f_k}{\partial q_2}=0\\ &\dots \\ &\frac{\partial F}{\partial q_{n-k}}+\lambda_1 \frac{\partial f_1}{\partial q_{n-k}}+\dots +\lambda_k \frac{\partial f_k}{\partial q_{n-k}}=0 \tag{2.34}\end{align}

Again, by inspection, these equations are the same as those in equation ?, removing any distinction between dependent and independent variables. We can thus construct the function \(\bar{F}\):

\begin{align} \bar{F}=F+\lambda_1 f_1+\lambda_2 f_2 +\dots +\lambda_k f_k \tag{2.35}\end{align}

and treat all of the coordinates in the same way. That is, instead of performing the variation of \(F\) subject to the auxiliary conditions, we can perform the variation on \(\bar{F}\) and “ignore” the auxiliary conditions. This gives us the \(n\) equations:

\begin{align} \frac{\partial F}{\partial q_i}+\lambda_1 \frac{\partial f_i}{\partial q_i}+\dots +\lambda_k \frac{\partial f_k}{\partial q_i}=0 \text{ (i=0…n)} \tag{2.36}\end{align}

together with the \(k\) constraint equations to determine the \(n\) coordinates \(q_i\) and the \(k\) Lagrange multipliers.