How to build a flying car IV: optimal control

 Let's have a look at the landscape of P-D controllers again:

You will remember it plots the standard deviation of the error, i.e. the difference of where the car was from where we wanted it, above a plane defined by just how hard we push given an error (left side axis) and the slope of the line above which we measure the error (right side). (Note that these numbers are just the coefficients of P and D in the standard formulation of a P-D controller.)

The valley indicating the best results appears to favor higher proportionality and lower, but non-zero, slope. But you can't increase proportionality forever; that would imply infinite force, and your motor is only capable of so much. Which should lead you to guess that we are going to have to look at bang-bang control again.

Here's the phase space again, with a controller that is stuck trying to go up as hard as it can. If, for example, you are sitting perfectly at the origin, position and velocity both zero, you would accelerate upward ad infinitum, to ever-higher altitudes and speeds.

That's represented by the red line in the upper right. But the line we are interested in is the blue one. It represents what happens when you are above the desired point, but decelerate as hard as you can. If you happen to be on the blue line, you will come in to the perfect one-point landing, hitting both zero altitude and zero speed exactly with no spiraling in.

Let's look at that quadrant more closely, pretending we are controlling a lunar lander on its way down to the moon:

Again, if you are on the blue line, perfect landing. If you are below it, you are too low and/or too fast, and you are S.O.L. -- you are going to hit no matter what. On the other hand, if you are above the blue line, you aren't going to hit at all. Remember the engine is on full pushing you up; you're just doing a takeoff from midair.

That's not what you wanted; you want to land. So what do you do?

You're hanging in space over the moon, and you need to get down. So you blast down as hard as you can until you are on the blue line, and then decelerate as hard as you can for a perfect landing. It should be fairly obvious with a little thought that this is the fastest way to get to the zero-zero point.

That is the essence of optimal control. It actually works from anywhere in the phase space, is the fastest possible way to get to zero, and gets you there in just two strokes, with no spiraling in.

In the parlance, the blue line is called the "switching curve."

That is, if your thrust and timing are perfectly exact, and there is no wind. Even with wind, it works quite well, although you hunt around the setpoint a bit:

The reason for the hunting or dithering is subtle. The switching curve is two halves of a parabola, and the closer you get to the origin the closer that gets to a straight line:

... and what's more, the closer the straight line is to absolutely flat! That means that within some small distance from the origin, we lose derivative control and the system orbits, like this:

We can get it back by having a hybrid controller that is mostly bang-bang optimal, but in a small band next to the switching curve, is P-D. We expand the switching curve like this:

placing the proportional region above it on the left, below on the right, and crossing in the middle, preserving an angle where the curve goes flat. The control signal as a function of the phase plane now looks like this:

This works quite well. In a run with no wind,

it does just about what we want from the optimal control in most of the space, with a very fast spiral at the end:

It does quite well with wind, too; in this graph orange is wind, green is the control signal, and blue the result. 

You can see that big sustained gusts push the system into bang-bang mode, but the rest of the time it maintains a nice (and smooth) hold on position with a proportional response. 

In phase space the result is a tight locus most of the time, with the excursions caused by major gusts exhibiting the characteristic clamshell orbits of optimal control.

There is one other phenomenon we need to look at, and that is lag. We noted way back that lag was the major problem for a controller, introducing derivative control to counter it. How well does our hybrid controller stand up to lag?

Here's a map of performance of a hybrid controller on a landscape with the width of the switching-curve band going up to the left and the amount of lag going up to the right. You can see that the controller gets better as it gets closer to the pure optimal controller, i.e. with a thinner band, except that lag just completely kills it. 

Luckily it's a fairly simple relationship, and if you know what to look for, it's not hard to design a good controller for the particular parameters of your system.

There's a lot more to control theory; I've barely scratched the surface here. Hopefully I have given you enough intuition to get a leg up on simple control system design problems.