Transforming Planes
If we have a plane vector n = [a, b, c, d] which describes a plane then for any point p = [x, y, z, 1] in that plane the follow equation holds:
nt p = ax + by + cz + d = 0
If for a point p on the plane, we apply an invertible transformation R to get the transformed point p1, then the plane vector n1 of the transformed plane is given by applying a corresponding transformation Q to the original plane vector n where Q is unknown.
p1 = R p
n1 = Q n
We can solve for Q by using the resulting plane equation: n1t p1 = 0
Begin by substituting for n1 and p1: (Q n)t (R p) = 0
nt Qt R p = 0
If Qt R = I then nt Qt R p = nt I p = nt p = 0 which is given.Qt R = I
Qt = R-1
Q = (R-1)t
Substituting Q back into our plane vector transformation equation we get: n1 = Q n = (R-1)t n