Estimators for the coefficients in linear regression which are better than the ordinary least squares estimator are obtained when at least three coefficients are to be estimated. In orthogonal linear regression the loss function is the sum, or weighted sum, of the componentwise mean squared errors. Some of the new estimators have interpretations as estimators which depend upon preliminary tests of significance. These estimators may be especially appropriate when the explanatory variables fall into two sets or are ordered, as in polynomial regression or regression on principal components.