Because the measurements of gyro and accelerometer are perturbed by swaying under the condition of swaying base, the initial attitude matrix cannot be obtained from the measurements. As to solve this problem, an alignment method based on gravity acceleration was proposed. And the validity of the proposed algorithm was verified by the stationary base experiment and swaying base experiment. The results show that the algorithm is valuable for engineering application.