It is shown that when measuring the phase difference of oscillations under nonstationary conditions in systems for optical investigation of the atmosphere and ocean, separation of the useful signal from a mixture with a nonstationary fluctuating process having the 1/f type spectrum is required. Rejection algorithms for quasideterministic nonstationary current average processes with the 1/f type spectrum are proposed and examined. It is found that at rejection of fluctuating processes with the 1/f type spectrum under nonstationary conditions, detection of harmonic phase signals with an amplitude of 10-7 rad is possible under long-term relative instability of laser frequency up to 10-9 and short-term relative instability less than 10-24.