A robust linear estimation scheme is newly proposed for solving the time delay estimation problem. The conventional linear estimators cannot effectively deal with the stochastic parameter uncertainty included in the measurement model. As a remedy for this problem, the recently proposed non-conservative robust Kalman filter (NCRKF)  can be used for time delay estimation. In spite of using the noisy measurement matrix, the time delay estimation performance of the proposed scheme is by no means inferior to that of the optimal Kalman filter-based method which cannot be realizable in actual situations. Since it assures efficient computation and uses the recursive structure, it should be preferred obviously in real-time applications. Through several computer simulations, the superiority of the proposed method in time delay estimation performance and computational efficiency is demonstrated.