In this paper, we investigate an optimal control problem of linear quantum systems despite feedback delays. The optimal controller, which is of the Smith predictor form, and an analytical expression of the best achievable performance are derived by applying existing results for control of classical (non-quantum) I/O delay systems. Then, we analyze the performance degrading effect caused by feedback delays in an illustrative example of a quantum free particle. In particular, we give a new insight for a typical experiment setup. This is accomplished by using the performance limit expression mentioned above.