It is known that the Sense of Agency is related to time lag between sensory feedbacks including vision and their causal movements. Most of previous studies about the Sense of Agency evaluated effects of time lag to the Sense of Agency on a single movement like pushing a button. The effects of the lagged visual feedback to the Sense of Agency on a continuous movement, however, were still unclear. The aim of our study was to investigate the effect of time lag to the Sense of Agency on participants' continuous movements, different from the single movement in previous studies. The participants manipulated a mouse continuously in the direction of right and left with their hand, and observed a cursor on a monitor screen controlled by the mouse. The cursor movements had some time lag to the mouse movements. The participants answered their subjective Sense of Agency on the control of the cursor. The result showed that the participants lost their Sense of Agency as the time lag increased, and the averaged value of Point of Subjective Equality (PSE), where the participants lost their Sense of Agency at the rate of 50 %, was estimated at 462 ms. The PSE value was close to the value on the single movement in the previous study. This result suggests that the effect of the lagged visual feedback to the Sense of Agency is not specific to types of bodily movements.