When embodying a virtual avatar in immersive VR applications where body tracking is enabled, users typically are and feel in control the avatar movements. However, there are situations in which the technology could be tweaked to flip this relationship so that an embodied avatar could affect the user's motor behavior without users noticing it. This has been shown in action retargeting applications and motor contagion experiments. Here we discuss a different way in which an embodied avatar could implicitly drive users movements: the self-avatar follower effect. We review previous evidences and present new experimental results showing how, whenever the virtual body does not overlay with their physical body, users tend to unconsciously follow their avatar, filling the gap if the system allows for it. We discuss this effect in the context of the relevant neuroscientific literature, and propose a theoretical account of the follower effect at the intersection of motor control and inference theories.