Detecting visual features in the environment, such as motion direction, is crucial for survival. The circuit mechanisms that give rise to direction selectivity in a major visual center, the superior colliculus (SC), are entirely unknown. We optogenetically isolate the retinal inputs that individual direction-selective SC neurons receive and find that they are already selective as a result of precisely converging inputs from similarly tuned retinal ganglion cells. The direction-selective retinal input is linearly amplified by intracollicular circuits without changing its preferred direction or level of selectivity. Finally, using two-photon calcium imaging, we show that SC direction selectivity is dramatically reduced in transgenic mice that have decreased retinal selectivity. Together, our studies demonstrate a retinal origin of direction selectivity in the SC and reveal a central visual deficit as a consequence of altered feature selectivity in the retina.