Amino acid levels are known to regulate anabolic and catabolic pathways. Here, the authors report that amino acids also affect membrane trafficking by stimulating endosome-to-Golgi retrograde trafficking and regulating cell surface localization of certain Golgi proteins through Ragulator and Arl5.