A Brooklyn, New York native, Randolph played for 6 teams during his 18-year MLB career, most notably spending 13 seasons with the Yankees (1976-1988). While in the Bronx, he played pivotal roles as the Yankees won the 1977 and 1978 World Series and he was also on the 1976 squad which lost to the Cincinnati Reds in the World Series.

Randolph was named to the American League All-Star team five times (1976, 1977, 1980, 1981 and 1987). During the 1980 season, he led the majors in walks with 119 while also earning a Silver Slugger award. He was Yankees team co-captain with Ron Guidry from 1986 to 1988. When he left the Yankees after the 1988 season, he was one of only four Yankees with more than 1,000 walks in franchise history, joining Babe Ruth, Mickey Mantle and Lou Gehrig.

Randolph also played for the Pittsburgh Pirates (1975, his rookie season), Los Angeles Dodgers (1989-90), Oakland Athletics (1990), Milwaukee Brewers (1991) and New York Mets (1992). He was named an All-Star in 1989 while with the Dodgers and batted .327, third in the American League, in 1991 with the Brewers.

Upon his retirement as a player, Randolph pivoted to coaching and was a third base and bench coach for the Yankees from 1994 to 2004, during which time the team won four World Series (1996, 1998-2000). He was manager of the Mets from 2005 to 2008, with his 2006 squad posting a league-best 97-65 record before losing to the St. Louis Cardinals in the National League Championship Series.

Randolph later was a coach with the Brewers (2009-2010) and Baltimore Orioles (2011) and was a coach for Team USA in the 2013 and 2017 World Baseball Classics.

Heading into the 2026 season, Randolph ranked fourth among Yankees all-time in stolen bases, sixth in walks, ninth in both WAR and runs scored, 12th in games played and 13th in hits.