The song is closely associated with the former governor of New York, Al Smith. Smith grew up in a poor family on the Lower East Side and was the descendant of Irish, Italian, and German immigrants. His background in Lower Manhattan informed his whole political life. He fought for progressive urban causes and was the Democratic nominee for President in 1928. Smith represented an urban, immigrant, and Catholic quadrant of America, and his choice of "The Sidewalks of New York" for his campaign theme song cemented the song's popularity. It represented Smith's background and politics well, but it also put his weaknesses as a candidate in sharp relief. Ultimately, Smith showed poorly in the 1928 election in large part because of his Catholic religion, urban mien, and opposition to Prohibition. In fact, his Catholicism alone was a source of vitriol and hatred. The Hoover campaign subtly made sure that all the Anti-Catholic sentiment in rural America was directed straight at Smith.
It wouldn't be all downhill for Smith after that though. Immediately after his unsuccessful Presidential bid, he became President of the Empire State, Inc, the developers of the Empire State Building. In his later years, Smith wasn't just the face of New York City, but of it's tallest building too. Here is a great video of an old 1928 pressing of the song made for Al Smith's campaign and sung by Willie Walker and His Gang.