Browsing through some old newspapers at the Library of Congress (site) and it turns out the New Years Ball Drop was an actual daily thing 120 years ago.
There was one in New York and one in DC at the Naval Observatory and a third one started in September 1902 in Atlanta. They all dropped at noon but since this was in that period when Atlanta was still in Central Time, ours dropped at 11. This Wikipedia article and this Times Square article suggest that ship captains used these to reset their time pieces. The Greenwich Observatory apparently had the first one.
At the northeast corner of Broad and Marietta Streets downtown was what was then the Empire Building, which later became C&S bank, Bank of America and now is part of the GSU campus. If you get the chance, the old lobby is beautiful and worth a visit.
Here’s the building at the time, but no evidence of a 10 foot gold ball on top and I haven’t found any photos where you can see it

I love all the awnings and open windows! Anyways here’s the article

The building did house the Atlanta office of the National Weather Service but I wasn’t able to find much about James Rossman and what the government’s official timekeeper really did. Seems like a sweet gig! In 1903 they were on the top floor (suite 1407) right next to King & Spalding’s law office.