New York City held the first public New Year's Eve celebration in its Times Square in 1904.
The event was organized by the New York Times to celebrate the opening of the newspaper's new headquarters at One Times Square. On December 31, 1904, fireworks were staged around the building, drawing a large crowd to the area then known as Longacre Square. The place was renamed Times Square earlier that year after the newspaper's move.
The famous ball drop came later, on December 31, 1907, welcoming 1908. The newspaper introduced the illuminated ball after city officials banned the fireworks that had been used for the earlier celebration. Thus, 1904 marks the first Times Square New Year's celebration, while 1907 marks the beginning of the ball-drop tradition.
These dates are often blended together in summaries. The distinction matters: the question asks about the public celebration in Times Square, not the first descent of the Times Square Ball.