When my girlfriend and I went, we arrived at like 10AM and left on a 2AM or so train that night.
So we had some time to kill around midnight while all the quick to go to touristy things were closed. We went down to South Ferry and just chilled on a bench watching the Statue Liberty out on the water. After a really awesome day, it just felt unreal. There was this massive, bright city behind us and all around, but everything was so quiet. There was no sounds but the waves lapping the sea wall and the occasional casting of a line by the dozens of people out fishing. It was such a great way to end the day.
We took the 1, I think, from Times Square after everything there closed up. The walk from South Ferry station was kinda creepy, through this dark park. I started to worry someone would jump out while we crossed through some park, lmao. Once we got to where people were fishing though, everyone seemed really cool. Just minding their own business taking in some quiet downtime in the city.