Based upon where I live, after ascertaining that the child was young and no coat...I would have dialed 911 and let the police handle the 2nd part. I would have waited on the sidewalk until the police arrived.
Let the Dad explain himself to the cops.
The cops may or may not turn the case over to Social Services who are trained to handle family crisis or just bad parenting. The family may have a record with Social Services. You may not be the first Good Samaritan to report the Dad or parents.
And if nothing is wrong with the whole situation, simply being a thing that can happen to anyone...then the Dad is still "on notice" regarding his child.
And, let me not get shot by a nutty, maybe drugged-out, or drunk, irate Dad.