According to police, the little boy was walked into the store by a man who handed him over to staff and then quickly left on foot.
The child was unable to name his parents or any relatives, but was able to indicate his age by holding two fingers up.
Police were called to the store and quickly arrived, but were unable to locate the man.
Officers released CCTV images of a man and woman they said were connected to the incident; after a flurry of tips from the public, and with assistance of FBI offices in both Mississippi and Tennessee, had arrested one suspect by 3:30 PM.
One Goodwill staff member, David, told WREG that the boy had no socks on, his nose was running and he had insufficient clothes on for the weather; external temperatures were just 38 degrees at the time.
"The guy came walking from the gas station with a little boy," he said. "He walked into the garage where we get our donations."
"He asked can we take the baby in 'cause the mother abandoned him... he just said he couldn't take him with him and left," he said. "He wouldn't leave his name."
The whole interaction, he said, took five or six minutes.
Waiting for your permission to load the Facebook Post.
David said his first thought was "what is he doing with a baby out here this cold?'"
Nevertheless, he said the boy seemed in good physical condition, and was in good spirits, laughing as he let him play with some of the donated toys.
Staff bought him food while they waited for police, who took him into child protective services.
"It just messes with your head," David said. "Somebody's gonna have to pay for it."