Depending on the level of threat to the child, refusing to return the child might be a valid option if you think the child is in imminent danger. If possible, speak with an attorney about your situation before taking any action.
Talk with the parent about your concerns, and try to work out a solution that allows the child to stay with you. If the parent has been away a long time, talk about making any transition as easy on the child as possible. You could offer to take the child to the new home for progressively longer visits to the new home for a gradual transition.