Are pulsars born with a hidden magnetic field?