The evolution of the rest-frame v-band luminosity function from z=4: a constant faint-end slope over the last 12 gyr of cosmic history