Remarkably, diffusion models can be steered towards optimizing given reward functions at inference time, generating samples that are both realistic and tailored to a reward objective. However, when these reward objectives are highly multimodal or compress too much information about the sample, these steering techniques fail spectacularly. We discover that strong latent initializations in noise space offer a new axis for desiging inference-time algorithms that surpass these prior limitations. We propose a simple algorithm (ReGuidance) leveraging this insight and both empirically and theoretically demonstrate its superiority over prior techniques. See the paper for more details!