Tidal streams are extended probes of the dark matter distribution within galactic halos. These halos are clumpy and evolving, but traditional methods of their gravitational potential recovery assume they are smooth and static. Here we assess the bias introduced by the assumption of a smooth and time-invariant halo using a sample of streams formed in the Via Lactea simulation. We present a novel method for gravitational potential recovery based on forward modeling of tidal streams and MCMC sampling of the parameter space. We find a median bias of up to 20% in total halo mass estimates when modeling the Via Lactea streams and assuming a parametric potential form. The bias depends on the choice of a parametric potential, and increases for streams with smaller perigalacticons. On the other hand, there is no such bias when using the same potential recovery method on a control sample of streams formed in analytic potentials. Mass estimates of the Milky Way using the known streams and assuming a smooth and static potential are affected by this bias on the order of 50%.