Stellar feedback plays a crucial role in regulating galaxy formation. However, most cosmological simulations do not include small, but potentially important component of this feedback that is in the forms of cosmic rays. We have implemented a transport scheme for cosmic rays in the code GIZMO and coupled it to the FIRE stellar feedback model to study their effect on galaxy evolution. We find that observations of local galaxies require cosmic rays to escape galactic ISM before they encounter significant energy losses, enabling them to interact with the CGM. Escaping cosmic rays can significantly modify the structure and properties of the CGM, including the gas flows around L* galaxies.