State District Judge Martin Hoffman was scheduled to hold another hearing this month in Dallas on efforts to settle the lawsuit.
The Dallas Morning News reports Hoffman voluntary stepped down. His motion, filed Friday, did not immediately explain why.
Regional Administrative Judge John D. Ovard said he will assign the case to a new judge.
Plans for the Bush library, expected to open in 2013, sparked a lawsuit by two former condominium owners who claim SMU bullied them into selling without disclosing its plans.
SMU denies the allegations and says it purchased the property legally.