Alexander Culbertson Swartz designed this Georgian Revival residence for Frank Rehorn, a pioneer Fresno building contractor and vocal member of the anti-union organizations, the Citizen's Alliance and the Builder's Exchange. Repeated strikes caused completion of the house to be delayed several times.

Mr. Rehorn occupied the house until his death in 1916 when it was sold to H. H. Holland. The Holland family lived here until 1941 then sold the house to the Roman Catholic diocese. Subsequently the house was a convent for the Sisters of the Holy Cross, the Villa Carmel Home for unwed mothers, and a communal crash pad for college students.

In 1976,the house was rescued from terminal decay by architects Allen Y. Lew and William E. Patnaude who purchased and rehabilitated it.