Sorry...Just my opinion...but right here is where I think you are screwed.
"The plans were mine, and I told the architect our budget, and
he continued to design for 4 months before saying "get a quote". I had no idea what building costs were and
assumed that he would tell me if the plans were more than my budget. Though
I was astounded that after all those months of work, and $20,000+ in fees, that the building was at twice budget, I was
even more astounded when he told me he had expected it to cost even more."
Needed to be way more proactive......