My lakefront house was and still is way below prime in appearance. In order to purchase it, the bank would only take 1/3, the owners took 1/3 back in a 2nd, and I had to come up with the another 1/3, which I didn't have. So I had to borrow some from family. After decades and decades and.... almost have it paid off. Then I can work on making it a place I can invite friends to.
Bottom line, it could be a long hard tow, but if you get what you want now, it will be worth it in the long run