Hi there all you smart people.
I am looking for a hand on a mortgage I currently hold. I will give a little back story to hopefully let you see where I am coming from.

I have a summer home (the only property I own) and was unable to get a conventional mortgage on it through a bank. Basically because there is no power, a bad road and a trailer on the property not a house. That means no mortgage insurance places will insurance me for mortgage insurance. There's no getting around that so it's not to do with my question. 

I ended up getting my rich boss to hold the mortgage for me. So he payed the $264,000 that I owed to the previous owners and that is what I now owe him. I really wanted to keep it simple since he is doing me a favor by letting me pay him back slowly so we worked out a deal and he's pretty casual about it. He is holding the mortgage at 6% for me. I am not sure how much of this he is claiming on his taxes so I pay interest payments in cash.  Instead of doing one of those mortgage calculators and getting a whole bunch of weird numbers to pay him, I just took.

Year 1 
$264,000 owing x 6% = $15,840 
So for that year I owed him $15,840 in interest, as well as I could pay any large sum of money I wanted down towards the principal at the end of the year. That way I payed him 3 payments of ($15,840 / 3 = $5,280) for interest. (I did three separate interest payments so I wasn't bugging him looking for him to pay him a wad of cash every month for interest and I didn't have a years worth of interest saved up). Than I payed $34,000 in principal.

Year 2
$230,000 owing x 6% = $13,800
So I did 3 payments of (13,800 / 3 = $4600) and payed another $30,000 in principal.

Year 3
$200,000 owing x 6% = $12,000

I am currently on year 3 and trying to save an extra $30,000 but not sure if i will make it this year. Either way I have been talking with my partner and he keeps saying that I should be re-calculating the mortgage interest payment each month and not yearly. We all know paying your mortgage off bi weekly is better than monthly which is better than yearly. But I feel when I did the calculations, over the year we are only losing out on about 200$. Yes substantial in a way but I am trying to keep this simple and not have to pay the mortgage holder with bills, loonies, and twoonies etc. since I pay him cash for the interest. However my husband thinks that the $200 a year will actually compound hugely over the next 7 years till we are payed off fully on the property. I just don't feel that's right for some reason. It's not like we will put that extra $200 dollars we save a year on interest into the principal we owe since we are trying to keep the numbers round and whole. And I feel like our "term" is yearly... So the $200 a year won't compound like if somebody had a 5 year term? 

Any insight is appreciated. I am finding this all a bit confusing when I use the mortgage calculators and having troubles working them to my weird situation (1 large principal payment a year and thrice yearly interest payments).