I am using Bill Pay at my local credit union. It works well in general. The big catch to me is, the money is removed from my account on the day the payee is to receive the check. Other than the payee contacting me, I have no idea if the check is ever cashed. The money disappears from my account. So, if I'm giving to a charity or someone with poor bookkeeping the check could never be cashed and whoever is holding the money - does what? Keeps it? I am not allowed to view the canceled checks and this sounds very fishy to me.
I had a bank that did this, and had to move on to a different bank. I attempted to pay my mortgage through them, and the check got "lost". The money came out, and no mortgage payment was made. Technically it was a web service call that was failing.
I was fortunate that the mortgage company and the checking account people talked and all fees were waived while it was figured out. However, it took three months or something ridiculous like that. In the end I never used that checking account again.
When doing automatic bill pay, I would only do a bank that pulls from your account when the check gets cashed. If I was in your shoes, I would find another bank. inconvenient I know, but probably the best course of action.
In the meantime I would not use their system to make bill payments. Either have the payee pull from your checking account or issue paper checks.
My CU removes the money from my account either on the day I submit the request or on the day they need to send the check depending on the timing I select.
I have had one occurrence where the company didn't cash the check within 30 days. The CU contacted me 30 days after the due date. I contacted the small company and they cashed it the next day. The message from the CU said that if they didn't cash the check by 90 days they would void the check and return the money to my account.
Contact your institution for their procedures.