tl;dr: Because you have been aggressively paying off your mortgage, you don't have much time left on your mortgage. If you continue to pay off your mortgage at this rate, it will most likely not be worth it for you to refinance.
When evaluating a refinance, you need to calculate the payback time. This is the time it will take for the money you are saving on interest with your new rate to make up for the cost of refinancing (closing costs).
Here is your original amortization schedule for your current mortgage (monthly payment of $910, total interest cost $41,800): (original mortgage)
However, you have been paying early. (Nice!) If you were to make the normal $910 monthly payment from here on out and not pay anything early anymore, your mortgage will end on March 2021, which is 5 years early. (current mortage with previous early payment)
If you continue to pay early at the rate that you have been paying (I estimate that you have been paying an extra $675 per month), your mortgage will end on September 2018, three years from today. (total interest cost: $19,900; interest between now and the end: $3,200) (current mortage with continuing early payment)
If you are planning on continuing to pay off your mortgage at the rate that you have been paying, this doesn't give you a whole lot of time to get your closing costs back in saved interest.
Let's assume that you are planning on paying off your mortgage at the same rate that you have been paying it. Closing costs and interest rates can vary, but for the sake of discussion, let's say that you can refinance today for 3.125%, and the closing costs will be $2000.
The normal amortization schedule for your new, 15 year mortgage would look like this: ($370 monthly payment, $13,500 total interest) (refinance)
At the rate you've been paying (about $1585 per month), if you continue this after you refinance, your mortgage will be done about 3 years from now. ($2,500 total interest) (refinance with early payment)
Looking at your two options, if you do not refinance, you will be paying about $3,200 in interest before your mortgage is paid off. If you refinance now, you'll be paying about $2,500 in interest, a difference of $700. It is very unlikely that you will be able to refinance for less than $700 in closing costs, so I would say that it is not worth it for you to refinance.