No, you cannot simply withdraw (non-excess) HSA contributions, you will owe a 20% fee (in addition to the due taxes); It doesn't matter if it is the same year.
Source: IRS (Notice 2004-50 Q&A 35)
Q-35. May an individual who has not made excess HSA contributions treat a distribution from an HSA other than for qualified medical expenses as the withdrawal of excess HSA contributions?
A-35. No. This withdrawal is deemed a withdrawal for non-qualified medical expenses and includable in the individual’s gross income under section 223(f)(2). (The additional tax under section 223(f)(4) also applies, unless otherwise excepted).
The only exceptions for the 20% fee are when you turn 65, become disabled, or die.