Yes you should worry and take care not to violate the law or provide any appearance of impropriety.
Every bank in the USA is required under the Bank Secrecy Act to report cash transactions over $10,000 the same day to the IRS -- and here's the fun secret part -- without notification to the depositor.
But splitting the deposits up into smaller amounts is also a crime, called "structuring".
On occasion there is a news story where a retail business that naturally must deposit cash from customers will be (falsely?) accused of structuring, e.g.:
Feds seize grocery store's entire bank account -- Institute for Justice defends grocer
Under the legal doctrine of civil asset forfeiture, your money can be accused of a crime, seized, and tried separately from its owner. The actual cases indicate the money as defendant, i.e. "US v $124,700"
In this somewhat bizarre system of "justice", the owner need not be charged with a crime, and is not in immediate peril of going to prison (about the only upside in this, but might be temporary because the authorities haven't charged the owner yet). When only the money is charged with a crime, there is no requirement for the government to supply a public defender for the owners who can not afford a lawyer.... can not afford a lawyer, because the government took all their money....