If you intend to avoid using credit cards, that leaves cash and the requirement to manage it carefully.
Of course, whatever method you choose to use to manage your money, willpower is a must.
As a youth, I followed a self-created program that I later learned was called the envelope method. Mine was a virtual envelope system based on my checking account (same as cash) with the envelopes being represented by line entries on a sheet of paper.
Later years and the development of computers (yes, I'm that old) allowed me to write a program in BASIC, followed by a spreadsheet and then the revelation of a commercially produced piece of software.
The link above describes physical envelopes and a sound practice. If you have the willpower to maintain the restrictions on spending, you can remove/reduce your debt and begin saving. In my youth, saving was out of the picture, debt was non-existent and I spent right up to my envelope allowance!
A summary could be as follows:
Create envelopes, real or virtual, and assign categories for each. Electricity, cable/internet, water rates, fuel for vehicles, groceries, anything and everything you can think of.
Assign each envelope a time period and an expense value. For example, cable/internet could be monthly and about $100, while your fuel costs could be aggregated as a weekly cost of $200. I almost forgot rent or mortgage, usually monthly, say, $1200.
If you have a weekly expense and a weekly income, your week-referenced envelopes are good to go. For the monthly ones, multiply the values by 12, then divide by 52 and round up. Doing so will provide a tiny automatic surplus for later. I tended to expand the rounding to the nearest five or ten dollars.
If you have a monthly income, the monthly envelopes are set, while the weekly ones should be multiplied by 52 and divided by 12, rounding as noted.
Do you spend straight cash from your wallet? Be sure to add in an envelope with that reference and an approximate figure.
Sum up the figures of your envelopes and compare them to your income(s). If you come out with a surplus, good for you. That's a good start to wiping out your debt, which should have its own envelopes too, of course. Adjust the debt payments higher and bring the surplus down, if you can.
If you come out in the red, it's time to consider what you can pare away. How many trips to FourBucks Coffee can you do without? Do you take in the cinema a few too many times? Anything you can chop off brings you closer to a positive balance.
Over time, your surplus could be directed to savings.
Ideally, you'll develop a practice of checking the balances, checking the expenses, adjusting as needed and it will become second nature.
In our household, nearly every expense has an entry in the budget program (all computer based now, we've entered the nineties!) and every purchase is registered in the program. Because of this level of precise management, nearly every purchase is by credit card, but when the bill becomes due, the money is already in the budget and the card is paid in full, no interest accrued. Another benefit of this is that the credit card cash-back program pays a couple percent.
It took decades (no exaggeration) to accomplish this, but time passes regardless.
The above link recommends a program known as Mvelopes, which was a different program when it was first developed, again decades ago. Unfortunately, they went to a subscription style of cost, adding yet another expense to the big picture.
Reading the comments, it's interesting to note others' recommendations, which I've not researched, but mostly the comments are supportive of this type of system as being valuable for getting out of debt and getting a "handle" on your money.