What are my capital gains? Can I say I "spent" $1 million even though
I borrowed half of the purchase price, and therefore claim I had no
capital gains, so I owe no taxes?
Don't try this. That would make the capital gains when you sell the rest equal to the purchase price. Also the loan doesn't factor into this calculation.
So what do you do? If the numbers are what you give in the example then take the time and effort to gather the evidence. You may have no choice but to get help to determine the value of the land when it was purchased, how the value changed over time and how much is the gain.
I am assume in the the value of the land is uniform. If it was one parcel when you bought the land, and then it was subdivided just prior to selling the land, you might not have enough time to know how it impacted the value of the rest of the property, until the local government reappraises the land.
If the land was either separate parcels when you bought it or you subdivided the land several years ago, use the information from the tax documents to track the value over time. You could use the percentages when the split was made to assign the initial percentages when you bought the land.
The problem can be what happens if the value of the land isn't uniform: if part of the land is more desirable then the other, you may need help to determine the value of the land. Again if the land was separate parcels, then that should be easy, if the split was new you might have to wait until the remaining land is reappraised.
Note: some of this should have been done before agreeing to sell. Otherwise how would you know if you got a good price.
Regarding the presence of the loan, the lender would have been very interested in this transaction. They are a part owner of the property, and would want to know how it impacted their investment. They might insist on a portion of the proceeded, and/or a refinancing of the existing loan. This may be discussed in the loan papers you signed.
Also the local government would have been involved in splitting process. There can be complex zoning processes involved which would require you getting legal help. Because the size of the lot can change what development can be done on that property, either the part you sold, or the part you kept could suffer a loss in value.