It is important to distinguish between cause and effect as well as the supply (saving) versus demand (borrowing) side of money to understand the relationship between interest rates, bond yields, and inflation.
What is mean by "interest rates" is usually based on the officially published rates determined by the central bank and is referenced to the overnight lending rate for meeting reserve requirements. In practice, what the means is, (for example) in the United States the Federal Reserve will have periodic meetings to determine whether to leave this rate alone or to raise or lower the rate. The new rate is generally determined by their assessment of current and forecast national and global economic conditions and factors in the votes of the various Regional Federal Reserve Presidents. If the Fed anticipates economic weakness they will tend to lower and keep rates lower, while when the economy seems to be overheated the tendency will be to raise rates.
Bond yields are also based on the expectation of future economic conditions, but as determined by market participants. At times the market will actually "lead" the Fed in bidding bond prices up or down, while at other times it will react after the Fed does. However, ignoring the varying time lag the two generally will track each other because they are really the same thing. The only difference is the participants which are collectively determining what the rates/yields are.
The inverse relationship between interest rates and inflation is the main reason for fluctuating rates in the first place. The Fed will tend to raise rates to try to slow inflation, and lower rates when it feels inflation is too low and economic growth should be stimulated. Likewise, when the economy is doing poorly there is both little inflationary pressure (driving interest rates down both in terms of what savers can accept to keep ahead of inflation and at) and depressed levels of borrowing (reduced demand for money, driving down rates to try to balance supply and demand), and the opposite is true when the economy is booming.
Bond yields are thus positively correlated to inflation because during periods of high inflation savers won't want to invest in bonds that don't provide them with an acceptable inflation adjusted yield. But high interest rates tend to have the effect or reining in inflation because it gets more costly for borrowers and thus puts a damper on new economic activity.
So to summarize,
- high interest rates are positively correlated to high bond yields because the two are basically the same thing (one determined by central banks, the other by the market)
- interest rates are negatively correlated to inflation because higher rates reduce demand for money (borrowing) and lower rates increase demand for money, which in turn (ceteris parabus) causes a lagging decrease or increase in inflation
- inflation and yields are positively correlated because as inflation rises or falls, yields tend to rise or fall to keep supply of savings balanced to match demand.
