Can I as a landlord cancel contract with the real estate agent?

I am a landlord. When I bought the apartment, there was a tenant already there who I knew I had to continue leasing to.

At that time I had the option to manage the tenant myself or get the former landlord’s real estate agent to continue to manage. I decided to let the agent continue so made a three-way contract with me as lessor and the agent’s company and tenant as subleasee and leasee. At that time the agent verbally told me I could not cancel my contract with them and that It would expire when the tenant would move out. However, this is not written in the contract. There is only a clause titled ‘Natural Termination’ which says if the property is destroyed in a natural disaster, or if the building is to be demolished to make way way for public infrastructure then the contract would expire naturally.

I now want to manage the tenant myself instead of letting the agent do it. The reason for this is that I don’t think the agent is acting in my best interests. There was a minor defect in the apartment (boiler).The boiler was still working but not stopping the hot water automatically. The agent arranged for a quotation which was 1,200,000 Yen. I told her I would get a quote myself and got one for half of that. She tried her best to force me to agree to her quote even threatening me that I would have to pay money to the tenant.
Eventually I did the repair using the company of my own choice. Now she is saying I have to reduce the rent for the period we spent on waiting for quotes even though the boiler was operational except it had to be stopped manually instead of it stopping automatically.

Can I cancel my contract with the agency and manage the tenant myself or is the contract indeed not ‘cancelable’? Just thought of asking here first before I go to a lawyer. Some of you may agree with her but please focus on my core question.