Returning to Japan in the same year you refund your pension – will you be taxed on the 20% you’ve gotten your tax representative to get back for you?

If, when I leave Japan, I do my pension refund and get the 20% withheld tax back, but return later the same year, would I then be taxed on my pension as though it were the current year’s income? From what I’ve read, when you get your tax representative to fill out the necessary paperwork they’re actually filing a tax return for you saying that you’re not in Japan.

I haven’t been able to find a clear answer to this anywhere because it seems like an unusual situation but I just wanted to plan ahead for it. I’m inclined to believe that they would count it as taxable income in the current year but I’m not entirely clear on where the cutoff is.

Ex: If you file in 2023 but get the money in 2024, does it count for 2024 taxes or 2023 taxes? I’m inclined to assume the former but it’s not completely clear to me. (Depending how this works I might alter future plans by a few months to avoid tax headaches).