Ex-Japan PM won't be at ceremony: report