Andrew also spoke about his deep “upset” after his planned interview with the Late Late Show was dramatically cancelled