Culture minister Ed Vaizey has labelled a potential EU exit "irresponsible" for the UK arts sector " despite culture secretary John Whittingdale