Conservatives and people on the political right more likely to have voted to leave the EU than remain