A controversial ban on literary gifts to UK prison inmates is to stay according to a government minister.