Gavin Williamson said Wednesday in Brussels that the two, part of a notorious British cell dubbed the Beatles, "have turned their back on British ideas, British values."
He said: "Do I want them back in the United Kingdom? No, I don't."
U.S.-backed Syrian Democratic Forces are holding thousands of IS detainees, including hundreds of foreign fighters.
U.S. Defense Secretary Jim Mattis said Tuesday that "the important thing is that the countries of origin keep responsibility for them."
Williamson said that he and Mattis spoke early Wednesday but declined to say what possible solutions they had discussed.
Copyright 2018 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.