Be honest with him and most of all with yourself. If you want to break up with him, it's better to get it over with now than prolong a one-sided relationship. You'll both suffer the longer you let the relationship go on if you don't want to be with him. Yeah, he'll be upset, but both of you will be more upset if it escalates later. Good luck.