What do I do now? I'm really confused?

I would give up if I were you as he indicated he really didn't want to get together. It's better to meet a person who has the same mutual feelings as yourself.