Ok, i am freaked out. What should i do?

Look, if you go with him you may get into a situation that you cant control. Who cares if he doesnt like you anymore, its better than you doing something you will regret and him telling everyone!