If I turn on all the feedback options then I get messages in the status window like this...

Attack roll 12 + 6 = 18: hit

What do the two numbers that get added mean? and what determines if the resulting number is a hit or not?

The first number is your roll, the second number is the modifier to the roll based on magic, specialization, etc. you hit if you do better than the armor class of the opponent. See the D&D rules section of the Walkthroughs, Guides on this site.