This is actually extremely fascinating to me (and I almost wrote my film paper on it).

They have a lot of information on this in the Appendices of the Extended Editions. They basically did it through three methods: forced perspective, replacement actors, and green screen.
For forced perspective, they would place one actor closer to the camera than the other, but the angle of the camera made it look like they were next to each other. This is how they did shots like Gandalf and Bilbo inside Bag End. In normal, everyday life it looks like >>
this.
They also had small people to stand in for the hobbits and Gimli, and larger people to stand in for the men. So when Gandalf greets Bilbo, and his hand looks so much larger, it's because it's a larger actor seen from the back. That was also common in the boat shots.
Green screen is just normal CGI. They'd film one actor against a green screen, then film the other actor, then superimpose them together to look like one is smaller.
They also built Bag End (and possibly other places, I don't remember) twice - one normalish, for when the hobbits were in it, and one very small for when the "big people" were in it.