There are very frequent buses between London and Oxford. They run approximately every 10 minutes.
http://www.oxfordtube.com
http://www.oxfordbus.co.uk/espress1.shtml

Additional travel information:
http://www.pmb.ox.ac.uk/visitors/gettingtopembroke.html

Gloucester Green is the Oxford central coach station, where the buses from the airport or from London leave you. It is a short walk (no more than 10 minutes) to Pembroke College. You can see both locations in Google Maps (link).

Pembroke College is NOT located at Pembroke Street but at Pembroke Square, which is not a square at all but a little cul-de-sac tucked against a church off a relatively busy street called St Aldates.

The best advice is to print the map and once in Oxford ask locals for guidance. If they don't know Pembrole College, they will certainly know Christ Church College. This college lies on the opposite side of St Aldates respect to Pembroke College. If your luggage is very heavy you may take a cab, but it really is only some 500m or so from the coach station.

Confusingly, one of the coach lines from London (but not from the airport!) is called the "Oxford Tube". This must be why you thought that Gloucester is a tube station.