I have been to to Twickenham Stadium several times for rugby games.
If you are interested in attending one, I recommend the Rugby Sevens. They have theme days where people dress up. There is a lot of drinking so not really good for the kids! But even though there is a lot of drinking, it is very amicable!
If you are going here on a game day, you can get off the underground at Twickenham and follow the crowds. You can also get off the stop before or after as it might be very busy.