The R&B crooner also won the monster single of the year prize for her smash-hit song, which stayed on top of the singles chart for ten weeks.
The star, who performed at the show, was nominated for honours in five categories.
She was presented with the video of the year award by rap legend Dr Dre. Looking resplendent in a pink dress, Rihanna said the win was the "most important" of her career.
Pop star Justin Timberlake also won two awards at the show: the quadruple threat of the year prize and male artist of the year.
British performer Amy Winehouse, who was nominated for three awards at the ceremony, failed to win prizes for best new artist and best female artist of the year.
Gym Class Heroes won the trophy for best new artist, while Black Eyed Peas singer Fergie nabbed the female artist of the year award instead of Winehouse, who cancelled her scheduled performance at the show.
Beyonce won the best collaboration award for her sultry video Beautiful Liar, with Shakira. Fall Out Boy won the award for best group.
