The Royal Fleet is the naval force directly commanded by the King of the Andals and the First Men. It consists of a large number of ships based at King's Landing and Dragonstone. It falls under the command of the Master of Ships, one of the members of the king's Small Council.
During the War of the Five Kings, the fleet becomes divided between the forces of Joffrey Baratheon and Stannis. When their two forces clashed at the Battle of the Blackwater, losses to the old "Royal Fleet" were bound to be high, as its ships were fighting each other, and in the resulting battle the old "Royal Fleet" was devastated. Stannis lost almost all of his ships, and was forced to withdraw on a few auxiliary sellsail ships that he had hired. The smaller number of ships that Joffrey controlled faired only slightly better, as Tyrion gave no more than token resistance before springing his wildfire trap.
As Stannis Baratheon was his brother Robert's Master of Ships, when the War of the Five Kings breaks out upon Robert's death, a disproportionately large portion of the Royal Fleet (primarily based at Dragonstone) joins Stannis' faction when he declares himself king. This then becomes "his" Royal Fleet. However, Stannis has a very small land army and only has the allegiance of a few Houses from the Crownlands holding islands in the Narrow Sea. Thus Stannis' dominance at sea is one of his few advantages at the outset of the war. Stannis then augments the Royal Fleet by recruiting an additional thirty warships belonging to sellsail and pirate Salladhor Saan.
Matthos Seaworth and his father debate the nomenclature of what to call "the Royal Fleet" where there are several kings in the realm. Later, when Stannis's forces approach King's Landing, King Joffrey Baratheon expects the twenty warships of the Royal Fleet squadron based at King's Landing to sail out to meet them. However, Tyrion Lannister, Acting Hand of the King, has ordered the Royal Fleet to fall back so as not to be destroyed by Stannis's numerically superior forces. Instead he commits a single ship packed with wildfire to the attack and has Bronn detonate the vessel when it has passed through the leading elements of Stannis's fleet. The resulting explosion destroys a large portion of Stannis's fleet and kills thousands of men. It is the opening move of the Battle of the Blackwater. Tyrion's move ensures that nineteen warships of the Royal Fleet survive the battle.
In the books
According to semi-canon material approved by George R.R. Martin, the Royal Fleet consists of 210 warships, at least 80 of which are full warships dipping 100 oars or more. The Royal Fleet's flagship is the King Robert's Hammer, which dips 400 oars and is the largest warship in Westeros. The Royal Fleet is divided into two squadrons, a smaller one of 50 based at King's Landing and a larger force of around 160 ships based at Dragonstone. The position of Dragonstone and the size of the fleet there makes an attack on King's Landing by sea almost impossible to mount for any foreign power.
In the A Song of Ice and Fire novels, when the War of the Five Kings breaks out, Stannis Baratheon seizes control of the ships at Dragonstone and uses them in his attack on King's Landing. Most of the Royal Fleet's crews and captains stay loyal to Stannis because he had been commanding them as Master of Ships during his brother Robert's entire reign, even leading them into a decisive naval victory at the Battle of Fair Isle during the Greyjoy Rebellion. Unlike in the TV series, the Royal Fleet does participate in the Battle of the Blackwater in the books. The fleet engages Stannis's force and draws them into the mouth of the Blackwater Rush. Tyrion raises a massive chain to trap the fleet in the river and detonates barrels of wildfire hidden on the floor of the river and inside several of the royal ships. This move destroys most of Stannis's fleet, but also more than half of Joffrey's fleet. Stannis was forced to retreat on the sellsail ships of Salladhor Saan, which survived the wildfire because Ser Imry Florent contemptuously insisted that they only follow behind the main fleet as a rearguard.
By the conclusion of the battle, the old "Royal Fleet" was functionally destroyed, with only about a dozen ships loyal to Joffrey still afloat in King's Landing. Another dozen or so survived but were severely damaged and forced into dock for repairs - such as the royal flagship King Robert's Hammer. Worse, the King's Landing dockyards were also destroyed in the blaze, forcing the damaged ships to limp up the coast to Duskendale for repairs, making the situation even more desperate. The rest of the Royal Fleet had been obliterated by the wildfire - including the Queen Cersei. All of Stannis' ships are destroyed, forcing him to rely on the thirty Lyseni sellsail ships of Salladhor Saan to retreat his few remaining men (under 2,000) to Dragonstone. Four or so ships were so far upriver that they managed to escape the wildfire, but were then trapped with the city guarding the river's mouth, so their crews decided to sink them rather than let them fall into Lannister hands (two remained active making piratical attacks on river shipping, but they were still trapped on the river with their capture inevitable).
The Royal Fleet is left crippled after the Battle of the Blackwater, reduced to barely a tenth its former size (and even that is being generous, as half of the surviving ships are crippled and laid up in drydock). Given that the ironborn have seceded from the realm and taken the Iron Fleet with them, and the Royal Fleet was functionally destroyed at the Battle of the Blackwater, this leaves the Redwyne Fleet as the one remaining significant naval force under the Iron Throne's control. This results in the Lannisters becoming even more dependent on the Tyrells, much to Cersei's consternation, as the Iron Throne must now rely on the Tyrells and their Redwyne vassals as their main naval force.