Stannis totally abandoned the castle, taking every last man with him to aid his campaign in the North.
Meanwhile, House Lannister in King's Landing made no attempt to capture the now totally abandoned castle, for potentially the next three full years during the reign of King Tommen Baratheon. The Lannisters were aware that the castle was utterly undefended - though Dragonstone is a vitally important island fortress commanding the naval approach to the capital city through Blackwater Bay.
After Tommen's death and Cersei's seizure of power, Cersei asked Jaime Lannister where Daenerys Targaryen was most likely to land first when she began her invasion of Westeros. Jaime correctly assessed that she would probably try to take Dragonstone first, as it has good deep water ports for her fleet, Daenerys was born there, and Stannis left the castle abandoned so she wouldn't face any resistance. Neither Jaime nor Cersei suggest rushing Lannister soldiers to the abandoned castle to defend it from Daenerys's landing.
When Euron Greyjoy came to King's Landing with his fleet to propose a marriage-alliance with now-Ruling Queen Cersei Lannister, he made no attempt to capture the empty fortress, despite passing it twice as he entered then left Blackwater Bay.
Ultimately, Daenerys lands her fleet at Dragonstone without opposition. She and her followers enter Dragonstone castle, and she assumes control, to begin planning her invasion of the rest of Westeros.
In the books
Daenerys Targaryen hasn't returned to Westeros yet as of the most recent book in the A Song of Ice and Fire series. The events regarding Dragonstone are as followed:
In the third novel, Stannis takes most of his army by ship to the Wall for the Battle of Castle Black, and leaves skeletal garrisons in Dragonstone and in Storm's End, under the command of Ser Rolland Storm and Ser Gilbert Farring respectively.
Since Stannis hopes to eventually retake all of the Seven Kingdoms, it wouldn't make sense to voluntarily abandon the defense of one of the strongest castles in Westeros. Dragonstone was constructed with lost Valyrian architectural techniques and is defensively quite formidable, so even a token defensive force can hold it against large armies.
While Stannis stays in the North, he barely speaks about Dragonstone. After hearing Samwell Tarly's story about destroying the Other with a dragonglass dagger, he mentions there is a lot of dragonglass in Dragonstone, and sends orders to Ser Rolland to mine the precious material for making weapons against the Others. It is unknown whether Ser Rolland received the order and fulfilled it.
Shortly after Cersei is appointed as the Queen Regent, she decides to conquer both Stannis's castles, in order to secure Tommen's hold upon the Iron Throne. She sends to Dragonstone 2,000 Lannister soldiers and the Redwyne fleet under the command of Lord Paxter Redwyne, and sends Mace Tyrell to Storm's End.
Lord Paxter, a veteran military commander, decides against launching an open assault, since it may result in unnecessary casualties; instead, he chooses a slower but safer tactic - laying a siege to the castle, in order to starve the besieged into submission (what may take at least six months). He also orders to dig a tunnel underneath the castle walls. In the meantime, Mace Tyrell besieges Storm's End.
In contrast to Riverrun, Cersei is in no hurry to take either of Stannis's castles, and does not urge the besiegers to act faster.
Loras Tyrell takes command
Shortly after Euron Greyjoy is crowned, he launches a military campaign against the Reach, cunningly taking advantage of the fact that the Redwyne fleet is away. The first stage in his plan is performed successfully: hundreds of the ironborn longships, including the Iron Fleet, capture the Shield Islands. Euron plans to sail east, but in view of his subordinates' objections, he changes his plans: he orders Victarion to take the Iron fleet to the Slaver's Bay and bring Daenerys, whom Euron wants to wed. In the meantime, Euron continues with his campaign against the Reach. He does not give orders to build even one additional ship (let alone 1,000), because he still has hundreds of ships at his disposal, sufficiently for raiding the Reach.
The Small Council receives reports that 1,000 ships raided the Shield Islands. Obviously the reports are exaggerated (500 is closer to the actual number of the raiding ships). Margaery and Loras urge Cersei to give Lord Paxter orders to send the Redwyne fleet back; that would have been the logical reaction, to eliminate the ironborn threat while it is still in its initial stage. Cersei, who secretly gloat over the Tyrells' distress, fails to estimate the danger that the ironborn's invasion pose to not only to the Reach, but to the entire realm (what Sam Tarly figures very quickly).
Lord Aurane Waters objects to the Tyrells' demand, pointing out that the presence of the Redwyne fleet is essential to maintain the sieges of Dragonstone and Storm's End. Cersei agrees with him, stating that Dragonstone and Storm's End are much more important than "those rocks", and that the Tyrells must take care of the problem themselves. Loras explains that his brothers Willas and Garlan can mass 20,000 soldiers within a month, but that will not do any good without watercrafts that can fight the ironborn's longships. Cersei insists that the Redwyne fleet will be released only after Dragonstone falls.
Loras, fearing for the safety of the residents of the Reach, asks Cersei to give him the command over the forces that besiege Dragonstone, declaring that he will take the castle within two weeks, even if he has to take it apart with his bare hands. Cersei is surprised and delighted at Loras's offer, thinking (incorrectly) it is a win-win situation: if Loras succeeds, Stannis will suffer a grievous blow, and the Redwyne fleet can sail off to meet the ironborn; if Loras fails, he will be blamed and disgraced. As a "bonus", whether Loras succeeds or not, Cersei hopes he will get himself killed due to his rash nature, what will cause a lot of sorrow to his sister and grandmother (whom Cersei deeply loathes). No one could blame Cersei, because Loras has volunteered willingly in public for that mission. As happens with each of Cersei's schemes, this one goes wrong too. Cersei fails to take into consideration two factors: first, that Loras may not be the only one to get harmed; second, she underestimates the ironborn threat.
Loras sails to Dragonstone and takes the command. First he offers the castellan of Dragonstone to settle the siege between them in single combat, but Ser Rolland refuses. Loras orders to assualt the castle. As the ram breaks the castle gates, Loras is the first to invade. He fights valiantly, even after being shot by two arrows and suffering a mace blow that broke some of his ribs. The defenders fall back to an inner keep once the curtain wall is taken. Loras leads the attack there as well, but then he is doused with boiling oil.
Dragonstone has been taken, but it proves to be a Phyrric victory: Loras's rashness has turned what could have been a bloodless victory into a slaughter. About one thousand men have been killed, most of them Lannister soldiers, the best and the bravest knights and young lords.
Morever, Euron has successfully completed the second stage in his campaign: the capture of Shield Islands has opened the Mander to the ironborn's raiding ships, and they have captured the Arbor and its neighboring islands, turning them into bases for further raiding. They even pose a serious threat to Oldtown. A galley captain describes the situation to Sam, emphasizing that there is little they can do without the Redwyne fleet. He comments bitterly "the best we can do is guard the Sound and wait for the bitch queen in King's Landing to let Lord Paxter off his leash". Sam muses fearfully "if King's Landing loses Oldtown and the Arbor, the whole realm will fall to pieces" (a very accurate estimation, which Cersei fails to perceive).
Cersei, however, is not concerned about the above "minorities". She is content that the last direct threat to King's Landing has been removed, and that Stannis's already failing cause has suffered a severe damage. She is also very pleased to hear that Loras is near death. She finally - perhaps too late - orders Lord Paxter to sail the Redwyne fleet back to the Reach and drive the ironborn away. Then she goes to tell Margaery and her cousins the bad news, drawing a sadistic pleasure from watching them mourning. Although Cersei sounds sympathetic, Margaery sees through her pretence. That puts a strain on the Lannister-Tyrell alliance, though it is still intact.
Mace Tyrell reports the Small Council that Loras's men have searched the castle and the whole island for treasures, mainly for dragon eggs, but found nothing; apparently they are unaware of the importance of dragonglass. Kevan thinks it is a waste of time, for Stannis had years to find any treasure that the Targaryens left behind.
It is not mentioned in the novels what has become of Ser Rolland and his men, whether they were captured alive or killed.
It is unknown whether Stannis has been informed about the fall of Dragonstone. He cannot do anything about that anyway, since he currently has much more urgent matter to deal with - the imminent Battle of Winterfell, which has not begun yet in the novels.
Although it is not mentioned explicitly, it can be assumed Dragonstone has been garrisoned by Lannister troops.
Books vs TV series
- The Season 7 premiere states that Dragonstone has been abandoned. It does not specify whether Stannis Baratheon abandoned it when he left with his army for the Wall, at the end of Season 4, or whether Stannis sent for the remaining soldiers when his campaign was failing in Season 5. This is a drastic condensation by the TV writers that doesn't really make sense: it means that for the past at most three years of story time, the Lannisters simply didn't bother to re-occupy an empty castle, one of the strongest in Westeros, which directly controls sea access to their capital city. There is zero precedent in the entire history of Westeros in which one side in a war didn't try to re-occupy a castle that the opposing side abandoned, much less a strategically important one.
- In the books, Stannis took most of his army by ship to the Wall for the Battle of Castle Black, but he left a small skeleton force garrison behind in Dragonstone (and also in Storm's End) - because Stannis hopes to eventually retake all of the Seven Kingdoms, and it wouldn't make sense to voluntarily abandon the defense of one of the strongest castles in Westeros. Dragonstone was constructed with lost Valyrian architectural techniques and is defensively quite formidable, so even a token defensive force can hold it against large armies.
- Given how strong the fortress is even this small garrison was enough to deter a direct attack by the Lannisters: as a result much of the remaining Lannister/Tyrell fleet, under the command of Lord Paxter Redwyne, was pinned down encircling and besieging Dragonstone, trying to wait them out through starvation over time; the siege might have lasted at least six months, but it was safer option than an open assault, and there was no reason to take the castle quickly. This prevented the Tyrells from dealing with Euron Greyjoy's military campaign against the Reach. Loras and Margaery urged Cersei to send the Redwyne fleet back to the Reach, but she (secretly gloating over the Tyrells' distress) refused, claiming that Dragonstone was more important than the Shield Islands. Loras Tyrell came with the idea to force a quick end to the siege by storming the castle, stating that he could conquer the castle within two weeks; Cersei agreed, in hopes he would die in the fighting. Loras takes the castle, but with heavy and unnecessary losses due to his rash nature, and after fighting through many defenders despite taking numerous wounds, he was doused with boiling oil and left barely clinging to life. Cersei then needlessly gloated to Margaery about Loras, breaking down the already fragile Lannister-Tyrell alliance. The TV series omitted all of this and just had Loras burn to death in the Destruction of the Great Sept of Baelor (in either version, he apparently burns to death - though there is some slim chance he might in fact survive in the books). As a result, as of the most current novel Dragonstone is weakly held by a remaining Lannister force after the siege.
- Even assuming that Stannis for some reason wanted to take every last available soldier he had with him to the Wall, no explanation is given for why the Lannisters wouldn't simply retake the abandoned castle. While it is plausible that Cersei would be incompetent enough to not order it taken, Kevan Lannister would have almost certainly done so when he became acting Hand of the King. Even within the episode, Jaime himself observes that Stannis abandoned the castle, the Lannisters are aware of this, it is relatively nearby, and he says that it's the most likely target for Daenerys Targaryen's initial landing...yet he doesn't make any attempt to suggest that they should rush even a small force of Lannister soldiers to the castle so they can defend it against her attack. Even if they hope to defeat Daenerys, they would eventually have to retake Dragonstone castle, losing thousands of men in the process, when they could have just sent a hundred or so guards there ahead of time. Nor does Jaime's dialogue give any explanation as to why the Lannisters didn't re-occupy the empty castle in all the time since Stannis left.
- Euron Greyjoy's fleet had to directly pass abandoned Dragonstone, both coming and going to King's Landing. This is also quite obvious given that Jaime is standing on a large map at the time, visibly showing that any ship approaching King's Landing has to pass Dragonstone. Yet the TV show presents that Euron wouldn't think to capture the island, perhaps as a gift for Cersei.
- Even if the storylines are out of synch, and Daenerys lands on Dragonstone before Euron arrives...then Daenerys's fleets would be based at Dragonstone, and attempt to stop Euron's fleet from reaching King's Landing. As presented by the episode, Daenerys simply hadn't reached Dragonstone yet, and Euron ignored capturing one of the strongest castles in Westeros even though it was empty.
- Ultimately, when the TV writers cut out Loras's assault they didn't come up with another way to wrap up the dangling storyline of what happened to Dragonstone - not even simple dialogue describing off-screen events, for example a report that the Lannisters had captured it but then Daenerys's fleet quickly seized it.