Star Wars, the original movie and original trilogy (1, 2, 3) as a whole, was sort of a singular story. A comparison arises in my mind because something I've talked a lot about is too: Deus Ex.
Beyond the first level and a bit of the second I still haven't played any of Human Revolution because I don't have money, certainly not enough money to buy a videogame. In spite of this I have been able to talk about continuity problems between Human Revolution and Deus Ex, to which it is allegedly a prequel. (They're blatant to the point you can spot at least one thing that makes it impossible for Deus Ex to take place 25 years later on the same planet by looking at almost any still picture released of the game, and they were acknowledged and owned by the HR design team well before the game hit shelves.)
When I have talked about continuity I've been accused of various things. Venerating the designers of Deus Ex as gods was one, but another one that has come up with surprising frequency is that I would not be satisfied unless the prequel was the exact same game just moved 25 years earlier and with different names for the characters. That I find baffling. The story of Deus Ex is one that changes the world in a way that prevents the story of Deus Ex from taking place again. Telling the same story a quarter century earlier would fuck up continuity to no end.
And that's sort of like Star Wars. To be consistent with the original requires a new story to be different from it. To maintain continuity with Deus Ex any other game set in the same universe needs to tell a completely different story. That's not all it has to do; that's merely necessary but not sufficient. But it is necessary. Star Wars is the same way.
You get that right from when Obi-Wan talks about the Jedi in the first movie.
They're living in what he calls the dark times, they're living in a fucked up galaxy. And it's pretty damn fucked up. To the point that in the first movie the rebels who are in theory supposed to be fighting to make things better are actually forced to fight against things getting worse instead.* But it was not always so.
Obi-Wan reveals that within living memory things were good. Not just good, but very good. The Jedi guarded peace and justice within the galaxy for over a thousand generations in The Old Republic. First off, so far as I know we haven't had a single generation in which peace was guarded in the history of this planet. Second, a thousand generations is a long damn time.
A thousand generations is longer than all of recorded history here in reality. Multiple times over. Now I think that that introduces some world-building (galaxy-building?) problems** but that's the story being told.
And in many ways I think that's what allows for the extreme idealism present in Star Wars (the original movie and the original trilogy) in spite of it being set in an extremely fucked-up galaxy where an evil oppressive government ruled the entirety of civilization and thought nothing of blowing up planets.
The extreme fucked-upness was an aberration. Steady stable sustained good times were the rule. In the original movie you learn that The Empire rose to power during the lifetime of Luke's father, in Return of the Jedi the Emperor is killed off along with his single greatest symbol of power in front of the assembled Imperial fleet. During Luke's lifetime (and still during the lifetime of Luke's father.)
The Emperor, the cause of these aberrant dark times, was in power for all of one generation. One generation of extreme bad times balanced against a thousand generations of extreme good times.
Going by the original trilogy there is absolutely no reason to believe that things wouldn't go back to the steady stable sustained good times.
That's the hope that's present in Star Wars. Fix this one thing and everything will be good for a basically unimaginably long time. That's pretty extreme idealism.
But that means you don't get to tell the same story again for at least (over) a thousand generations. Otherwise the hope of the original is dashed. It turns out to be a false hope, and the idealism shatters on the rocks.
Instead you have to tell different stories. Specifically it would go like this: Stories of guarding peace and justice, rather than fighting to establish them, under a benevolent and extremely stable government. Or stories that have nothing to do with that sort of thing (not all stories can be about Jedi) but are set in the same universe during the same period of sustained good times. The fall of that government, the good times along with it, and the rise of the Empire. The original trilogy. Re-instituting the Republic and rebuilding the Jedi. Stories set during a new period of sustained good times.
There's nothing wrong with these stories, and I think they could be quite interesting. But they're different stories. Sustaining good times is very different from fighting to get back to them. Building a good government is different from overthrowing a bad one.
You don't get to have more stories of blowing up superweapons or needing to overthrow governments unless it takes place in what should be the relatively short period of mopping up after the Emperor is killed and before the Republic is fully reinstated because reversion to the mean, in the universe of the original trilogy, means that once the cause of the aberrant bad times is gone things will, or at least should, go back to stable good times.
So that's the thing. You can't tell the same story again because the idealism and hope of the original rests on the idea that the story will not need to take place again. Once the immediate problem (the Emperor and the Empire) is dealt with everything will be good for a very, very, very (over a thousand generations of very) long time. If you try to tell the same story again what you do is make it so that the default is no longer over a thousand generations of peace and justice being guarded but instead much closer to the fucked up state of the galaxy shown in the trilogy itself.
Tell the same story again twice and you're really pushing things from idealism to cynicism. Do it three time and even more so.
But by telling the story again and again you're actually not telling the same story anymore because the repetition itself changes the story.
Evil had to triumph for Star Wars to take place. To rehash the same story evil needs to triumph again. To rehash the same story repeatedly evil has to triumph always. Even if every story ends with good beating back the the repetition of evil triumphant to set up those stories shifts the setting from a thousand generations of good times being the default scenario to one where good times can never last for long and evil will always be triumphant given time.
Star Wars began with evil having triumphed and all that was good and right in the galaxy being in a state of existential peril. If you're not willing to branch out and try new and different stories you need to keep going back to that. By keeping on going back to that you snuff out the hope that was present in Star Wars because the hope was that this situation would pass. And once passed stay in the past. It wouldn't happen again and again. Because it wasn't the usual state of things. It was a dark exception to the rule that the usual state of things is, hands down, good.
And I think that's part of the problem. People weren't willing to let it go and try new kinds stories set in the same universe. If you've read the "Tales from..." series of books you know that everyone in existence was either Hero or a Villain on a pretty much galactically significant scale all around.*** Smaller more personal stories didn't really get told much. And even on the occasions when they were told there was always another superweapon coming down the line. Or, if not a superweapon, an invasion. Or, if not an invasion, internal corruption that screwed things over in a way external threats never could. There is always something bad coming, no matter how good of an end an individual story might seem to have.
More than that, almost always something worse. Which I think comes from a desire to differentiate works when combined with the desire to not really try much of anything new. If all that is good and right in the galaxy is always in a state of existential peril, if evil is always turning out to be triumphant so that it will need to be overthrown by the underdog, if you're telling the same story again and again, how do you distinguish things? It basically seems to come down to, "How bad it is this time?" and the answer always ends up being, "Worse," because if the answer is, "Well, not as bad as last time," it's treated as retreading the same territory.
Mind you it doesn't have to be. The thing is, to distinguish the stories by more than degree of badness requires originality. Which some EU authors do have. As you'd expect they're usually the ones you find doing the smaller scale more personal stories instead of the "The Galaxy Is At Risk (again)" stories.
So instead you tend to get things like Newer Better Space Nazis (now with organic technology!) that via their death toll make the old space Nazis seem not nearly so scary by comparison.
You've now got stories set a century down the line. Are they stories about what it's like when you're 100 years into those good times we heard about in 1977 that the original trilogy was to return things to? No. Not a fucking chance.
They're stories that by their existence show that the hope and idealism of the original was nothing more than a pipe dream because, turns out, rather than returning to the previous "over a thousand generations of good times" standard, a new standard of, "The galaxy is really fucked up and never stops being that way for long," has replaced the old standard that the characters of the original trilogy hoped in their idealism that they might return to.
And in this way the Expanded Universe has forced itself to go more and more to the cynicism side of the scale. Nothing good ever lasts, nothing bright ever stands the test of time, the only constant is that evil will always be with you and war will never end. "This, too, shall pass," doesn't apply to the general fucked-upness of the galaxy. Or, if it does, applies to the bad as slowly as it did the good (maybe in over a thousand generations things will start to default to good again.)
And once you've driven from the extreme idealism of, "Fix this one thing and everything will get better and stay that way," to, "It never stays that way; get used to war without end," it's probably unsurprising that things went from campy fun to the equally unrealistic but completely opposite end of the spectrum: grimdark.
The currently longest gap between, "Even worse than last time," in the EU after the original trilogy takes place is merely an artifact of the fact that the books that don't skip into the future have yet to catch up to the comic books that skipped 100 years into the future. The next no-timeskip book won't be out until July and will only venture one year closer to the comics. Thus there's currently a 78 year gap during which the only things that are known are that: at the beginning of the gap things really sucked and were tending toward darkness, at the beginning of the gap it was foreseen that another great war was coming down the line within the generation, for the entirety of the gap and then some on both sides of it evil with a slow plan to take over the galaxy was prosperous and successful in its efforts (slow plan coming to completion is what the timeskip comics are about), hatred seethed among the worlds of the theoretical good guys so much that, given an excuse, they'd tear civilization apart just to settle scores almost a hundred years old (the beginning of them being given this excuse is what marks the end of the gap) and stuff like that. (I'm getting the info off a wiki, so any more intricate details can't really be included.)
Hardly the visceral hope present in the original trilogy that once the problem at hand was dealt with everything would return to an extended period of peace and justice.
And I think that's really it. If you're not willing to try new and different (and the galaxy at peace where justice was the default rather than the exception that the original trilogy implied was to follow would be new and different for Star Wars) basically the only thing you can do to distinguish yourself from what's come before is to push a bit further in the same direction. Star Wars was originally the story of the dark times, so as people push in that same direction it becomes the story of the darker and darker times. The threats get bigger, the blows get more significant, the death tolls rise, the heroes become less pure in their goodness, and everything slowly marches from, "Idealistic campy fun," to, "Grimdark all the time."
No one can ever meaningfully improve things because if they do then that's a push back. Back toward the less dark setting one had to pass through to get to the point things are in the story. And that's something people telling the same story but pushed further (over and over and over again) won't do.
Even if every book ended with a happy ending (which... they don't) there would be no way for the overall universe to not get darker and darker as each new venture had things worse and worse. Hope has to be killed off when you can see that, even if things seem to end well, what comes next will be even worse than whatever was just overcome.
The original trilogy's idealism can't really survive that.
And I still don't think I've hit a grand unified theory as I probably haven't covered half of what was in this thread, which is what made me want to put things together to start with. I wanted to put things together because when taken separately things that actually go together can seem contradictory.
"Star Wars is extremely idealistic" and "Star Wars requires triumphant evil" seem sort of contradictory until you put them together to get, "In the original trilogy triumphant evil was presented as a single exception to the rule that peace and justice can be maintained for multiple times longer than actual recorded human history and thus if evil's only triumph was dealt with steady stable sustained good times would return, which is an extremely idealistic stance to take." Which is true, and I think tells you a lot about how the original trilogy was able to do what it did.
Anyway, the first two words in the title are, "Working toward," so the fact I didn't get there isn't too much of a problem.
* The movie is about them not getting blown up and making it so the Death Star, which wasn't there last week, isn't there next week. Those two things sort of go together since the Death Star was the thing that put them at risk of being blown up. The movie doesn't end with them making strides toward restoring the republic but instead with them surviving.
** Specifically either technology has been completely stagnant for a very long time, or I kind of think a group that's been spanning a galaxy for a thousand generations would have better tech. There's a reason that when I did a thousand generation thing I set it in a period before the rise of civilization. That's a place where I can realistically believe that there would be a thousand generations of relative stability. After civilization gets kicked off I don't see things remaining stable for a thousand years, much less a thousand generations.
*** It was this observation that led to the creation of the, so far as I know never finished, parody "The Jedi Who Sucked" which can now only be found on the internet archive.