The chess comparison is actually very valid. It takes special hard- and software to beat the best human chess players. Chess, although a difficult game to master at top level, is in essence limited when it comes to a computer game where the grid is not 8 by 8 but oh i dunno what a huge size map is... and where the pieces are not easily defined as "can move over all the white squares, in a straight line, in any direction" for a bishop, but "can move 4 squares, except when on a road, or using Cloudwalk, or Tireless March, can hit any piece as long as it does not have lightning resistancee, can summon an entire army in your face or cast a volcano under a city, can cast 36 other spells" And hten you have four of those units, each with a small army behind them, and seven stacks that are simple troops without magical support... Perhaps if you had a cray supercomputer to play upon you might be able to emulate vile human behaviour

One of the mythical adages of magic is that "there is always a cost".  The greater the power, the greater the cost.

It's a good rule of thumb to consider when using it in a game.

I read a China Mieville story where a modern-day "magician" managed to create a teleport spell modelled after the original Star Trek transporter concept:  Your atoms are ripped to pieces, shot across space and reassembled into a new copy of "you".

After some time, the magician started to become unstable and almost insane.  He had teleported himself hundreds of times and when an exorcist was called in they discovered that he was possessed by over a hundred ghosts of himself--each the spirit of a previous incarnation that had died when ripped apart.

They were all there to torment him into ceasing murdering himself.

Gotta think outside the box.  Instead of, "you lose a few hit points" or "you become exhausted" after casting you might consider more permanent effects with each use.

There's a reason Gandalf didn't summon his own balrogs.  ; )

That is quite insulting.

Or are you in any position to objectivly rate the effort of AI programmers?

Whilst I agree with the general sentiment of your post, I can't resist commenting on the fact that neither Sauron nor Morgoth did summon any Balrogs... rather they arrived of their free will.

I'm now 100% sure I can discount everything you say from now on. Your posts are as good as the banner ads to me. 

Idiot. 

This was poorly put, as others have pointed out, but I think it really does get to the heart of the problem.  Let's reword it to be less harsh:

The AI in 4x games is terrible because developers do not have enough time, appropriate resources, and a cohesive plan to properly program computer opponents.

From what I understand of game development (and I admittedly am looking from the outside in), AI is something that is developed to deal with the systems in a game after those systems are in place, almost as an afterthought, and never with enough time or money.  Even the much lauded AI of GalCiv 2 was the result of Brad tweaking stuff in his spare time over the course of many patches, after the game was released.  A better approach would be to develop systems and AI simultaneously, so that every step of the way you are building something that the AI can evaluate properly and use to effect.  Of course this still needs to be built into a unified whole that becomes a computer opponents, but the result should be well built decision trees that mesh seemlessly with the various systems that make up the game.

not quite sure what this has to do with fantasy. you can have sci-fi terms like, i don't know.. asteroid strike, planet killer, cloning, that essentially does the same thing.

Comparing the games to chess is not entirely a good correlation.

While in a sense it's true that in a game you got more options (build 1 of 10 buildings on 1 of 10 colonize-able locations and move to 1 of 10 locations instead of which of the 8 peons to move, and 1 or 2 moves for the first turn), you can also mop them a lot better- out of the 10 buildings, only 3 actually worth building. from the 10 locations, 2 are different and better than the rest in all respects, and out of the 10 movement options you do not want to stay put, so it's actually 9 options instead of 10. The chess still has 16 options. 3*2*9=54, which is 3 times as many play options, but in order to decide which one to pick you will usually need to look at 3 turns ahead, not 10(as "the further i look the better" goes in chess).

This refinement of looking only in the near future with certain goals, and on the further future with a bit different goals means that overall many of your choices are predetermined as "best course of action", which means your option narrow down very fast.

Regarding the AI- I'd like to know how much time/developers are really put onto the issue. Brad said that the reason GalCiv has great AI (personally I didn't connect with GalCiv) is because he worked on it for years. On many of the games, you build the game, fix it up a bit and ship it. You got literally hundred of hours(if not more) built into things like the Graphic Engine for it to work, with extra time for people to use it. You got more time spent on building the game, combining it with the graphic engine etc. But considering that the game can be "played" as anything beyond meager testing only in its late stages, the AI development starts only near the end.

This also means that it doesn't get the love it deserves before shipping, and doesn't tend to be a priority.

As a "bonus feature", how much is the expected return of the AI? According to brad, investing to raise the AI from being challenging to 90% of the player-base to 95% of the player-base is not worth it.

Getting into hardware issues is not really an issue, I do believe you can free that much with good coding after seeing a video of a couple of guys talking thro a skype-like system at the 80's (which means they did on something like 56K bit connection at most), or the assembly 4K intro competitions. Both examples shows how wasteful our usage of our resources is, and that we can do better, much much better if we ever so wish. (with the obvious problem of investing time in it)

it's not insulting or derogatory. it's just the sad reality of the industry as it exists today.

true observations:

1) many 4X game developers admit that they don't prioritize good AI because people will buy the games anyways. game developers (as in the organizations who develop games) cut corners and are lazy. it's standard operating procedure for businesses.

2) developers don't have the budget to pay a lot of smart people to work on AI. look at the game credits to see an idea of how many people contributed in some way to a game's AI. 4X game development is almost always a low-budget operation, with AI being a tiny fraction of a tiny budget. they haven't even moved AI to the cloud yet, because that's an additional expense.

3) development organizations don't have enough skill to ship a decent AI within their time-frame/budget. part of that lack of skill is programmer talent. part of it is bad project management. part of it is game designers making the strategic decision-making too hard relative to the abilities of the AI.

the exact ratios of each limitation vary from developer to developer, but some combination exists pretty much everywhere, and it's why AI is terrible.

say what you want about AI being 'hard' (whatever that means), but don't pretend to use existing games as 'evidence' of what the limitations actually are.

I can go along with points 1-3, but not the last line.    In fact, I would also add Tic-Tac-Toe as evidence; that's as easy an AI as you're going to get.   At the end of the day, there is really only one limitation to artificial intelligence:   human intelligence.    One need only turn on the TV to see that clearly we have a dearth of resources on that front.

I think the only real impetus to change is to bring back a form of genuine character death.

It would focus designs sharply on consequences before they created "fantastic abilities" and it would turn the end game dilemma on its head again.

If a low level and a high level mage could each be killed by a single bullet when careless it would foster cooperative play, genuine alliances and betrayals and really put people in the story.

I don't think gimmicks and layered penalties are the whole solution to being overpowered--magic or otherwise.

AI is just one aspect of making a game permanent fun. 

Exactly. When magic is taken to the extreme, like a lot of fantasy settings do, unmagical armies quickly become useless. Soon after that human civilization becomes pointless except as amusement for the wizards and that would tend to get destroyed when they fight.

The only problem with magic being taken to the extreme is that it's done badly.

Lets assume Earth had a very strong magic force around it. Life, when evolved, would utilize it. That means that a pack of Storm Wolves would run lightening-fast to catch the cow by surprise. The cow, if it manages to in time, will try to harden itself to be like a rock in order to survive.

When working the fields, the farmers would use their magic in attempts to repel bugs and ravens from eating the harvest, and so on.

A strong wizard being born in a city will be nothing but the local bully. The same way that certain men are physically stronger than others, if the villagers decide they don't like the wizard, they will kick his butt. They are spellcasters just as well, they know a few spells for fighting (what, do you think bar fights won't happen?) and the same way the strongest man will fall against numbers, so will the said wizard.

The problem comes with the restriction that magic is "inherent" and that only some people can use it. In a world where there is strong magic, if only a small part of the population can utilize it, they will quickly flourish, dominate their surrounding to extinction, and breed their magic using ability, so (assuming they don't drive themselves to extinction) they will, in turn, turn the world into the one I described above.

I honestly think the problem isn't that magic is over powered, it's the concept of mana that is.  Sorry going to be rushed an unformatted here. I appologize.

 I'm currently working on my own 4 X game. I've been currently looking at this puzzle as I design the combat system.

I want to see what it's like to approach it with mana being a scare but powerful resource.  Each node will give 1 - 3 mana points.  Those mana points can be used to cast a limited number of magic spells from units with the magic ability. So lets say you have 4 fire mana.  Fireball takes 3 mana but it can do a ton of damage. If you expend the 3 mana you are limited to 1 mana for the remainder of the turn.  Each turn you regenerate only .5 mana from each node up to your mana capacity (basically 1 bucket per node plus a few bonuses.)

If instead you want to buff your units, which should be more efficient then instant spells, you can imbue them which will halt regeneration of one of your mana buckets. 

Brad, I am going to disagree with you.  One of my favorite fantasy series is the Dominions series (currently up to number 4).  You can only win by destroying your opponents, it has a large empire building aspect, detailed magic (and very powerful magic), and a great back story.  It's AI while weak (compared to human players who have been playing the series in MP for the past decade) will give new players a run for their money and they will likely lost the first few games (unlike most of the titles you have listed).

The problem isn't magic.  If the AI used magic effectively in one of these games you might have a different opinion that that is where the problem is.

My favorite 4x-ish (it doesn't really have exploration and extermination isn't your goal) game lately is Crusader Kings 2.  The interesting thing is that there is no victory condition, only loss (if your entire family line dies out).  You do what you feel like and set your own goals.  It is all about the journey, not the destination.  It is the stories that come out of the game that make the game interesting for me, not casting the uber-spell or taking out the last city.

Another game to look at for inspiration is King of Dragon Pass.  While I haven't played this game in years, I still remember events from it.

In any case, you know where to find me if you want to discuss it more.