As a mechanism, the fog can:
- Add latency to player perception - like the boss could leave turbulence in the fog that is visible after a while, and of course if you have to move to a position to see whats there, that adds latency as well. The player is always a few steps behind boss, encouraging prediction / tracking skills (like if player is familiar with boss behavior, and its deterministic, these latent signals are enough for player to know whats going on).
- Add hidden information, creating uncertainty - this really changes everything. Instead of single focused strategy, it makes sense to use multiple redundant strategies to cover the full possibility space of what could be happening (like place traps all over the place, have fallback plans, cover every direction, etc). Statistical/probabilistic thinking encouraged (risk/reward tends to arise too).
- Discourage or encourage mobility. Either have players set up a defense perimeter and let boss come to them (encourage fast reaction to unpredictable attacks), or do the opposite and have players attack the boss from surprise directions protected by the fog (of course fog needs to benefit boss in some way for boss to take this disadvantage, unless fog is environmental).
- Reveal what is hidden. If players/boss/weapon has some invisibility, fog could reveal that - like perhaps boss has some invisible aspects (intent orbs...?) that reveal where boss will attack, but you can only see those in fog.
- Have synergies with weapons/abilities. Maybe you can consume the fog to do something. Maybe the fog turns into poison if hit by fire weapons. Maybe it diffuses light, giving laser weapons AoE damage.
- Spread. It would be interesting if fog was emitted from specific locations (like monster), so player has to limit the propagation. Alternatively, emitting fog could tire monster, so player can encourage propagation (keep the place ventilated).
- Age (if fog is a fluid or particles). Forms a gradient that allows many things. Player can track monster by following that gradient. Weapons could be guided along that gradient. Aging fog could become damaging to player (or have any number of effects at specific points along the gradient).
- Have varying density over space/time (coherent noise or specific pattern). This could drive the movement of players/boss, if there is an optimal density that either prefers. Seeds the gameplay with some diversity in environment since these randomly moving areas could drive the battle into regions of the map normally avoided (and overall just keep things moving on a somewhat unpredictable path). Boss navigation could be more interesting if boss is constrained to the most-fogged areas (that keep shifting) - like perhaps players are worried that a specific location shifts into the fogged area, giving boss huge advantage (and seek to get something done before that happens). Perhaps circumstances cause boss to get temporarily stuck in an island of fog.
- Be 'denser than air'. If game has height, fog level can slowly increase until players have to decide what hilltop they get stuck on (if being in fog with boss is unsafe). This would slowly transition the mobility from players (low fog) to monster (high fog). This sort of avoid-the-lava mechanic can be more interesting if getting to higher and higher locations requires overcoming obstacles to clear the path to high ground. If boss monster flies, increasing fog allows boss monster to fly higher (or stand taller, I guess). Fog could also do the opposite and descend from above, pushing players into scary tunnels.
- Separate (players or monsters). Assuming single-player, multiple monsters could be separated by players to prevent them from cooperating (assuming they need LOS now and then to stay together) - of course, this would spread them over the map and be less predictable, so its up to players to decide what is wisest.
The primary effect of basic fog is probably the vast increase in hidden information (uncertainty), assuming players normally rely on looking at the map to see what is going on. Importantly, one can offer alternative information pathways for players. Memorization/deduction (cognitive load) is naturally available, but you still need to design for it (ensure monster behavior has clear patterns, ensure environment is actually fun to memorize). Alternative ingame senses reduce cognitive load and allow more control over the experience (hearing/smell/vibrations/magical auras, or maybe you can connect into a network that lets you sense some things if the network reaches there, or just use abilities to reveal information). Delegate memorization/tracking to the game (like mark map locations ahead of the time) to further reduce cognitive load. Even prediction of monsters path could be delegated to the game (render the possibility space some time into the future from where boss was last seen), if you want player to focus on strategy instead of prediction.
In the end, you could have as much information in the fog, as you have without, just a different perspective - that should ensure player skill determines outcome, not randomness.
A fog that actually hides a lot of information, is good for either a stressful test of skill (overreliance on mental model of gamestate), or a moment of relaxation (since if youre not doing much, you dont need all that information - the fog could enforce these moments without really having a big role in gameplay). Maybe you can let players decide which outcome they want through their own action?
