I will try to be concise: (Re-reading this now to edit it. I failed.)
Realized something while discussing psionics with Telecanter.
The Weird is a historical/aesthetic idea with a definite philosophical implication...
A Short History Of The Super-Human In Art and Popular Belief:
Stories and Images Meant to Educate and Entertain People In The Magic Belief System Era: (existing pre-Enlightenment, fading during the Enlightenment, pretty much dead by the 20th century)
Things and effects that are super-human (magic, elves, angels, etc.) are an organic part of the universe--not just in the sense of organically tied to nature but also tied to human ideas of morality, behavior, symbolism, fate etc. Like: A black cat crossing your path is bad luck, eating human flesh will turn you into a werewolf, a vampire can only come into your house if you invite it in, etc. Fantasy as a genre is built on this implied philosophy. In short: superhuman things care about your behavior--especially in its moral or at least symbolic aspect--and react to it.
Stories and Images Meant to Educate and Entertain People In The Technological Belief System Era: (ascending during the enlightenment, in full bloom by the mid-20th century)
Things and effects that are super-human (robots, aliens, cyborgs) are an organic part of the universe, but no more so than we are, and are not tied in any intrinsic way to human morality, behavior, fate, etc. Like: The spacefungus just wants to reproduce, Mars wants our women due to no fault of our own, The Alien is built that way to survive, etc. Most science fiction is built on this implied philosophy. In short: we're products of motiveless, amoral scientific processes and so are the rest of the wonders of the universe.
The Weird represents a transition historically (in the '30s) aesthetically, and philosophically between these two belief systems, in the words of the Man Himself:
The true weird tale has something more than secret murder, bloody bones, or a sheeted form clanking chains according to rule. [i.e. The Magical] A certain atmosphere of breathless and unexplainable dread of outer, unknown forces must be present; and there must be a hint, expressed with a seriousness and portentousness becoming its subject, of that most terrible conception of the human brain--a malign and particular suspension or defeat of those fixed laws of Nature which are our only safeguard against the assaults of chaos and the daemons of unplumbed space.
The creations of The Weird--in it's most typical form--are consistently described as organic yet unnatural. Cthulhu is weird, green slime with eyeballs (a thoroughly 20th century idea) is weird, the Tripheg is weird, things that mix the (magic) symbolic with the (science-fiction) seemingly biologically plausible (an eel with a man's face that is blind and bloated from living beneath the water for years) are weird.
Scientific belief and traditional sci-fi don't accept this category: nothing is "unnatural"--all is explicable, and magic only accepts the unnatural as an expression of something morally repugnant. In fantasy and magic belief systems, stuff Satan made is unnatural, the rest isn't.
The weird fits neither. Whatever's going on is rarely as explicable as Satan.
Therefore the fundamental underlying philosophy of the Weird is a sort of difficult-to-categorize thing. Lovecraft is the premier example. Is Cthulhu:
A) actually bad or
B) just a big alien that doesn't care about us any more than we care about our stomach parasites?
It is purposefully left unclear*: if it is A, then Cthulhu is (thematically if not aesthetically) just a Demon, which is a fantasy idea from the era of magical thinking and we can relax into the moonlight and ravens of that romantic world, and trust that if we are good and righteous or at least obedient to the signs we will get what we deserve. If it's B, then Cthulhu is just a Big Green Monster, and we can run or shoot it with lasers.
But it isn't clear: in a million ways that I will not categorize here, Lovecraft and other avatars of the weird purposefully cloud the issue. When we summon him, are we Wrong to do it, or just Incorrect to think it's a good idea? We cannot rely on our goodness or our lasers to save us, it is an unsettling genre.
The other Weird Tales authors don't necessarily hit the weird nail on the weird head quite as hard as Lovecraft, but they all seem to partake of a certain moral/aesthetic confusion not seen in earlier heroic and horrific tales: does the hero succeed because s/he is good (or at least philosophically "correct"), because s/he is clever, or neither? Does the victim die because s/he is sinful, foolish, or both? Or neither? Is the thing encountered hostile, or just an alien predator?
In a sense, the weird is a spike of uncertainty between two valleys of clarity. When thinking Magically, we know we are in a world of symbols, and that to be human is to somehow be woven into the symbols: the Sky may want something from us, but we want something from it, so that's fair and, perhaps eventually, explicable, when thinking Scientifically, we know we are stupid, but one day we may learn. The Weird is another thing altogether: can we learn about it or not? Will it turn us evil just to learn about it? Or is it simply a phenomenon we can eventually master?
The Call of Cthulhu game is very elegant in mechanically modelling this. You start with no knowledge of the premise of the game--"Mythos: 0%"--but the more knowledge you gain (the more fit you are to play the game) the more crazy you go (the less fit you are to play the game). The instability and the uncertainty grow and grow. You become more skilled and crazier, so those skills seem to be less and less relevant--or just more dangerous since there's always the chance you'll turn them on your friends. Surgery 90% seemed like such a good thing in the beginning. Just like science.
Magic has a dramatic logic (like a fairy tale), science has a procedural one (learn or die), the Weird might not have any. It is significant that Lovecraft stories do not always end with death or always with escape. They are not definitely horror or definitely adventure.
The Weird is a delicate balance, in a story or a game, to maintain. I groan when the laser swords come out at the end of Masks of Nyarlathotep, though I also was obscurely disappointed to hear about the space ships the Old Ones seem to (maybe) have had.
So: the weird is built on an uncertainty about whether our morality, symbols and emotions do have any meaning in this science-run universe or not. Whether any badness is actually bad or just "different". Magic knows the snake is evil, science knows it is just other, The Weird is not sure.
Boring academics will reduce this all down to anxieties about race: are they evil or just different? But I think the confusion goes farther than that, and to far more interesting places.
Now that we've spent quite some time (in art years) in a science fiction era, some authors--mostly ones who are doing that thing certain kinds of leftists do where they ask science if it can truly be its own morality--are swinging back toward The Weird. Because they want to make us unsure and unsettled about whether or not our morals and symbols can be left behind. And also because artists ALWAYS use The Magical because they are all about manipulating symbols--The Alien may be written from a science perspective, but it's big and black and slippery for psychological reasons.
The Surreal, in the classic High Art sense (which happened around the same time as the Weird Tale) is just The Weird with its connections to the Magic past and Scientific future (those two popular ideas fit for mass consumption) cut off. It is all the present. Floating Mars bars and cars with elephant heads. When the Surreal seems to rely too much on the clarity of science or magic, it ceases to feel Surreal.
So the Surreal is purer, but the Weird is more useful for games. You might not be able to understand The Weird, but at least you might be able to transition to something--the symbolic logic of Magic or the procedural logic of Science--by going through it. The second season of Twin Peaks seemed to struggle on this point: Were aliens involved? Was magic involved? Should we try to figure out what's going on before Cooper does? Or was it just Surreal?
Mandy proposes a similar division she'd call Catholic Weird vs. Protestant Weird. Protestant Weird being David Lynch and Edward Scissorhands ("clean") and Catholic Weird being Thomas Pynchon and Cormac McCarthy and Warhammer ("messy").
Some considerations for games:
-For me, psionics in D&D feel wrong unless they involve The Weird. Since the Weird is a transition between the Magical (D&D) and the Sci Fi, I feel like the Weird and the Sci Fi can exist in a world (Gigacrawler, Rifts), and the Weird and the Magical can coexist (Mind Flayers) and all three can be together (Shadowrun, and Warhammer 40k and Warhammer Fantasy sharing the same Chaos army lists--a good and rare example of the artists successfully merging the creative and commercial requirements of the situation), but skipping straight from Magic to Sci Fi (look, now I have swords for hands! It was easy!) without a certain level of Weirdness and Confusion seems like it just ruins the aesthetic effect.
-Psionic powers as an idea are Weird when they move into the zone where they overlap with psychological ideas. Like: the Star Trek guys who can just move things with their minds are not Weird. The ones who create things out of thin air because they have nightmares about them are Weird. This kind of thing uses a pseudoscientific explanation to bridge the gap between an indifferent natural world and a Magic one where the environment cares about what we think or believe.
-The whole "Magic-is-unnatural" trope seems to reek of The Weird. When Mandy and I playtested a Weird Fantasy Roleplaying adventure and found out that Detect Magic and Detect Evil were pretty hard to tell apart in the system, it seemed like a rather keen symbol for the whole moral-confusion issue here.
-"Chaos" is a key word. Is chaos merely unpredictability (the biggest barrier to clarity in science) or is it actual moral badness? The confusion in early D&D (and in Warhammer, and in the clones) about whether or to what degree the alignment "Chaotic" just meant "Evil" is a prime symptom of the doubt at the center of The Weird. As is everything in Michael Moorcock and the other works the Chaos idea came from. I do not mean to resolve the issue, just point to it as an example of how this aesthetically Weird idea of "these parts don't fit together" and the philosophical lack of clarity almost always come packaged together.
-Since the '80s, the Japanese seem to have an easier (and sometimes oddly frictionless) time negotiating back and forth across Lake Weird from Sci Fi to Fantasyland and back again than artists in the west. It may (may) have something to do with the fact that an aesthetic of abstraction and unnaturalness are more deeply expressed in traditional Japanese art and design than in any other culture. The unnatural is totally ok. Or, if not ok, than something you might have to get used to: Akira: you're weird, deal.
*Like we are constantly told the mythos creatures don't care about us and are alien, yet everyone involved with them practices dark arts and becomes totally weak, corrupt and, basically, Pure Evil.