American Institutes for Research
A private, non-profit behavioural and social science research organisation, founded 1946. Not a government body: it worked here under CIA contract.
Remote viewing: research and applications evaluation
1995-09-29 · 206 source pages
A catalogued report evaluating remote-viewing research and its potential applications.
Use the source copy of the 1995 evaluation. This 206-page research report is not currently reproduced in the archive reader.
Source scans are unavailable here. Publisher annotations remain available below.
Publisher annotations
Commentary by The Declassified Collection, separate from the original source.
Source PDF page 1 · 1 notes
Source PDF page 2 · 1 notes
The three names
Mumford, Rose and Goslin are AIR staff — two senior behavioural scientists and the organisation's president. The outside reviewers are not named on the cover.
Source PDF page 3 · 1 notes
Chapter Three's five entries
The chapter does not summarise the two reviews. It contains them: folios 3-2 to 3-82 are the reviewers' own text, printed whole.
Source PDF page 5 · 2 notes
Remote viewing
The report's working definition, given in the first paragraph: the ability to describe locations one has not visited.
Blue-ribbon panel
A panel selected for professional standing. Here the scientific review was deliberately balanced between Utts and Hyman, with statistical advice from Lincoln Moses; its authority came from that composition, not agreement about the phenomenon.
Source PDF page 6 · 2 notes
Lincoln Moses
Emeritus professor of statistics at Stanford and the first Administrator of the Energy Information Administration. He advised on statistics but took no part in the panel's conference calls.
Beacon
The person who travels to the target site while the viewer describes it. Much of the later work reviewed here dispenses with a beacon entirely.
Source PDF page 7 · 2 notes
Hit
A judge's ranking of a viewer's description as the best match to the real target rather than to the decoys.
End users
The report's term for the intelligence officers who requested a viewing and received the result. Chapter Four's case is built out of their answers.
Source PDF page 8 · 2 notes
Actionable intelligence
Information specific and reliable enough to be acted on. AIR concluded that the operational material it examined had produced no actionable intelligence.
Inter-judge reliability
Whether independent judges scoring the same material agree with each other. It had never been tested here: one judge scored almost everything.
Source PDF page 9 · 1 notes
Not warranted
The Executive Summary's verdict on operational use. Chapter Five states the same finding as 'not justified'. The wording differs; the conclusion does not.
Source PDF page 12 · 1 notes
Enhancing Human Performance
The 1988 National Research Council report on techniques for improving performance, which examined remote viewing and found little support for it. It is the shadow over this whole evaluation.
Source PDF page 13 · 2 notes
Anomalous cognition
The researchers' preferred term for remote viewing, chosen to name an effect without asserting a mechanism for it. The report uses both terms.
Judge
The person who ranks a viewer's description against the candidate targets. Not the viewer and not the experimenter — in this programme, usually the Principal Investigator.
Source PDF page 14 · 2 notes
Star Gate
The Defense Intelligence Agency's name, adopted in 1991, for the last phase of the programme. Earlier names for the same line of work included Grill Flame, Center Lane and Sun Streak.
Foreign Assessment
The third leg of the programme: analysing what other countries were doing with the paranormal. The CIA excluded it from this evaluation.
Source PDF page 15 · 2 notes
Office of Research and Development
The CIA office that contracted AIR in June 1995. The report names it only on this page.
Declassified ... to facilitate a new, external review
The report says the Agency declassified its past parapsychological research to facilitate this external evaluation. The review itself followed a request from Congress.
Source PDF page 16 · 2 notes
Utility and scientific validity ... separately
The CIA's instruction, and the report's architecture: Chapter Three answers whether the science holds, Chapter Four whether the thing is any use.
We were directed
The evaluation was confined to material DIA and CIA supplied. AIR did not survey the wider parapsychology literature, and says so here.
Source PDF page 19 · 1 notes
Cook & Campbell, 1979
Quasi-Experimentation, the standard text on drawing causal conclusions from imperfect designs. Its appearance here signals the frame AIR brought to the job.
Source PDF page 20 · 2 notes
Edwin May
Physicist and research investigator at SRI International and then SAIC; project director from 1986. He supplied the list of studies the panel reviewed.
Statistical Editor
Utts edited statistics for the Journal of the American Society for Psychical Research. The panel was built to be partisan in balance, not neutral in composition.
Source PDF page 21 · 2 notes
One document ... remained classified
Footnote 1. A single item stayed classified through the review; Dr Mumford read it alone and gave the panel an unclassified synopsis. The evidence base has this one hole in it.
Strongest evidence
May was asked to nominate the studies he considered the most compelling. The panel then reviewed his selection, which means the programme chose the ground it was judged on.
Source PDF page 22 · 1 notes
The four questions
The reviewers' terms of reference: is there a statistically significant effect, is it paranormal, what explains it, and is it of any use. Chapter Three answers them in that order.
Source PDF page 23 · 1 notes
Suspended in the Spring of 1995
Operations had already stopped before this evaluation began. The report was written about a programme that was not running.
Source PDF page 24 · 2 notes
Starting in 1994
Formal user ratings of accuracy and value begin only in the programme's last full year. That single year of data becomes Appendix D.
Messick, 1989
Samuel Messick's work on validity, which holds that no single piece of evidence establishes it. AIR's justification for evaluating from several sources at once.
Source PDF page 25 · 2 notes
Three interview groups
End users, the Program Manager, and the viewers. The protocols used on each, and the resulting interview reports, are Appendix C.
Only those persons ... at the time of its suspension
The interviews were limited to personnel associated with the programme when it was suspended. Earlier operations were not assessed through a separate historical interview programme.
Source PDF page 26 · 2 notes
Eleven interviews
Seven end-user representatives, three remote viewers and the incumbent Program Manager, over July and August 1995. That is the whole interview base for Chapter Four.
A retired intelligence officer took notes
And a CIA representative attended as necessary. The interviews were not conducted in private between AIR and the subject.
Source PDF page 27 · 1 notes
Accuracy and value
The two judgements users were asked to make about each viewing. They are the axes of every table in Appendix D.
Source PDF page 28 · 1 notes
Collected prior to the start of the current review
AIR treats the 1994 user feedback as independent evidence precisely because it predates the evaluation. That feedback is Appendix D.
Source PDF page 30 · 2 notes
Rejoinder
AIR's word for Hyman's review. He was commissioned to review the same evidence rather than to answer Utts, though his report does respond to hers at length.
Not only classified program research, but also a number of earlier studies
Both reviewers went beyond the material they were given. The evaluation's own boundary was narrower than its reviewers' reading.
Source PDF page 31 · 2 notes
Review 1
Utts's review opens here and runs unbroken to folio 3-42. It was written first, dated 1 September 1995, and Hyman read it before writing his.
Psychic functioning has been well established
The report's most quoted sentence, and the reason it is still argued over. It is Utts's conclusion, printed in full by AIR, not AIR's own.
Source PDF page 32 · 2 notes
Precognition
Knowledge of something before it is knowable — here, describing a target chosen only after the description was written. Utts argues it is the cleanest test, because there is nothing yet to leak.
Little benefit to continuing experiments designed to offer proof
Utts's recommendation is to stop proving and start explaining. Hyman reaches the opposite conclusion from the same evidence.
Source PDF page 33 · 2 notes
SRI International
Stanford Research Institute, Menlo Park; independent of the university since 1970. It ran the government's psychic research from 1972 to 1989.
SAIC
Science Applications International Corporation, which took the contract over in 1990 under Edwin May.
Source PDF page 34 · 2 notes
Anomalous cognition and anomalous perturbation
The laboratory's neutral replacements for extrasensory perception and psychokinesis: names for the effect that assume nothing about its cause.
Free-response
The viewer describes anything at all, rather than choosing among a few known options. Harder to score, but the format almost all of this work used.
Source PDF page 35 · 1 notes
Replication over the long run
Utts asks for repeatable average performance, not a success on demand: a good hitter can fail at the plate while maintaining a batting average. Hyman's objection later is that similar averages do not establish when the underlying phenomenon occurs.
Source PDF page 36 · 2 notes
Effect size
A measure of the magnitude of a result, as distinct from the evidence against chance. In Utts's example, 7 boys in 10 births and 70 in 100 have the same proportion but different p-values. This is why she judges replication by effect size rather than by whether each study crosses a significance threshold.
Five percent of the time
The rate at which chance alone produces a statistically significant result at the usual threshold. Both reviewers agree this is why replication matters.
Source PDF page 37 · 2 notes
Rank-order judging
After a session, a judge who does not know the answer ranks the real target and four decoys against the description. Random target selection supplies the chance baseline: the description itself need not have been generated randomly.
Five-choice effect size
With five equally likely choices, the expected rank is 3. A run of ranks below 3 is better than chance. Utts's effect-size calculation measures that departure; the derivation appears at Remote viewing: research and applications evaluation — source PDF page 61.
Source PDF page 38 · 2 notes
National Geographic photographs
Knowing the pool is different from knowing which target was selected. Utts argues that random selection still gives each candidate an equal chance, so familiarity alone does not invalidate ranking. Hyman later questions whether repeated use of the same pool supports generalisation beyond it.
Conservative
Rank-order judging gives no extra credit for a brilliant match. A near-perfect description and a barely-sufficient one both score a rank of one.
Source PDF page 39 · 2 notes
The methodological checklist
The checklist separates several ways an answer could leak: through people, photographs, judging, or decisions about which trials to retain. Utts applies it to Experiment 1 at Remote viewing: research and applications evaluation — source PDF page 52. These controls address different routes of contamination, not one interchangeable safeguard.
Duplicate sets of target photographs
One set for the session, another for judging, so that a fingerprint or crease on the used photograph cannot cue the judge.
Source PDF page 40 · 2 notes
Prima facie evidence
Utts borrows the legal term for operational viewings so striking that they would stand unless rebutted. She concedes they cannot be scored statistically.
Post hoc
After the fact. Utts flags that May's 80 percent accuracy figure for the microwave generator was arrived at after the viewing, not against a standard set in advance.
Source PDF page 41 · 2 notes
Puthoff and Targ
Harold Puthoff and Russell Targ, the physicists who founded the SRI programme in 1972. Their 1975 paper is the source for the early operational successes described here.
The West Virginia site
One of the three early cases Utts says kept the programme alive: a viewer's description was reportedly accurate enough to start a leak investigation.
Source PDF page 42 · 2 notes
Semipalatinsk
The Soviet nuclear test site in Kazakhstan. The viewer's drawing of a gantry crane there is the most-cited hit in the programme's history.
The same analyst
Utts's point: the reviewing official called the crane inexplicable and the session unsuccessful in the same document, because he expected a successful viewing to contain no errors.
Source PDF page 43 · 2 notes
Without replacement
Sites were not reused once drawn. Critics showed this leaks information: knowing yesterday's answer removes it from today's possibilities.
The criticisms ... have resulted in substantially improved methodology
Utts's concession, and the frame of her whole review: the early work was flawed, which is why she rests her case on the later work.
Source PDF page 44 · 2 notes
May et al, 1988
The internal analysis of all 154 SRI experiments from 1973 to 1988. It is the baseline Utts measures the SAIC results against.
About one percent
The share of volunteers who scored consistently in mass screening. Utts reads this as evidence the ability varies between people, like athletic or musical talent.
Source PDF page 45 · 2 notes
Electromagnetic shielding does not appear to inhibit performance
One of Utts's nine summary findings: shielding did not appear to inhibit performance. It bears on proposed electromagnetic explanations; it does not identify the mechanism.
Ganzfeld
A sensory-homogenisation protocol: diffuse light over the eyes and unvarying sound in the ears reduce changing sensory input. Utts uses the ganzfeld literature as outside corroboration; Hyman disputes whether it and remote viewing measure the same thing.
Source PDF page 46 · 2 notes
Scientific Oversight Committee
The SAIC programme's outside committee, which Utts notes included a Nobel laureate in physics and a retired U.S. Army Major General.
Ten experiments
The SAIC set listed by May in July 1995: six remote-viewing experiments and four using other procedures. It is the review's intensive sample, not the total number of experiments in the programme's two-decade history.
Source PDF page 47 · 2 notes
Table 1
May's table separates the six remote-viewing studies from four other experiments. MCE means mean chance expectation. The distinction matters: physiology and binary-choice tests do not use the photograph-ranking procedure explained earlier.
Entropy II
Experiment 10, the strongest single result in the table: 90 trials, effect size .550. Hyman's review concentrates its fire on this one.
Source PDF page 48 · 2 notes
Homogeneous sets of sessions
Utts breaks the totals down by viewer and target type, on the argument that an average across different people and procedures hides what is actually happening.
Sealed envelopes
In Experiment 6 viewers took targets home. Utts flags the departure herself rather than leaving it to be found.
Source PDF page 49 · 2 notes
Viewer numbers
Viewers are identified by number — 009, 131, 372, 389, 518 — at both SRI and SAIC. The same numbers mean the same people, which is what makes the comparison possible.
Static and dynamic targets
Photographs against video clips. In Experiment 1 the photographs produced an effect and the video clips produced none, under otherwise identical procedure.
Source PDF page 50 · 2 notes
Six expert viewers
Numbers 002, 009, 131, 372, 414 and 504, whose 196 SRI sessions produced an effect size of 0.385. Three of them also worked at SAIC.
The same subject identifying numbers
The identifiers were carried from SRI to SAIC, allowing Utts to compare three returning viewers. Two had SAIC effect sizes close to the earlier pooled SRI result; the 1988 analysis did not give individual scores.
Source PDF page 51 · 2 notes
A methodological problem would have had to differentially affect
Utts's strongest argument: any flaw big enough to explain the results would have to have spared the dynamic targets in the same experiment.
Dr. Nevin Lantz
Principal investigator for Experiment 1, working from Pennsylvania while the viewers worked from their homes in four other states.
Source PDF page 52 · 2 notes
Item by item
Utts applies her checklist from Remote viewing: research and applications evaluation — source PDF page 39 to Experiment 1, setting each requirement beside an account of how it was met. The distinction between a general safeguard and evidence that it was actually followed becomes central to Hyman's response.
According to Dr. May
Several of the assurances rest on the Principal Investigator's word rather than on a document. Hyman makes exactly this the basis of his objection.
Source PDF page 53 · 2 notes
Twenty effect sizes
Experiment 1's design fixed in advance: five viewers, forty trials each, four conditions. Utts's point is that nothing here was decided after the results were in.
No such reasons were given
On aborted or discarded trials, Utts records an absence rather than an assurance. The distinction matters, and she keeps it.
Source PDF page 54 · 1 notes
Change in visual entropy
A measure of how much the colours change across a target image. May's team reported a correlation between this property and viewing quality; Utts speculates about a sense responsive to change. A target property correlated with performance is a proposed regularity, not yet an account of how information reaches the viewer.
Source PDF page 55 · 2 notes
Remote staring
Experiment 7: whether being watched from another room changes a person's physiology. Utts notes it is a conceptual replication of Soviet-era claims.
Target pool bandwidth
The proposed range of distinct things a target pool can contain. The preceding page gives the hypothesis: too narrow, and the viewer may imagine the whole pool; too broad, and there is no useful filter on imagination. AIR later asks whether uncontrolled intelligence targets would fall within a favourable range.
Source PDF page 56 · 2 notes
Ganzfeld
A sensory-homogenisation protocol: diffuse red light through halved ping-pong balls, white noise in the ears, a sender in another room. It is the field's other main paradigm.
Bem and Honorton, 1994
The Psychological Bulletin paper that brought ganzfeld results into mainstream psychology. Both reviewers cite it; they read it very differently.
Source PDF page 57 · 2 notes
Table 3
Utts's external check: SRI and SAIC alongside four independent ganzfeld laboratories. The effect sizes cluster between .167 and .261.
Cohen's h
A standard effect-size measure for comparing two proportions. Utts uses it so ganzfeld hit rates can be set beside rank-order results on one scale.
Source PDF page 58 · 2 notes
Would no longer be questioned
Utts's claim in its sharpest form: the consistency across laboratories would settle the matter in any less unusual field.
Beyond the scope of this report
Utts separates the scientific question from practical government use. AIR takes up the latter through operational records and interviews in Part Two, and concludes that continued operational support is unjustified.
Source PDF page 59 · 2 notes
It would require an equivalent amount of fraud
Utts's argument from scale: a single flawed laboratory can be explained, but a consistent effect across many would need a conspiracy of matching size.
We know very little
Her concession, stated as plainly as her conclusion. No sender is needed, distance does not matter, and beyond that the mechanism is unknown.
Source PDF page 60 · 1 notes
May, Utts, Trask, Luke, Frivold and Humphrey, 1988
Utts is a co-author of this SRI review, a major source for the historical comparison in her assessment. The bibliography identifies that earlier role directly.
Source PDF page 61 · 1 notes
Variance of a rank
The variance is N squared minus 1, divided by 12. With five choices it is 2, so the effect-size calculation divides by the square root of 2.
Source PDF page 62 · 2 notes
Appendix 2
Utts describes the ten SAIC experiments under four headings: purpose, method, results and discussion. This lets a reader distinguish what an experiment intended to test from what its result could establish.
I asked Dr. May
Where the written reports were silent, Utts questioned the Principal Investigator and says so. Hyman's objection is that this makes the record depend on him.
Source PDF page 63 · 1 notes
Binary coding
Experiment 4 tried to turn viewing into a message channel by coding five yes/no target features. It is the only experiment here aimed at transmission rather than description.
Source PDF page 64 · 2 notes
Despite its name
Experiment 5 involved no lucid dreaming at all — it was a baseline test of three novices before the dream study. Utts flags the misleading title herself.
Eight trials
One novice reached an effect size of .265 on eight trials, which is not significant. Utts reports it as promising and not significant, in the same sentence.
Source PDF page 65 · 2 notes
Covert threads
Targets went home with viewers in sealed envelopes, threaded so tampering would show. None showed. It is still the loosest control in the SAIC set.
ERD
Event-related desynchronisation — the drop in a brain rhythm when attention shifts. Experiments 8 and 9 looked for it as a physical signature of viewing.
Source PDF page 66 · 2 notes
Entropy II
Experiment 10, the flagship: a rebuilt dynamic target pool matched to the photographs for bandwidth. It produced the strongest result in the programme.
Viewer 389
For Experiment 9, Utts reports a positive estimate for viewer 389 but not a statistically significant result. A positive effect size and a result distinguishable from chance are different claims; the individual results matter as well as the pooled total.
Source PDF page 67 · 2 notes
Table 4
Experiment 10 broken out by viewer. Two viewers score far above chance on both target types; one scores at chance on both. The total conceals both facts.
MEG
Magnetoencephalography, which measures the magnetic fields of brain activity. Experiment 3 was abandoned once the team realised the signal could not exceed the noise.
Source PDF page 68 · 2 notes
Sequential analysis
A decision is built from repeated binary guesses rather than from one guess. Here the computer stops when a choice wins clearly or a continuing tie is apparent. Utts's earlier checklist requires stopping rules to be fixed in advance: this is not permission to keep sampling until a favourable result appears.
Is the lost child still in this city
The proposed application turns a search into successive yes/no decisions. For the successful subject, 100 such decisions required 21,337 guesses. That cost separates an improved laboratory hit rate from a practical way to find someone.
Source PDF page 69 · 2 notes
Galvanic skin response
Skin conductance, which rises with arousal. It is the physiological measure behind the remote staring result.
Frightening consequences
Utts's phrase for the Soviet reading of these experiments — that the observer was manipulating the subject's body rather than the subject sensing observation.
Source PDF page 70 · 1 notes
Post hoc observation
The opposite-sex effect was noticed after the data were in, and Utts labels it as such. She applies the same label to May's operational accuracy figures at Remote viewing: research and applications evaluation — source PDF page 40.
Source PDF page 72 · 2 notes
Review 2
Hyman's review, dated 11 September 1995, ten days after Utts's. He had read hers, and quotes it; she had not read his, which is why the Reply exists.
Psi
The field's own umbrella term, covering both extrasensory perception and psychokinesis. Hyman uses it where the programme prefers its neutral coinages.
Source PDF page 73 · 2 notes
I will not duplicate what she has written
Hyman takes Utts's account of the experiments as read and attacks only her conclusions. The two reviews are therefore not symmetrical documents.
Subcontracted by the SAIC investigators
Some of the outside replication Utts cites as independent was paid for by the programme she is corroborating. Hyman raises this early and returns to it.
Source PDF page 74 · 2 notes
Three questions
Hyman's whole review is organised around answering no to each: the effects do not establish anomalous cognition, flaws are not eliminated, and the results are not consistent with other laboratories.
Disciplinary matrix
Thomas Kuhn's term for the shared assumptions and standards a scientific community works inside. Hyman's case turns on the programme having been outside one.
Source PDF page 75 · 2 notes
The secrecy ... cut them off
Hyman argues that secrecy deprived the work of ordinary peer review and independent attempts to reproduce it. His immediate object is the ten SAIC experiments selected for close examination, not a claim that only ten experiments had been conducted since 1973.
Shake-down and debugging
His analogy for how flaws in a new protocol get found — by other laboratories using it. Secrecy prevented it here.
Source PDF page 76 · 2 notes
Society for Psychical Research
Founded in London in 1882 and still active. Hyman uses its history to date the field's search for scientific standing at over a century.
J. B. Rhine
The Duke University psychologist who from the 1930s tried to make ESP an experimental, statistical science. The card-guessing tradition descends from him.
Source PDF page 77 · 1 notes
J. G. Pratt
Rhine's colleague, who came to argue that psi was real but would never be reliably reproducible. Hyman cites him as the field's own most damaging admission.
Source PDF page 78 · 2 notes
Lack of cumulativeness
Hyman's charge is that the evidence base is replaced rather than added to: yesterday's showcase experiment does not remain a dependable starting point for tomorrow's work. His Newton comparison asks for an enduring result, not an unchanged theory.
Ebbinghaus
Hermann Ebbinghaus, whose 1880s forgetting curve still replicates even though the theory around it has changed. Hyman's example of what a cumulative database looks like.
Source PDF page 79 · 1 notes
A shifting database
Each generation's rock-solid evidence is discarded by the next — usually for good reason, once a flaw is found. Hyman's caution about treating today's results as final.
Source PDF page 80 · 2 notes
Unpack just what these claims entail
Hyman restates Utts's conclusion as three separate claims and then takes them one at a time. AIR's Discussion at Remote viewing: research and applications evaluation — source PDF page 112 follows the same structure.
Relying solely on significant departures from a chance baseline
His definitional objection: every other science can point at its phenomenon; this one can only reject a null hypothesis.
Source PDF page 81 · 2 notes
Null hypothesis
Here, the statistical model of chance performance. A small p-value means the observed result, or one more extreme, would be unlikely under that model. It is not the probability that chance caused the result. Hyman's objection concerns the further step from rejecting that model to identifying a paranormal cause.
No relationship between distance and effect size
Hyman grants that the programme did some genuine process research. He argues it still cannot say when the phenomenon is present.
Source PDF page 82 · 2 notes
Too much power can lead to rejections
A subtle statistical point: with enough trials, a test will reject the null simply because the idealised model does not perfectly match reality.
I agree with Jessica Utts
Hyman accepts that the reported effects probably cannot be dismissed as chance, multiple testing or selective publication of positive results. He is willing to assume genuine effects while disputing that their cause has been identified. The concession is substantial; it is not acceptance of anomalous cognition.
Source PDF page 83 · 1 notes
I cannot provide suitable candidates
Hyman admits he cannot name the flaw he believes exists. He argues that naming it has historically taken years, which is his answer to the objection.
Source PDF page 84 · 2 notes
Soal
S. G. Soal, the British parapsychologist whose celebrated 1940s results were shown decades later to have been manipulated — by a method none of his critics had guessed.
Betty Markwick
The parapsychologist who found how Soal cheated, in 1978. Hyman's point is that the discovery came from inside the field, and by accident.
Source PDF page 85 · 2 notes
Autoganzfeld
Honorton's automated ganzfeld, built specifically to close the flaws Hyman had catalogued. Hyman then found a new pattern in it, and says so here.
Not the result of a fishing expedition
Hyman states that he went looking for this pattern deliberately, as a randomisation check. He applies the post hoc standard to himself.
Source PDF page 86 · 2 notes
A meaningless composite of arbitrary units
Hyman's attack on averaged effect sizes: one experimenter contributed almost half the ganzfeld trials, and different weightings give different averages.
Heterogeneity
The spread of results between investigators. One contributed only negative effect sizes. Averaging across them hides that the laboratories disagree.
Source PDF page 87 · 2 notes
The same laboratory, the same viewers, the same targets
Hyman inverts Utts's consistency argument: sameness across the experiments makes them one experiment repeated, not several independent ones.
Ed May ... the sole judge
The single-judge problem, stated most bluntly here. It is the one methodological point on which both reviewers and AIR end up agreeing.
Source PDF page 88 · 2 notes
Here, both Professor Utts and I agree
The explicit meeting point of the two reviews. AIR quotes it in Chapter Five and makes it the report's central methodological finding.
A far cry from compelling evidence
Hyman's formulation of the gap: an unexplained statistical departure is an anomaly, and an anomaly is not yet a phenomenon.
Source PDF page 89 · 2 notes
Palmer
John Palmer, a parapsychologist Hyman cites for the argument that the field needs a positive theory before it can call its anomalies paranormal.
The decline effect
Rhine's observation that scoring falls off during a run. Hyman's example of a sign that is taken as evidence when present and ignored when absent.
Source PDF page 90 · 2 notes
Displacement
Scoring against the target before or after the intended one. Hyman notes it was invoked to rescue Soal's failed replications — a rule that cannot lose.
Handbook of Parapsychology
Benjamin Wolman's 1977 reference volume. Hyman cites it as the source list for the field's various alternative signs of psi.
Source PDF page 91 · 2 notes
The Edinburgh group
Robert Morris's laboratory, working under subcontract to SAIC. Hyman notes its replication found introverts scoring higher, reversing the predicted personality correlate.
Only for the dynamic targets
The autoganzfeld found effects on video clips and none on photographs. SAIC's Experiment 1 found the exact opposite. Hyman's evidence that the two paradigms are not the same phenomenon.
Source PDF page 92 · 2 notes
Subjects as their own judges
Ganzfeld works best when the receiver judges; remote viewing is held to work best with an external judge. Hyman uses the contradiction to break Utts's consistency claim.
Because of his familiarity with the response styles
Familiarity with a viewer's habits might help a judge interpret a description, but it also makes the score depend on that particular judge. Hyman asks whether the same material would still score well when other, independently blinded judges assess it.
Source PDF page 93 · 2 notes
Necessary ... never sufficient
Hyman's formal position: rejecting the null is a required step toward claiming an effect and is not by itself a claim about what caused it.
The standards employed by the other sciences
Utts's phrase, which Hyman accepts on the statistics and rejects on the inference. The disagreement is about what statistics are for.
Source PDF page 94 · 2 notes
Thomas Kuhn
The Structure of Scientific Revolutions, 1962. Hyman brings it in to argue that a revolutionary claim needs more than a rejected null hypothesis.
I see it as a minor concern
Hyman ranks his own statistical objection as the weakest of his arguments, and sets it aside to make the two he considers stronger.
Source PDF page 95 · 1 notes
Exemplar
A worked experiment that teaches students how to recognise and reproduce a phenomenon. Hyman asks for such a dependable example. Utts's reply challenges the demand where an effect is too small for a classroom-sized study; see Remote viewing: research and applications evaluation — source PDF page 111.
Source PDF page 96 · 2 notes
Mitogenetic radiation
A claimed cell-division radiation announced by Soviet scientists in the 1920s, pursued for decades and abandoned when nobody could say when it appeared.
N-rays and polywater
Two more effects that were reported, replicated by some laboratories, and eventually dropped. Hyman's precedents for an anomaly that dissolves under scrutiny.
Source PDF page 97 · 2 notes
Two senses of anomaly
Hyman's distinction: in other sciences an anomaly is a precise deviation from a theory; here it is any inexplicable departure from chance.
At the moment this is just a hope
His verdict on entropy and bandwidth — the programme's only candidate for a lawful relationship. Utts rests a good deal on the same finding.
Source PDF page 98 · 2 notes
Leverrier and Neptune
The 1846 case Hyman uses as his model anomaly: Uranus's orbit deviated in a measurable way, a planet was predicted from the deviation, and it was there.
We do not know what we are being asked to explain
The core of Hyman's objection, in one line. Without boundary conditions there is nothing precise enough to have a theory about.
Source PDF page 99 · 2 notes
Approximately 20 percent
May's own estimate to the panel of how much of a viewer's output is accurate — with no way to tell, at the time, which fifth. Chapter Four builds on this.
The matter is not this simple
Hyman separates the two questions: the ability could be real and useless, or useful without being real. He then examines both.
Source PDF page 100 · 2 notes
A catalyst
Hyman's alternative account of any operational value: the viewer stimulates new thinking, as a brainstorming technique would. He adds that cheaper methods exist.
Separation of the operational program
His concern that the operational side was being assessed without reference to the laboratory findings. Chapter Four does what he asks.
Source PDF page 101 · 2 notes
Subjective validation
The tendency to find a match compelling once you are told what it is a match to. Hyman's central psychological mechanism, and his own experiment demonstrates it.
The judge had matched the protocols to the wrong target sites
The case Hyman recounts: viewers and judge were all certain, and all wrong. The certainty survived the discovery.
Source PDF page 102 · 2 notes
A gantry
Hyman's glance at the Semipalatinsk crane, the programme's most famous hit. His argument is about background knowledge and leading questions, not about fraud.
Equally compelling regardless
In his double-blind versions, subjects found the match just as convincing when paired with the wrong target. That is the whole force of the objection.
Source PDF page 103 · 2 notes
It might be worthwhile to allocate some resources
Hyman does not recommend closing the research. He recommends independent replication first, and reassessment after.
The very first consideration
Independent judges. Both reviewers name the same next step, and AIR promotes it to a finding of the report in Chapter Five.
Source PDF page 104 · 2 notes
Statistically and methodologically superior
Hyman's first numbered conclusion is a concession: the SAIC work is better than the SRI work and better than most of the field's.
Too many things ... too few resources
Hyman argues that the ten SAIC experiments were asked to answer too many questions with too few resources. His criticism is of a scattered research design, not an absence of work before SAIC.
Source PDF page 105 · 2 notes
Real effects are occurring
Hyman's fourth conclusion, and the sentence most often left out when this review is quoted. He concedes the effects; he denies the interpretation.
Without a positive theory
His fifth conclusion and the hinge of the disagreement: an effect without a theory cannot be attributed to a single cause, let alone a paranormal one.
Source PDF page 106 · 2 notes
Static and dynamic, reversed
Autoganzfeld found effects only on video; SAIC's Experiment 1 only on photographs. Hyman's evidence that the two literatures are not measuring one thing.
The challenge to parapsychology
Specify when the phenomenon appears, relate it lawfully to something else, and state its boundary conditions. Hyman's terms for what would convince him.
Source PDF page 107 · 1 notes
Suggestions for future research
Hyman ends with two: assess operational effectiveness under blind conditions, and do not divorce that assessment from the laboratory work.
Source PDF page 108 · 2 notes
The Reply
Utts answering Hyman, and the only piece of the exchange written after both reviews existed. It is short, and it concedes more than either review does.
I basically concur with his assessment
Utts's opening move. She disputes three claims about the standing of the field, not his reading of the experiments.
Source PDF page 109 · 1 notes
Maimonides Medical Center
Honorton's dream-telepathy laboratory in Brooklyn. Utts traces ganzfeld, autoganzfeld and the SAIC entropy work back to it as one line of inquiry.
Source PDF page 110 · 2 notes
What causes people to fall in love
Utts's rhetorical parallel: psychology accepts phenomena it cannot explain. Her claim is that observation precedes mechanism in any science.
Electromagnetic fields on health
Her chosen comparison for where she thinks the field stands — an effect argued about statistically while the mechanism is unknown.
Source PDF page 111 · 1 notes
The Physicians' Health Study
The 22,000-man aspirin trial. Utts's answer to the exemplar charge: a small effect needs a large study, and no undergraduate class could run one.
Source PDF page 112 · 2 notes
AIR's voice resumes
The contractor writes again from here to folio 3-88, summarising where the reviewers agree. The reviews themselves end on the previous leaf.
Converged upon a number of issues
AIR's framing: it treats the exchange as having produced agreement, and organises the rest of the chapter around the points of it.
Source PDF page 113 · 2 notes
Monomethod bias
AIR's term for the report's central methodological finding: the same viewers, judge, target set and scoring across nearly every result, so a quirk of the method could produce it all.
A running head that lags
This recto still reads 'The Reply: Dr. Jessica Utts' although the text beneath it is AIR's own. The heads do not catch up before the chapter ends.
Source PDF page 114 · 2 notes
Criterion contamination
Unconscious influence on a result by whoever scores it. AIR raises it because the Principal Investigator judged his own experiments, and stops short of alleging it happened.
If laboratory research is to continue
AIR's conditional. The chapter does not recommend closure of the research; Chapter Five's flat conclusion is about the operational programme.
Source PDF page 115 · 1 notes
We agree with Dr. Utts
AIR takes Utts's side on one point: that a phenomenon's rarity is no reason not to study it. It then takes Hyman's on what follows for practical use.
Source PDF page 116 · 1 notes
The same viewers, the same judge, the same target set
AIR's own statement of the methodological problem, drawn from both reviews. It becomes a numbered finding in Chapter Five.
Source PDF page 118 · 1 notes
Chapter Four
AIR changes the test here: suppose remote viewing exists, then ask what an intelligence officer could do with its output. The chapter compares laboratory conditions, current users' accounts and the scored tasks in Appendix D.
Source PDF page 119 · 2 notes
For the moment, assume that the phenomenon does indeed exist
A temporary assumption, not a reversal of Chapter Three's scientific finding. AIR asks whether a real ability would survive the move from controlled experiments to operational targets and decisions.
Boundary conditions
The conditions under which an effect appears, changes or disappears. Laboratory targets, senders, feedback and judging can be controlled; an intelligence problem may supply none of those conditions. The following pages test that transfer one condition at a time.
Source PDF page 120 · 2 notes
Only in the case of missing persons
The one operational task where a sender might exist. Everywhere else the laboratory's setup has no counterpart in the field.
The laboratory's sender
In the SAIC work the nearest thing to a sender is the assistant fetching a photograph, or the computer choosing it. AIR notes the paradigm effectively has none.
Source PDF page 121 · 2 notes
Diverse and novel
Operational targets ranged from the location of ships to a person's background. The laboratory effect was found on a familiar, fixed pool of photographs.
No reason to assume
Even granting the bandwidth theory, AIR points out that intelligence targets do not arrive pre-sorted into a favourable range.
Source PDF page 122 · 2 notes
An unknown percentage
Every viewing contains material unrelated to the target, and nobody can say which part at the time. This is the practical form of the accuracy problem.
Feedback
In the laboratory the viewer is shown the selected target afterwards. That supplies a known answer against which to learn. In operations, the target is often unknown to the customer too; an unverified report cannot provide the same feedback loop.
Source PDF page 123 · 2 notes
Inter-rater agreement
Whether two analysts reading the same viewing reach the same conclusion. No evidence existed either way, which AIR treats as a limitation in itself.
Prior practice
Laboratory viewers worked a familiar protocol repeatedly. AIR asks whether results from practised conditions transfer to unrehearsed operational ones.
Source PDF page 124 · 1 notes
One method, judgmental matching
AIR criticises the remote-viewing studies' reliance on judgmental matching: results tied to one method provide a weak basis for generalising to intelligence work.
Source PDF page 125 · 2 notes
Provide information otherwise difficult or impossible to obtain
Users described seeking information that ordinary channels could not readily supply. That explains willingness to try the service, but not whether the resulting descriptions were accurate or useful; AIR asks those as separate questions.
On an exploratory basis
The recurring phrase in the interviews. No user group had adopted the technique; all were trying it out.
Source PDF page 126 · 2 notes
General stereotypical characteristics
What the reports got right — and AIR notes at once that background knowledge would produce the same thing.
Never used as a primary source
Not even by the most favourable user group. This is the finding Chapter Five turns into the report's conclusion.
Source PDF page 127 · 2 notes
Automatic writing, meditation, and channelling
The techniques the three viewers described using. AIR records them without comment, in a list, in a federal evaluation.
Primarily selected on the basis of their interest in parapsychology
Only one viewer had been through a formal screening. Selection was largely by enthusiasm.
Source PDF page 128 · 2 notes
The Congressional mandate
The Program Manager's account of why clients came: they were directed to, not persuaded. The programme's demand was partly manufactured.
Rapid turnaround
The service's real selling point, by the manager's own account — speed and difficulty of the question, not accuracy of the answer.
Source PDF page 129 · 1 notes
Eleven participants
AIR states its own limits here: a small qualitative sample, current operations only, and no counterfactual about ideal conditions.
Source PDF page 130 · 2 notes
Blind alleys
AIR's phrase for the operational cost: not merely useless information, but information expensive to chase and easy to read into.
They modified reports
The viewers said they adjusted reports to fit known background. AIR treats this as fatal to interpreting any hit, and Chapter Five repeats it.
Source PDF page 131 · 1 notes
A representative of the U.S. Government
The only attribution Appendix D ever gets. Its author is never named, on this page or on its own.
Source PDF page 132 · 2 notes
Average accuracy 3.0, average value 3.5
These are customer ratings, not laboratory target ranks or percentages correct. The value scale's 3 means 'of value', 4 'low value' and 5 'no value'; a mean of 3.5 lies between two verbal categories, not at a success rate of 70 percent. Appendix D gives the separate accuracy scale at Remote viewing: research and applications evaluation — source PDF page 185 and value definitions at Remote viewing: research and applications evaluation — source PDF page 186.
No one viewer performed exceptionally well
All three fell within a tenth of a standard deviation on accuracy — no star performer, which cuts against the selected-talent argument.
Source PDF page 133 · 1 notes
Could be acquired in other ways
Where the viewings were useful at all, AIR found the same information was obtainable by ordinary means.
Source PDF page 136 · 2 notes
Two expert reviewers
AIR summarises an advocate and a sceptic who agree that there are statistical effects but disagree about what those effects establish. Its numbered conclusions distinguish significance, cause, mechanism and practical use.
A statistically significant effect has been observed
The report's first numbered finding, and the one most often quoted alone. The three that follow qualify it.
Source PDF page 137 · 2 notes
Apparently the Principal Investigator
AIR's own hedge about who did the judging. It had not been able to establish it beyond doubt, and says so in the word 'apparently'.
Subtle, unintentional factors
The furthest AIR goes. It names the risk of unconscious influence and does not allege that any occurred.
Source PDF page 138 · 2 notes
A significant change in focus and methods
The condition AIR sets for further laboratory research. It is not a recommendation to stop researching.
This is not to say definitively that paranormal phenomena do not exist
AIR distinguishes an inadequate demonstration from a proof of nonexistence. Its next conclusion is nevertheless operationally decisive: continued support for the current operational programme is not justified.
Source PDF page 139 · 2 notes
Reports were changed
Viewers and managers told AIR that reports had been altered to fit known background cues. AIR says this casts doubt on the celebrated hits.
Not justified
The last line of Chapter Five, and the sentence the programme did not survive. It is about the operational component, not the research.
Source PDF page 141 · 1 notes
A very abbreviated ... version
Hyman's footnote says this is a very abbreviated and somewhat modified version of his full resume. His vita occupies four leaves; Utts's occupies eight.
Source PDF page 142 · 2 notes
The Stanford chair
Hyman held Stanford's Thomas Welton Stanford Chair for Psychical Research in 1982–83. The appointment is one example in the vita of a sceptical psychologist working inside institutions devoted to studying these claims.
Committee for the Scientific Investigation
CSICOP, the sceptics' organisation founded in 1976. Hyman was a founder and sat on its executive council; the vita lists it without comment.
Source PDF page 143 · 1 notes
The Elusive Quarry
His 1989 book on psychical research, published by Prometheus. It is the fuller statement of the position Review 2 compresses into thirty-six pages.
Source PDF page 144 · 2 notes
Uri Geller
The performer associated with claims of paranormal metal bending. Hyman lists observation of Geller among his professional experience; the same vita records his own paid work in mentalism and palm reading.
I earned my way through college performing mentalism and reading palms
Hyman's own sentence. His argument about subjective validation at Remote viewing: research and applications evaluation — source PDF page 101 is written by someone who used to produce the effect for money.
Source PDF page 145 · 2 notes
SRI International, Visiting Scientist, 1987-88
Utts spent a year in the Cognitive Sciences Program at SRI — the laboratory whose work she was reviewing. The vita states it; the report's own text never mentions it.
University of Edinburgh, Winter 1994
A visiting fellowship in the psychology department whose ganzfeld replication Review 1 cites as independent corroboration.
Source PDF page 146 · 2 notes
Institute of Mathematical Statistics
Utts was its Treasurer from 1988 to 1994 — the same society whose president defended parapsychology's statistics in 1937, as Hyman recounts at Remote viewing: research and applications evaluation — source PDF page 76.
Parapsychological Association
Listed alongside the American Statistical Association and the Biometric Society. She was its representative to the AAAS from 1989.
Source PDF page 148 · 2 notes
The Ganzfeld Debate: a statistician's perspective
Utts's 1986 paper. Her role in the field began as the statistician brought in to adjudicate an argument between Hyman and Honorton.
May, E.C. ... and J.M. Utts
She co-authored papers with the programme's Principal Investigator from 1985 onward, including the 1988 SRI review Review 1 relies on.
Source PDF page 149 · 1 notes
Successful replication versus statistical significance
Her 1988 paper. The effect-size argument Review 1 rests on was her published position seven years before this evaluation.
Source PDF page 150 · 1 notes
Honorton the Meta-Analyst
Her 1993 assessment of Charles Honorton's statistical work. Hyman's review argues against much of the same body of analysis.
Source PDF page 151 · 1 notes
Technical reports
A separate numbered list of unpublished University of California and Stanford reports. Several results Review 1 relies on exist only in this form.
Source PDF page 153 · 1 notes
Appendix B
The bibliography: the list of documents the CIA transferred to AIR, with starred entries marking Edwin May's own nomination of the strongest evidence.
Source PDF page 154 · 1 notes
MDA908-93-C-0004
A Defense Intelligence Agency contract number. Most of the research reviewed in this report exists only as deliverables under this contract and MDA908-91-C-0037.
Source PDF page 155 · 2 notes
Mass Screening for Psychoenergetic Talent
The 1988 SRI report behind the Executive Summary's claim that about one percent of volunteers scored consistently.
The starred entries
A star marks a document from May's list of the strongest evidence, as Chapter Two explains in Part One, original folio 2-3. The stars are the programme's own self-assessment.
Source PDF page 156 · 2 notes
(U)
Unclassified, marked title by title. The marking survives on the titles of documents that were classified when they were written.
Periodic Status Report
Much of the bibliography is contract progress reporting rather than published science. This is Hyman's point about peer review, in list form.
Source PDF page 157 · 1 notes
Technical Protocol for the MEG Investigation
The protocol for Experiment 3, the magnetoencephalography study SAIC abandoned once the signal proved smaller than the noise.
Source PDF page 158 · 1 notes
Review of the Psychoenergetic Research Conducted at SRI International (1973-1988)
The starred 1989 report that Review 1 leans on throughout — and Jessica Utts is one of its six authors.
Source PDF page 159 · 1 notes
SAIC Papers
A separate unauthored section: contract proposals and modifications, listed by title rather than by person.
Source PDF page 160 · 1 notes
Audiolinguistic Correlations
A 1982 study of whether how a viewer speaks predicts how well they view. The titles are the best surviving index of what the programme actually tried.
Source PDF page 161 · 2 notes
Geophysical Effects Study
Interim and final reports, 1984. The programme tested whether local geophysics affected viewing; the result is not reported here.
Free World Psychoenergetics Research Survey
A 1983 survey of non-Soviet work in the field — the foreign assessment leg, which this evaluation was told to leave alone.
Source PDF page 162 · 1 notes
Special Orientation Techniques
Six reports under one title, S-I through S-VI, from 1979 to 1984. The euphemism is the programme's own.
Source PDF page 163 · 1 notes
Appendix C
The interviewer's instructions, the three question sets, and the reports of the user, viewer and manager interviews. It is the raw material behind Chapter Four.
Source PDF page 164 · 2 notes
All information ... will be treated as confidential
The undertaking given at the start of every interview, which is why no interviewee is named anywhere in the report.
A not-for-profit research organization
How interviewers were told to describe AIR. The interviewees were told the CIA was conducting the evaluation.
Source PDF page 165 · 1 notes
Returned to Dr. Michael Mumford
The instructions direct all completed interview material back to Mumford, one of AIR's two behavioural scientists on the panel.
Source PDF page 166 · 2 notes
Twelve questions for users
They end with the sharpest one in the appendix: would you pay for the services of a remote viewer.
Question 8
The question separates corroborating an existing lead from supplying usable intelligence on its own. AIR returns to this distinction when users report that they could not rely on viewings without information from other sources.
Source PDF page 167 · 2 notes
Fifteen questions for viewers
Recruitment, training, technique, tasking, and what would have made the viewings better. The viewers were asked to diagnose their own programme.
Question 10
How viewings by different viewers on the same tasking compared. The answers describe reports that frequently disagreed.
Source PDF page 168 · 2 notes
Questions for the Program Manager
The manager is asked about clients, tasking, information supplied to viewers and organisational support. Comparing these answers with the viewers' accounts separates the programme's intended procedure from the way sessions were actually conducted.
Question 13
How often viewings had a major influence on operational decisions. The manager's answer was none.
Source PDF page 169 · 1 notes
Interview #1
7 July 1995. A unit commander who had used remote viewing exactly once, on a suspected espionage case, after hearing about its use in police work.
Source PDF page 170 · 2 notes
Hits were often stereotypic
The user's own analysis: the accurate material was the kind that background cues would supply. Chapter Four builds its accuracy finding on this.
Used objectively without forcing them into preconceptions
AIR notes the user did not read the viewings favourably. The failure was not one of hostile handling.
Source PDF page 171 · 2 notes
Ships and criminal organizations
This user group's targets: people in criminal organisations, ships and their cargo, and manufacturing or shipping sites. Counter-narcotics work, not described as such.
Described in terms of names and locations
What the viewer was given to work from. Chapter Four's point about background cues starts here.
Source PDF page 172 · 2 notes
Relatively few alternative sources
This is the most favourable user in the appendix, and the reason is scarcity: its other intelligence was indirect too.
Not deemed useful for operations
Even here. The viewings filled in background and never supported an operational decision.
Source PDF page 173 · 2 notes
Convenience
Targets chosen for being near the office. One user group had reduced its use of the service to local exercises with a beacon.
Three- to four-page report
The standard product: drawings and a verbal description, produced independently by each viewer for each tasking.
Source PDF page 174 · 1 notes
Under the auspices of a formal government program
This user's condition for ever using it: credibility came from the programme's official standing, not from its results.
Source PDF page 175 · 2 notes
Six to 18 months long
The viewers' initial training. There was no refresher training; the viewers kept up by reading the parapsychology literature themselves.
Coordinate viewing, meditation, channelling, automatic writing
The three viewers' techniques, all different. None of them is the laboratory protocol the research was validating.
Source PDF page 176 · 2 notes
Reports ... brought into line with the known characteristics of the target
The viewers' own account of editing their reports to fit background information. Chapter Five calls this particularly troublesome.
Rather different reports
Different viewers on the same tasking produced materially different results, which the viewers attributed to using different techniques.
Source PDF page 177 · 2 notes
The last manager
He took the programme over in 1991 when his predecessor retired. His background was human intelligence, not parapsychology.
In accordance with congressional guidelines
The manager's charter was to expand operational use. The demand Chapter Four measures was one he had been directed to create.
Source PDF page 178 · 2 notes
A one- or two-word description
Usually all the viewers got, along with the sponsoring organisation's name. It is also the background against which any hit must be read.
Did not explicitly consider the findings
The manager knew the research literature and did not use it to run the operation. Hyman's objection in Part One, original folio 3-71 is exactly this.
Source PDF page 179 · 1 notes
Obtained only as needed on a contractual basis
The manager's own suggestion for the programme's future: keep research and foreign assessment, and buy viewing in when required.
Source PDF page 180 · 1 notes
Appendix D
The one part of the report not written by AIR or by the reviewers. Chapter Four attributes it to 'a representative of the U.S. Government'; no page of it carries a name.
Source PDF page 181 · 2 notes
More than 200 tasks
The workload from 1986 to early fiscal 1995. Appendix D examines the 40 tasks covered by the numerical evaluation introduced in 1994. A long operating history and a much shorter scored record are different measures of the programme.
Telegraphing
Letting the answer sought show through the question. Sparse tasking was intended to avoid such cues, but it also left customers without a precise question against which to judge the response. The appendix treats both as practical problems.
Source PDF page 182 · 2 notes
Until 1994
Numerical evaluation began in 1994; the author reports finding no earlier numerical accuracy evaluation for this period of operations. The appendix therefore examines that recent scored record, not all of the more than 200 tasks listed in its introduction.
Five operational organizations
Named throughout only as Org. A to Org. E. The report protects them by letter while describing their work in detail.
Source PDF page 183 · 2 notes
Figure 2-1
A Star Gate tasking sheet, with a date, source number and target instruction entered. It shows how a request reached a viewer.
Essential Elements of Information: Access and describe target
The instruction asks for a description without spelling out what would count as success. Later the author argues that such open tasking makes it hard to separate a useful answer from something an evaluator can fit to the target; see Remote viewing: research and applications evaluation — source PDF page 193.
Source PDF page 184 · 2 notes
Twenty-six of the 40
Most of the scored tasking came from DIA in support of two joint task forces. Two organisations account for the bulk of the evidence.
The identity of the supported organizations also was provided
Knowing which organisation requested a viewing could suggest the sort of target involved. The author identifies that as a possible source of ordinary information, complicating any inference from an accurate description to a paranormal ability.
Source PDF page 185 · 2 notes
DIAM 58-13
The Defense Intelligence Agency manual from which the ratings were taken. Accuracy has six categories; value has five. The author's footnote warns that accuracy codes 5 and 6 are not simply worse than 4, which means 'No'. These are labelled judgements, not an evenly spaced measure of how much of a report is correct.
Whether the individual evaluators reviewed these definitions ... is unknown
A numerical scale only standardises a judgement if users apply its definitions alike. The author could not establish that these evaluators had reviewed them; he later compares organisations whose scores diverge on similar targets.
Source PDF page 186 · 2 notes
Major significance to no value
The five-point value scale, quoted from DIAM 58-13. A 1 means a report that alters national policy; a 5 means it was worth nothing.
Most DoD HUMINT System reporting falls into this category
The manual's own note on 'of value' — a 3. It puts the programme's average value score of 3.5 in perspective.
Source PDF page 188 · 2 notes
Table 4-1
Every scored task, one row each: document number, date, tasking organisation, evaluator number, and an accuracy and value score from each of the three viewers.
1A, 1V, 2A, 2V, 3A, 3V
A and V mean accuracy and value; the preceding number identifies the viewer. Blank cells are unscored entries, not zeroes. The same description can receive different marks on the two measures: being partly right is not the same as helping an operation.
Source PDF page 189 · 2 notes
Broken out by organisation
The same scores regrouped by tasking organisation, with sums and averages. This is where the difference between Org. B and Org. C becomes visible.
Averages near 2 and 3
Org. E's accuracy averages run from 4.0 to 5.0, while Org. D's run from 1.8 to 2.2. Those are averages of category codes, not percentages correct: accuracy code 6 means 'Unsure', while code 4 means 'No'. Read the scale at Remote viewing: research and applications evaluation — source PDF page 185 before treating the numerical spread as a measure of performance.
Source PDF page 190 · 1 notes
Org. C's nineteen tasks
The largest single block, with more favourable scores than Org. B. Folio D-15 explains why, and the explanation is not about accuracy.
Source PDF page 191 · 2 notes
No 1s among the 100 value scores
Not one report in the scored record was rated of major significance. Thirteen accuracy scores were 1s, and the author discounts nine of them.
Widely publicized in the open media
Five of the thirteen accurate calls concerned topics that were in the newspapers at the time. The author says the source of the information is suspect.
Source PDF page 192 · 2 notes
0.7 to 1.5 marks lower
Here 'lower' means less favourable ratings in the author's comparison, not smaller printed numerals: on the scales, 1 is best. He links the gap between Org. B and Org. C to evaluators' different attitudes, developed at Remote viewing: research and applications evaluation — source PDF page 195.
Standard deviations
Value scores cluster more tightly than accuracy scores, which the author reads as evaluators understanding 'value' more consistently than 'accuracy'.
Source PDF page 193 · 2 notes
Free association or stream of consciousness
The author's characterisation of what the viewers produced, from reading the operational folders himself. It is the appendix's most quoted phrase.
Makes evaluation difficult if not impossible
Vague tasking plus unstructured output leaves nothing objective to score. The author's central complaint is about the process, not the phenomenon.
Source PDF page 194 · 1 notes
North Korea and the Middle East
Two bodies of tasking the author says could be checked against what actually happened. The check was not made, and the results are not in this report.
Source PDF page 195 · 2 notes
Free
The author identifies free access as one reason organisations kept trying the service. A willingness to accept reports at no charge is not the same decision as committing funds; the later user interviews make that distinction explicit.
Skeptical ... a believer in parapsychology
The Org. B and Org. C evaluators respectively. The author names belief as the likeliest cause of the scoring gap, and says it impugns nobody's honesty.
Source PDF page 196 · 2 notes
Either wrong or irrelevant
The author's summary of a typical viewing, with the qualification that nobody can tell which is which without going to check.
Adds to the overburden
His operational objection: an analyst already drowning in unreliable material does not benefit from more of it.
Source PDF page 197 · 2 notes
Some scientific method
The author's only recommendation, and it is conditional: if the process continues, make it measurable. He is not optimistic that it can be.
Blind reading of playing cards
His example of the only kind of experiment he thinks can be properly controlled — and he notes even those have produced controversy.
Source PDF page 198 · 2 notes
The North Korean targets
Where targets were more specific, the scores were higher. It is the appendix's one constructive finding, and it argues for narrower tasking.
Faith ... or conversely disbelief
The author's assessment of what more testing would achieve. He does not expect it to settle the value question.
Source PDF page 200 · 1 notes
Next 1 Page(s) In Document Exempt
The CIA's placeholder is marked SG1A. It says one page is exempt, but the surrounding folios jump from D-19 to D-22, leaving D-20 and D-21 absent. The release does not explain that discrepancy.
Source PDF page 201 · 2 notes
Appendix B to Appendix D
The operational user interviews, conducted July and August 1995 — a separate set from the AIR interviews in Appendix C, by a different interviewer.
Beacons
Org. A used the service for beacon experiments: staff visited sites and the viewers were asked to describe what they were seeing.
Source PDF page 202 · 2 notes
Would be willing to pay ... if it was not too expensive
This user would pay if the price were low enough. Read beside Org. B's refusal to commit funds on the next leaf: willingness to keep trying a service and a judgement that it has demonstrated value are separate findings.
Commercial sources
Told they could buy the same service on the open market, the user objected that their work was too highly classified. The programme's advantage was its clearance.
Source PDF page 203 · 2 notes
Demonstrated value and utility are not adequate
Org. B would use the service again and would not fund it. That distinction is the operational finding in one line.
May not be best qualified
One user suggested recruiting viewers from outside government, on the view that the programme's own three were not the best available.
Source PDF page 204 · 2 notes
Historical events ... more accurate than predictions
This user's own pattern: descriptions of the past scored better than forecasts. Nobody in the report follows the observation up.
No audit trail
Some viewings went into target packages for field operatives, and no record was kept of what happened next. The most favourable claim in the appendix cannot be checked.
Source PDF page 205 · 2 notes
Sequential tasking
Org. E ran four linked tasks rather than one-off questions — the only user to do so. None of the four produced anything specific enough to act on.
Willing to try anything
The user's attitude after the experience was unchanged by the result. The author records it without editorialising.