n January 1979, The New York Times reported that despite repeated, feverish denials, the CIA had indeed investigated the UFO phenomenon: "CIA Papers Detail UFO Surveillance" screamed the headline. The report is said to have so upset the then CIA director, Stansfield Turner, that he reportedly asked his staff: "Are we in UFOs?"

The answer was yes - since the late 1940s, apparently. But exactly how, what, when, why and who remained layered in mystery, leaving grist for the conspiracy mill.

But this year a raft of newly unclassified CIA documents revealed that the remote possibility of alien invasion elicited greater fear than the threat of a Soviet nuclear attack.

More interesting still, the CIA documents show that despite decades of repeated public denials, behind the scenes there raged a series of inter-agency feuds that involved the highest levels of the US government.

The subject of UFOs - and dabbling in psychological warfare techniques - not only focused the attention of the US government elite for 50 years, but of some of the greatest scientific and military minds of the era.

Throughout the 1950s CIA files clearly document an explosion of activity by US intelligence and military bodies concerned with studying every possible implication for the US, and other Western democracies, of UFOs. The phenomenon, so adored by the cinematic world, was reflected in the CIA's fixations. Indeed, while highly educated CIA employees experimented by giving each other surprise LSD trips in 1953, there were others, in other parts of the agency, dealing with a flood of UFO reports.

But significantly, after a burst of intense scrutiny in the early '50s, the available documents effectively go cold. Why? The Kafkaesque explanation provided is that few files were kept because these would only confirm that the CIA was investigating UFOs. A 1995 CIA review stated: "There was no formal or official UFO project within the agency in the '80s, and agency officials purposely kept files on UFOs to a minimum to avoid creating records that might mislead the public if released."

But the wildly eclectic UFO files cover everything from "flying saucers over Belgian Congo uranium mines" to Nazi "flying saucers".

A 1953 memo shows that the physicist John Wheeler, while critically involved with Edward Teller in the creation of the hydrogen bomb, was available to the "CIA attack on the flying saucer" problem. The urgency of the H-bomb race was his priority, but he "would be pleased at any time to discuss the issue briefly", the memo said.

Wheeler recommended two "foreign nationals" who could help with the "problem", including the "mysterious problems of ion paths and magnetic focusing" and "cosmological electrodynamics".

A secret 1995 report was titled: CIA's role in the study of UFOs 1947-90: a diehard issue. Collated and written by Gerald Haines, the CIA's National Reconnaissance Office historian, its detailed summary of CIA involvement inadvertently undermined its "UFOs-don't-exist" conclusion. The document begins with a June 24, 1947, report from the pilot Kenneth Arnold, who spotted nine unidentified objects near Mount Rainier, Washington state, travelling at an estimated 1600 kmh. Haines did not mention that days later, on July 8, 1947, the Roswell Daily Record reported a US Army press release below the headline "RAAF captures flying saucer on ranch in Roswell region".

The report noted that that controversy, coloured with Byzantine denials, dogged the CIA and its UFO investigations for decades. Using operational names like Project Blue Book, Story, Grudge, Sign, Saucer, Moon Dust and Twinkle, the US Air Force and other entities always looked into UFO sightings with the CIA peering over their shoulders.

The US Army, of course, promptly retracted the Roswell story but it and the "flying saucers" spotted by Arnold triggered a flurry of sightings and conspiracy theories that continue to this day.

The US Air Force finally admitted in 1994 that there had been a cover-up at Roswell - of a secret project known as Mogul, created to monitor Soviet nuclear tests using high-flying balloons - and that the "aliens" were crash-test dummies.

"Ufologists", naturally, were sceptical of this belated explanation. For 50 years now, right across the globe, people have been reporting sightings of giant, luminous flying saucers, cigars, globes, triangles and doughnuts. Aliens have allegedly abducted, probed and impregnated scores of hapless earthlings. Some believe that a top-secret entity, called Majestic-12, was formed in 1947 by the then president, Harry Truman, in an attempt to deal with the Roswell event. It was supposedly established to aid interaction with aliens. The FBI labelled the Majestic-12 documents a hoax, but the story persists to this day.

Intriguingly, the unclassified documents show that within the CIA, there was an uber-intelligence group called ONE, created by a CIA director, General William Bedell Smith. His tenure spanned the period between October 1950 and January 1953. These documents confirm that ONE was concerned with UFOs.

In 1978 the CIA came under strong pressure from a series of freedom of information requests about UFOs and reluctantly released about 800 documents. The reasonable claim by The New York Times at the time was that the files confirmed intensive government concern about UFOs.

This was branded by the CIA as the press being sensationalist. According to the CIA's self-critique on the issue, bureaucratic clumsiness, charges that witnesses were being asked to keep sightings secret, and CIA officers talking to civilians about UFOs while wearing air force uniforms, had added "fuel to the growing mystery surrounding UFOs and the CIA's role in their investigation". The 1995 Haines report concluded: "The belief that we are not alone in the universe is too emotionally appealing and the distrust of our government is too pervasive to make the issue amenable to traditional scientific studies of rational explanation and evidence."

My painstaking review of hundreds of unclassified documents reveals that the CIA at the highest level, far from being incompetent, displayed good faith in its efforts to examine the mystery of UFOs over a period of decades. These investigations covered a gamut of inquiries: scientific, political, cultural and military.

And although the air force was the agency given the task of investigating UFOs from 1948 onwards, the CIA remained deeply involved. This is best reflected in a memo to the agency's deputy director for scientific intelligence, titled Flying Saucers and dated August 3, 1952: "It is recommended that CIA surveillance of subject matter (flying saucers), in co-ordination with proper authorities of primary operational concern at the Air Technical Intelligence Centre (ATIC), be continued. It is strongly urged, however, that that no indication of CIA interest or concern reach the press or public, in view of their probable alarmist tendencies to accept such interest as 'confirmatory' of the soundness of 'unpublished facts' in the hands of the US government."

Although most reports were "phoney" or explainable, it said, "caution requires that intelligence continue coverage of the subject".

On July 28, 1952, Winston Churchill wrote to his secretary of state for air: "What does all this stuff about flying saucers amount to? What can it mean? What is the truth?" The minister's response on August 9, 1952, provided the ground rules for most official responses that continue until today. These were that a 1951 study had found that all reports could be explained by astronomical or meteorological phenomena, mistaken identification of aircraft, balloons, birds, optical illusions and psychological delusions, or were deliberate hoaxes.

But in the CIA at the time, two other responses were countenanced: the need for vigilance and caution because extraterrestrial life could exist, and the potential for "psychological warfare", including fears that popular hysteria could be exploited by an enemy.

The sceptics are best represented in a memo in March 1949 from a Dr Stone in the CIA Office of Scientific Intelligence to a Dr Machle that states: "A rapid perusal of your [flying saucer] documents leaves one confused and inclined to supineness."

Yet with a deluge of UFO reports over the next four years, the matter suddenly assumed a modicum of gravitas, reflected in many top-secret documents. General Smith said: "There was one chance in 10,000 that the phenomenon posed a threat to the security of the country, but even that chance could not be taken." On July 1, 1952, there was an about-turn: General Smith wrote to the director of the Psychological Strategy Board established by Truman the previous year: "I am today transmitting to the National Security Council a proposal in which it is concluded that the problems associated with unidentified flying objects appear to have implications for psychological warfare as well as for intelligence and operations. I suggest that we discuss at an early board meeting the possible offensive and defensive utilisation of these phenomena for psychological warfare purposes."

Searching for this "proposal", I found versions addressed also to the secretary of defence. Some of their highlights, quoting directly from the documents, include: "[Since] 1947 there have been about 1500 official reports of sightings and [of these] the air force carries 20 per cent as unexplained." And: "Operational problems are of primary importance and should be attacked at once [including] determination of what [use could] be made of these phenomena by US psychological warfare planners and what … defences should be planned in anticipation of Soviet attempts to utilise them."

This memo suggested a plot that transcends Stanley Kubrick's Dr Strangelove: the CIA, in the face of unknown phenomena - or even an attack from outer space - was seemingly more concerned about what the Russians might do with UFOs than with the objects themselves. The CIA's interest in the Soviet and Chinese study of UFOs continued for decades. But on October 2, 1952, General Smith received this ominous note from his Office of Scientific Intelligence: "Flying saucers pose two elements of danger which have national security implications. The first involves mass psychological considerations and the second concerns the vulnerability of the US to air attack." In January 1953 the Office of Scientific Intelligence convened a committee to review the UFO "problem". Its members reviewed "75 case histories of sightings", taking intense interest in a Tremonton, Utah, sighting that included a Kodachrome movie of "1600 frames".

At the air force's request, the US Photo Interpretation Laboratory spent 1000 hours making "graph plots" of the film frames, concluding that the objects were not birds, balloons, aircraft or reflections and that they were "self-luminous". In a tone of reasonable scepticism, it suggested that the public be educated to avoid hysteria.

But the Office of Scientific Intelligence panel dismissed the military conclusions, suggesting instead that the mysterious objects were seagulls reflecting sunlight.

On January 21, 1953, another memo concluded that the panel had found no evidence of "physical threat to the security of the US". The convoluted memo stated: "The subject UFO is not of direct intelligence interest. It is of indirect intelligence interest only insofar as any knowledge about innumerable unsolved mysteries of the universe are of intelligence interest." But it also noted the potential for "interference with air defence by intentional enemy jazzing", the possibility of interference by "overloading communication lines", or the possibility of "psychological offensive by the enemy timed with respect to an actual attack".

This report and the original Tremonton "seagull" film were then made part of an Office of Scientific Investigation briefing on January 29, 1953, to the entity known as ONE. The air force briefed ONE on UFOs the next day and its 11 members included "Dr Edgar Hoover [sic], William Bundy, General H. Pull and Admiral B. Bieri [Eisenhower's chief of staff]".

These documents reveal that ONE was an elite think tank within the CIA and that General Smith created the Office of National Estimates on the issue.

But it was said its "ultimate approval should rest on the collective judgment of the highest officials in various intelligence agencies". This was to give it the prestige of the best available and most authoritative advice from the government.

General Smith created the Office of National Estimates under the auspices of the National Security Act of 1947. His opinion was that ONE would form the "heart of the CIA and of the national intelligence machinery".

William Langer, a Harvard historian, was its chairman, and while there is no record of whether ONE thought the Tremonton film showed seagulls or UFOs - or of what the air force told them the next morning - ONE is as close as we get to a documented version of the rumoured Majestic-12 group.

With the Cold War in full swing, the CIA was also watching for UFO activity behind the Iron Curtain. Field stations were to be alerted to any mention of flying saucers by Iron Curtain countries and the CIA discovered that the Soviet establishment mirrored its own ambiguity about UFOs.

The files spotlight Soviet articles in 1968 that show some scientists thought they were real, while others ridiculed the sightings as US propaganda.

One Soviet sceptic noted, with tongue firmly in cheek: "The number of saucers always grows sharply on the eve of presidential elections. This is difficult to explain.

"Maybe people on other planets lay bets on who will win in the next elections - the Republicans or the Democrats."