In 1968 the film 2001: A Space Odyssey was released. Produced and directed by Stanley Kubrick, with a screenplay he wrote in collaboration with Arthur C. Clarke, 2001 broke new cinematic ground and remains a stunning visual achievement. Currently ranked 15th on The American Film Institute’s “100 Greatest American Films of all Time” webpage and 6th on the list of “The Greatest Films of All Time,” as compiled by the British Film Institute, 2001 is the highest ranked science fiction movie on both lists.
The film is set in three time periods: the “dawn of man,” the late 20th century, and 18 months later. The third time period follows the 5-person crew of the spaceship Discovery–three of whom are in a state of suspended animation–on their long voyage to Jupiter, the true purpose of which is unstated. In addition to the two men who are conscious, Dr. Dave Bowman and Dr. Frank Poole, the third main character in this part of the film is the computer known as HAL-9000. HAL, as it is addressed by the crew, is a computer that exhibits human-like traits. It observes the crew and all areas of the spaceship, hears what Dave and Frank say, controls the life support of crew members in suspended animation, verbally responds to inquiries, asks questions of the crew, monitors the ship, and generally runs most of its functions. HAL appears to be all-seeing, all-hearing, all-knowing, and ever-present.
Many who viewed the film when it was first released and since then noticed that the letters in HAL each occur one letter ahead of the letters I, B, and M, respectively, in the alphabet. This did not seem to be a coincidence. For several decades of the 20th century, IBM was almost synonymous with computers, dominating the computing field in a way that might be comparable to Microsoft and Apple combined into one company today. Of course, IBM is an acronym, or more precisely an initialism, for International Business Machines. The “international” part of IBM is significant. During the twelve years of Nazi rule, Germany relied on IBM punch card and card sorting equipment. According to the United States Holocaust Memorial Museum’s online Holocaust Encyclopedia, “Hollerith machine technology [produced by an IBM subsidiary] was introduced at the headquarters of the concentration camp system in 1942 and was used there not to locate and identify people to be killed (this was done by paper record) but as a way of managing and tracking data on living prisoners. This was particularly true with regard to their professional and occupational skills, some of which the Germans needed to support their war production.”1https://encyclopedia.ushmm.org/content/en/article/locating-the-victims As Edwin Black writes in IBM and the Holocaust: The Strategic Alliance Between Nazi Germany and America’s Most Powerful Corporation, “Solipsistic and dazzled by its own swirling universe of technical possibilities, IBM was self-gripped by a special amoral corporate mantra: if it can be done, it should be done” (8; italics in the original). On the positive side, the Holocaust Encyclopedia also notes that “The Nuremberg Trial was an early experiment in simultaneous translation” using a translation system developed by IBM.2https://encyclopedia.ushmm.org/content/en/article/the-nuremberg-courtroom
In the film, 2001, we learn that pre-flight tests of HAL showed it completed all its tasks without fail, never showing any sign of error in carrying out its computations. It seemed to be infallible. Anyone who has not seen the film and doesn’t want to learn about a critical plot twist is advised to stop reading now.
While on the voyage to Jupiter, HAL detects a potential problem with one of the ship’s communication components, on the exterior of the spaceship, and determines the part will likely soon fail. After the crew replaces the suspect part with a spare, onboard tests show the original device to be in fine working order. At HAL’s suggestion, the crew agrees to reinstall the original part and let it fail, which would then allow the crew and HAL to identify the problem with certainty. Although Dave and Frank are advised by Mission Control that results from a duplicate HAL-9000 on Earth indicate the part in question is not defective, and thus their onboard computer likely made an error, HAL attributes the discrepancy between the two computers to human error.
But this raises a problem for the human crew. What if the component that HAL determined would soon fail doesn’t fail? That is, what if HAL is wrong?
Dave and Frank discuss this predicament while sitting inside one of the small pods, a place where HAL cannot hear them, used to transport the crew outside the main ship and back. They agree that if it turns out the original component functions properly and doesn’t fail after being reinstalled, that would mean HAL was wrong in HAL’s diagnosis of a potential problem with the part. And if HAL is wrong about that, what other errors might the computer system also be wrong about, a computer system that controls the vast majority of spaceship and mission functions? Dave and Frank decide that if the part is reinstalled and doesn’t fail, they will have no other option but to disconnect HAL.
Unfortunately for the crew, although HAL cannot hear their discussion taking place inside the pod, HAL can see their lips move, visible through a pod window to one of HAL’s many lenses, and HAL can read lips. Since the reason for and full significance of the mission is known to HAL but not the crew, HAL determines the possibility of being disconnected by the crew is something that cannot happen. The mission, HAL knows, is too important.
“I’m sorry Dave, I’m afraid I can’t do that”
When Frank exits a pod to reinstall the original component, HAL uses the pod’s mechanical arms to cut Frank’s airline, sending him hurtling away from the ship. Using another pod to retrieve Frank’s by-then lifeless body, Dave returns to the main ship and instructs HAL to “open the pod bay doors.” After repeating the instruction a few times without any response, HAL finally says, “I’m sorry Dave, I’m afraid I can’t do that.” HAL follows this by telling Dave that the crew’s plan was known to HAL through HAL’s lipreading, and that HAL knows the mission is too significant for the crew to disconnect HAL.
I am reminded here of John Cotton (1585-1652), a British clergyman who came to the Massachusetts Bay Colony and was a minister in Boston. In “Limitation of Government” Cotton gave this advice:
Let all the world learn to give mortal men no greater power than they are content they shall use — for use it they will. … Whatever power he hath received, he hath a corrupt nature that will improve it in one thing or other; if he have liberty, he will think why may he not use it? … It is necessary, therefore, that all power that is on earth be limited, church-power or other. (The American Puritans: Their Prose and Poetry, Perry Miller, ed. Anchor Books, 1956, 85-86.)
In writing this, Cotton reflected a commonly held Puritan understanding of humans: created good but fallen. Even those among the elect are nevertheless subject to the effects of sin, which often play out in selfish and self-centered ways. And although HAL is not human, HAL was programmed by “mortal men.” Similarly, various aspects of generative AI are based on and draw from sinful and fallible humans: from the computer code of programs, to the vast amounts of data used to build large language models, to the prompts humans give to generative AI systems.
The fallibility of technology was brought home to me this summer, while conducting research in England. Unfortunately, when I was there in June 2024, the British Library was still suffering the effects of a cyber-attack that occurred in October 2023. Although the library’s main facility and reading rooms are in London, significant parts of its holdings are in other locations. As with many other infrequently accessed materials, most of the items I planned to use are stored in an off-site facility with automated vaults that use robotic technology to retrieve and reshelve materials.
Because of the cyber-attack eight months earlier, materials stored in that off-site location were unavailable. Library employees could not provide library patrons with physical items held in the off-site facility that relied on robotic technology because humans didn’t know the locations where materials were shelved — only the computers and robots did. Catalog searches for items housed in off-site facilities at one of the world’s leading libraries generated messages such as the one below.
Uses of then cutting-edge IBM technology by Nazi Germany, the ongoing harmful effects of the 2023 cyber-attack on the British Library, and HAL’s decision to kill the spaceship’s crew in order to save the mission can serve as cautionary tales in our rush to embrace generative AI — indeed any technology. As with the humans who conceive of, design, build, test, program, market, and maintain them, technologies have the potential for both great good and great evil.
