Character Input
The entire Character Input reconstruction, displayed at ISEA '97. To the right is the subject scanning chair. Hanging from the track next to it is the X-Y visual input device, which first scans the subject and then moves over the visual database, finding correlations in facial structure between the subject and the faces in the database. When it find a strong area of correlation, it then refers to the character attribute data, stored in the research station, at the extreme left. Finally, predicted personality traits of the subject are displayed on the small round cathode tube of the research station.
Last known photograph of Dr. Prokopoff, with subject in laboratory, circa 1964.
"Even though machine recognition of faces has not been attained, the investigation of how it might be done has led to a number of related issues that in themselves are worthwhile (and tractable) areas of research..."
"The problems of the automatic analysis of faces have received little attention. The work begun by W.W. Bledsoe and his colleagues is one of the few attempts I know of to automate the recognition of faces; the method uses a hybrid man-machine system in which a computer sorts and classifies faces on the basis of fiducial marks entered manually on photographs. The technique is called the Bertillon method, after Alphonse Bertillon, a French criminologist, and is better known for its application to fingerprint classification. A similar method has been developed by Makoto Nagao and his colleagues in Japan in an attempt to devise an automated system that would produce simple numerical descriptions of faces..."
"Our studies have touched a host of questions about human perception, automatic pattern recognition and procedures for information retrieval. Although the ultimate question of how a face is recognized remains unanswered, a few promising lines of inquiry have emerged."
"It has once again been clearly shown that the human viewer is a fantastically competent information processor."
The Prokopoff perceptron, circa 1968, was discovered by the artist during a visit to the museum of medicine in Tallinn, Estonia, and was reanimated as "Character Input," the second part of a five year project to reconstruct three obscure technologies from the history of computing. "Character Input" was unveiled at 1997's International Symposium on Electronic Art.
1997
“Faces, like fingerprints and snowflakes, come in virtually infinite variety. There is little chance of encountering two so similar they cannot be distinguished, even on casual inspection...”
In 1968, a Soviet scientist from Lomonosov University in Moscow, while working on simple perceptron systems, came across a profoundly significant discovery. His research involved the recognition and discrimination of human faces. Funded mostly by a branch of the MVD for organizing criminal databases, Dr. Ilya Prokopoff was hoping to categorize faces through the use of perceptrons, machines built on models of the architecture of the brain, extremely facile at pattern recognition.
This inquiry was inspired by yet another question: How can a computer be made to recognize a face? This question remains unanswered, because pattern recognition by computer is still too crude to achieve automatic identification of objects as complex as faces...
Prokopoff's third research trial was a hybrid between work done by Frank Rosenblatt, a prominent American perceptron researcher, and various mathematical models of neurons authored at the Pavlov Institute in the early 1960's. The combination proved to be very powerful. Rosenblatt's prior work had created a perceptron that could distinguish gender from a photograph of a subject's face.
Prokopoff quickly reproduced these results with a similar analog computer, but at the same time noticed that his perceptron output a slightly different, and at first seemingly noisy, signal. Along with the major harmonic output signalling the subject's gender, other, sub-harmonics were being generated consistently for each subject. They were consistent, however, thus not noise.
Prokopoff's subject photos had been taken from the MVD's police files, so Prokopoff had access to the other, extremely detailed data they contained. The most minute and esoteric data gathered by police, the subject's neighbors, colleagues, boss, even their ex lovers — from purchases to poetry — all was cross referenced against the photos.
After further analysis, Prokopoff realized that one of the major harmonics, regardless of gender, corresponded with whether a subject was a member of the Communist Party. Spurred to further research, he invented a statistical method similar to Eigenvectors — some 20 years before this was discovered in the West — and built a large machine for correlating faces with MVD records.
Because digital storage was too expensive in the 1960's, Prokopoff's database was divided into two parts: 1) citizen records in text form (stored digitally), and 2) facial photos (stored as hardcopy and accessed through an 8x1 meter robotic x-y scanning mechanism)
Soon the perceptron was outputting data about the photographic subject of an extremely detailed nature: "Regrets the loss of Vavilov" or "Occasionally drives beyond the speed limit." Like the police files themselves, the qualities the machine could recognize about the subject's character were often an odd mixture of bureaucratic, criminal, extremely personal, and vaguely accusatory. Prokopoff's results caught the attention of another Soviet organization, which was primarily responsible for finding and exploiting potentially useful military technologies. It was felt that the kinds of details revealed by Prokopff's perceptron could potentially be used to investigate suspects, compromise enemies, and also help with the management of huge personal files in various government agencies.
Prokopoff and his lab were moved to Tallinn, Estonia, to insure his project's secrecy. The perceptron was revised, and improved steadily. Oddly, research into perceptrons, and related neural networks, had almost completely halted in the West, due to the availability of cheap digital computers. Other strategies of investigation were possible with computers, and the symbolic logic school, called Artificial Intelligence, managed to divert nearly all defense research dollars away from perceptrons and towards their agenda.
At first, the project was received glowingly by its new sponsor. Many and different character attributes were successfully detected by the perceptron. As researchers in the United States learned much later, however, perceptrons and neural networks may be extremely powerful, but they are also extremely hard to predict or explain. The more the perceptron was trained, the more apparently useless the information. Observations like "No taste for anchovies," or "Has a little bit of a belly" left Prokopoff confused and his bosses impatient. Prokopoff's budget was reduced yearly between 1972 and 1975. In his last research report, recently declassified, he promises that he has found a way to constrain the perceptron toward pertinent issues. No further records containing information on his machine, or his personal life or whereabouts, are known at this time.