School of Film and Video
California Institute of the Arts
The Contribution of the Artist to Scientific Visualization
1987 delivered as a talk at the Jet Propulsion Laboratory, Pasadena, California
1989 published in condensed form by the San Diego Supercomputer Center, California
It has been stated that approximately 50 - 75 % of our brains are comprised
of neurons devoted to processing visual stimuli. 99% of the American population
watches television, with a large proportion relying on it as their primary
source of news. The saying "seeing is believing" attests to the
trust we place in our visual sensory apparatus. We speak of "eye-witnesses"
in legal proceedings, and in the accuracy of visual memory. This concept
of visual, or "photographic memory" is powerful and useful. Having
a very high bandwidth (as opposed to our ears), the eyes are sending vast
amounts of data to the brain every second for processing. The method by
which we process this information is still largely a mystery, but it is
clear that "visual thinking" is based upon an inter-relationship
between visual perception and memory. E.W. Sinnot, the American biologist
and philosopher, has stated that, "Stored images in the mind are the
basis for new creative ideas."
With this in mind, we can begin to look at the history of visual art as
a recorded history of ideas and knowledge. We know that cave paintings showed
us not only elegant and aesthetic (even by today's standards) representations
of ancient concerns, but also that primitive man, to the best of his abilities,
possessed a functional understanding of the nature of the materials used
to make the paintings. As Paul Klee has stated, "The artist knows the
nature of all his materials, for part of his work has consisted in examining
them precisely, one after the other, in order to master them with intelligence."
So, from the cave painter up through current time, artists have to some
degree also been chemists or technologists. In fact, until last few hundred
years, art and science were closely allied. The relationship between creative
thinking and scientific advancement has always been close. Specifically,
the role of intuition in the creation of a theory or hypothesis, the application
of a knowledge of geometry, in addition to the use of a rigorous method
for recording observed experience, has formed the basis for many scientific
advances over the years. Albert Einstein, in "Essays in Science,"
stated, "Man tries to make for himself in the fashion that suits him
best, a simplified and intelligible picture of the world; he then tries
to some extent to substitute this cosmos of his for the world of experience,
and thus to overcome it. This is what the painter, the poet, the speculative
philosopher and the natural scientist do, each in his own fashion."
("Principles of Research" from Essays in Science, 1934)
In light of Einstein's insight, and in the context of Scientific Visualization,
I have asked myself three questions:
- What is an artist?
- How are artists like or not like scientists?
- How can artists and scientists work together?
1. What is an artist?
An artist is many things, but generally it could be said that artists are
individuals who are working towards a deeper understanding of the world
in and around them. Artists are acutely sensitive to the environment, organizers
of large amounts of data into an aesthetic expression of his or her world.
Artists are explainers and representers, pattern matchers, people who find
unusual relationships between events and images. Artists are creative intedisciplinarians.
They rework and integrate information, subject data to functions and tests,
render or give the data shape, express essence and make clear. They help
us to see in new ways and understand that which is difficult to see, making
the invisible visible.
Artists are idea people, creators and conceptualizers, predictors, people
with ideas connected to the future. They are people who create something
completely original and new, something beyond the known boundaries of the
information base. By using or inventing new tools, they show new uses and
applications that synergize and synthesize fields. Artists push the limits
of technologies, bringing them to previously unattained goals. They see
the next steps, make leaps, put together elements in new and powerful ways.
Artists are bridges to the rest of the world. They understand perception
and know how to communicate information. They know the technologies and
language of media. Their work is linked to the general population as their
inventions and ideas translate directly, often seen as improvements in understanding
and the quality of life, improvements upon less efficient and more laborious
methods of working, becoming mainstream as society discovers their work
and catches up.
2. How are artists like or not like scientists?
Mutual Respect and Common Background
A good number of scientists have backgrounds in visual art, having formally
studied both art history and creative art production. Many have a deep respect
for art, and rather than studying it formally, attend museums and galleries
and maintain contacts with colleagues in the art world. Scientists know
from these associations that the work of an artist is often as difficult
and complicated as work in any field. They appreciate and respect the vision
and the hard work. Conversely, many artists, especially in the more technical
fields of photography, film, video and now computer art, have good science
backgrounds, and have a reverence for the achievements of science. There
is a parallel respect and a fair amount of common knowledge and concern.
This results from living in the contemporary world with widespread exposure
to current ideas and technologies through communication media and personal
computers, and from a good education that includes training in both the
humanities and the sciences.
Artists as well as scientists work with abstract symbols, representations
for various realities and working tools. Even the language used by the two
groups is similar. Scientists working with mathematics frequently describe
a particularly good explanation or solution as "elegant." Usually,
"elegant" means simple and original. This has also been said of
good art and design. Scientists apply their taste in visual aesthetics to
their visual displays of data as do artists, as in the case of a scientist
who carefully selects colors or arranges forms used in his/her design of
a chip or rendered image. These are aesthetic decisions, signs that the
scientist is to some degree thinking as does a visual artist. The intellectual
bridge of abstraction and aesthetic consideration is fundamental to both
Art and the Scientific Method
Just as scientists consider aesthetics, artists often use the scientific
method in their work, consciously and unconsciously. For example, a ceramicist
developing a new glaze may begin with a hypothesis he/she wishes to test,
and with the results of the experiments, will continue to test and modify
the original hypothesis until the results confirm the hypothesis. This process
may actually lead the artist to the exploration of fundamental technological
problems, even though the original motivation for the work was otherwise.
Eventually, the tests yield data which we see in the form of exotic creations,
some successful and some unsuccessful. The craft or technique developed
may be used to bring forth statements by the artist, or as in experimental
or process art, the tests and/or their results may be viewed as aesthetic
statements in themselves. Whatever the purpose, it is through the artist's
methodology of experimentation and creation that new visions, new techniques,
and new ideas enter the collective knowledge base of the culture.
Once an experiment has proven repeatable, the results of the experiments
end up influencing the culture in that they become things to be imitated.
Techniques and ideas are appropriated by the mainstream and become known
and constant ways of creating images and representing information. This
was, and still is, the case in computer graphics. Art Durinski, computer
artist/animator formerly with Information International Inc. (III), Omnibus
Computer Graphics, and Toyo-Links of Japan, told me that when he was at
III in the late seventies, he "made lots of flying logos over grids
because they were easy to make." That look first set forth by Durinski
and his colleagues has been imitated not so much because it was easy, but
because it was synonymous with computer graphics. The same is true of today's
metallic and ray traced objects. I have recently seen a demo reel of a production
company in Southern California, wherein the metallic spinning logo was actually
made by hand using an airbrush. This computer look is the current icon of
sophistication desired by our visually demanding, high-tech society.
Visual artists usually think visually, working with drawings and creating
spacial models in their heads which lead directly to finished works. Art
schools investigate the models from which artists work in depth, studying
their goals, work processes, and end results in formal critiques. Like the
3-D artist, the sculptor, many of the best scientists with whom I have spoken
tell me that they frequently work from visual geometric models in their
minds, and identify research problems on the basis of these visual models
or paradigms. From the paradigm, the analysis and experiment is derived.
This ability to pre-visualize a potential solution to a problem and to build
3 or 4-D (3-D plus time) conceptual models is basic to their scientific
process. But, unlike the art schools, science schools do not generally teach
or analyze this spatial-conceptual skill. Dr. Carver Mead, of the California
Institute of Technology Computational Neurosystems and Computer Science
Departments, often works with visual models and informed me that the problem
in the sciences is that those who do visualize take it for granted, and
those who do not know about it, do not know about it. Dr. Mead has suggested
the importance of teaching conceptual-visual skills in secondary and university
science education, including drawing, filmmaking, and related perception
and communication courses. He feels that it is important to articulate and
formalize the role that conceptual visualization (creation of visual paradigms)
as well as descriptive visualization (display of data) skills play in science
education and research. He has stated that the lack of training in communication
skills has led to a slowing of knowledge transfer between various disciplines,
an important element in scientific research and in the development of new
branches of science. This is especially relevant to the emerging field of
Scientific Visualization, wherein visual thinking applies directly to many
levels of the process. Being a good communicator, he says, is an extremely
valuable asset for a scientist.
Clarity and the Imagination
How do artists and scientists create models and paradigms? How do they come
up with fresh and new ideas? What is creative thinking? Both groups are
working with their imaginations and their intuition, the powerful organizing
force of the brain which is always active, reaching beyond "normal"
thinking processes. Creative people often use "educated guessing"
or inherent trust in their "intuition," the natural organizing
force which seems indirect, but often is more direct than realized. Intuition
builds upon stored knowledge and to me means "clarity of perception."
It seems that the brain naturally forges neural pathways linking information
in new ways, reorganizing the data, and then reaching beyond it, creating
something new and logical, perhaps just not so obvious.This form of logic
may not necessarily be verbal or mathematical, it may instead utilize visual
thinking techniques, such as geometric pattern matching. The path to the
idea may be based on the acceptance of string of unproven assumptions, so
that part of the work of the scientist or artist involves exploration of
the assumptions. In any case, the combined skills of logical (visual, mathematical,
or verbal) and free(the ability to work with unproven assumptions) thinking
can lead to interesting questions and potential breakthroughs.
This idea formation or "insight of clarity" sometimes occurs in
one's sleep, and at other times when playing or daydreaming. This process
is useful for artists in identifying a concept or image, and helps researchers
of all kinds in identifying problems and their solutions. The subconscious
mind keeps working as our conscious mind goes onto something else, then
suddenly, the Eureka! experience. Both artists and scientists use their
imaginations and logic, both consciously and unconsciously, within a given
framework on a particular set of events or problems. Creative thinking is
inherent to all fields of endeavor, since it is a human characteristic.
The brain appears to synthesize the accumulated information and leap to
new ideas, new conceptions, new models for understanding the world.
Art is to Design as Science is to Engineering
In visual art, the database of information with which a creative mind works
consists to a large degree of visual images, drawn from a myriad of sources.
Sorting and making sense of this data by the artist is a highly specialized
and refined task, potentially occurring partly in the retina or optic nerve,
and partly in the brain. (In a way, an artist is like a very good computer
While both artists and designers have large stored databases of images,
it is important to consider the difference between the two. Designers usually
work as rearrangers of existing data, applying it to an existing model.
The fine artist, however, works with and beyond repetition of existing patterns,
if necessary synthesizing wholly new modes of representing the data. The
fine artist works from his/her own thinking process, and is capable of creating
original ideas and models, forging new relationships beyond the known, stored
patterns. A good designer can also work with the spirit of risk-taking,
exploring unusual options. Normally, however, this is not the case, as most
designers are strongly discouraged from exploring unconventional approaches
to design while students. This attitude typically goes with them into the
professional world. Ironically, the winners of design competitions are those
who break out of the rigid visual molds, those working more in the realm
of fine artist. Hence, it is truly the spirit of the fine artist that makes
the greatest contributions to our visual environment. An individual considering
new information, new problems with new data, synthesizing it and reaching
towards the new and the fresh, towards clarity and refinement, is either
a fine artist or designer working as a fine artist. This person brings a
fresh perspective to the field and may be able to create new conceptual
or visual models which better fit the new problems.
Art is to design as science is to engineering. An artist, like a scientist,
is involved with the imagination in the conceptualization stage of a problem,
whereas a designer and an engineer is usually executing the application
of art and science, usually in repetition of what has been done before.
This is not to belittle the work of the engineer and designer, who is grounded
in the pragmatic application of known principles. It is just that the roles
are very different. If one is discussing the research environment, the focus
is usually upon the future and on gaining a deeper understanding of the
world, and hence, a fine artist is more sympathetic to the purpose of the
scientist than is a designer.
3. How can artists and scientists work together?
Unless the scientist is also an artist, it seems that the very best situation
in a scientific environment is for scientists to work with a fine artist
in a team. This artist should be a person with enough science background
to understand the basis of the research, and potentially a person whose
own work is related to the research field. This person can help to identify
significant problem areas and keep a stimulating dialog with the scientists.
This person should also be sensitive to the purpose of the research, as
well as applications to society, a thinker with a conscience. This is very
important, for such a person could give very valuable feedback about long
term implications, which could very well impact the next level of research
undertaken. This artist should also be able to work as a designer, applying
their sense of the pragmatic to original, fresh representations, utilizing
their well developed visual perception and communication skills in the final
display of the data. As in any field, care should be taken in interviewing
a new member of a research team, to make sure that the personalities are
a good match.
I have broken down the research process into 5 parts, and have indicated
with an * where I feel artists "plug in" well:
Artists "plug" directly into 3 of the items listed above, including
(1) conceptualization, (3) data gathering, modeling and (5) dissemination,
or designing the display of the information for communication. While typically,
artists are thought to be useful in the last stage only, and though I feel
that this application of the artist to Visualization is important, it is
a limited use of the artist, missing out on the full benefit of interchange
between disciplines. There exist excellent models from industry and education
to support the integration of artists into stages 1. and 3., conceptualization
and testing, as well as dissemination. I would even argue that in Scientific
Visualization, the most important may be the first stage.
- * Identification of research problems, conceptualization of visual models,
especially if research is related to vision and visual perception, image
processing, image making and control of space. An artist with some mathematics
and science background, whose own work relates to the research, may have
good insights and may be very helpful and stimulating.
- Description of the experiment
- * Running of the experiment, gathering data, testing, and modeling for
study (very important). In the case of tool development and evaluation,
artists are great testers, they push a medium to its limits, working very
hard and long hours. It has been said that "If there is a bug, they
will find it." Artists spend a great deal of time making and refining
the data, where scientists spend more time making the tools.
- Studying of results and drawing conclusions; alteration of experiment,
then go on to next problem (go to 1.)
- * Dissemination, giving form to the display of the results, for communication
purposes. Artists plug in here very well. This is because of their fluency
with visual communication skills and media. For example, an animator knows
how to pace changes in time, what movements and positions in space "read"
well. A painter knows how brightness and color saturation affect apparent
spacial location, and how the eye and brain responds to juxtaposition of
form and color. An architect or sculptor knows how spatial constructs will
look from different vantage points, even before the representation is made,
speeding up the visual selection process as well as improving understanding
of a final design, and providing more aesthetic results.
Sample Models from Industry and Education
There exist many excellent examples of research environments which make
use of artists, exploring their ideas and integrating them into the process.
The examples I am listing below in this and in the section called Historical
Models in Related Media, are but a fraction of those in existence, but will
serve as models to make my point. Often, it seems as though the use of artists
is a kind of trade secret! Apple Computer is known to make use of artists,
especially conceptual artists, in their user interface design and research
groups. The Apple MacIntosh computer is very popular, and it is likely that
the artists had a great deal to do with the success of the Mac. The use
of black type on a white field, which we take for granted, is actually unusual
for a graphics system. Most computer displays are white on black, which
is more difficult to look at. Using the printed page is an excellent model,
probably the suggestion of an artist.
Pacific Data Images (PDI), a major computer animation company which is known
for its continuingly fresh ideas and innovative look, has hired artists
and animators for various production tasks, but expects that they spend
approximately 25% of their time on-the-job on their own work. It appears
that PDI understands that the work the artists do on their own, feeds back
to the work they do for the company.
Examples from Caltech
Computer scientist, Brian Von Herzen of California Institute of Technology's
Computer Science Graphics Group, has stated that access to art and artists
has helped him in his own computer graphic work. He has become more aware
of aesthetic considerations such as color, from looking at art and from
studying his own perceptions. He incorporates his, by now, very developed
sensitivity to color into his work. In addition, he tells me that while
he can compute a scene which, according to the algorithm, should model nature
correctly, he calibrates his creation with an artist's eyes. The artist
can tell very quickly if the scene in fact is "correct" and looks
like reality, since the artist has spent so much time studying the visual
Dr. Dan Whelan, who received his Ph.D. in Computer Science from Caltech
in 1985, wrote a rendering program which I used for computing a sequence
of images during the same year. There was a serious problem which I discussed
with him: there were no shadows in the scene, and it just didn't look right.
If the program couldn't compute the shadows, then I would add them with
the paint program, which I did. Over the following months, Dr. Whelan developed
a shadow algorithm, which ended up being a significant portion of his Ph.D.
thesis. Perhaps he would have written it anyway, but my feedback surely
helped to accelerate this development.
Misha Mahowald, graduate student in Computational Neurosystems at Caltech,
has designed a synthetic retina which makes use of the hexagon as a design
element. She came up with the idea for that structure from looking at the
work of M.C. Escher. Ms. Mahowald has stated that "artists are windows
to understanding perception." She and I keep an active dialog about
the relationship between art and science, especially interactive computer
graphics and music as it relates to research in understanding and computer
modeling of vision and hearing.
Professor Alan Barr, also of the Caltech Computer Science Graphics Group,
works with the simulation of natural phenomena. He once worked as a graphic
designer, and this work, he says, has helped him a great deal in his current
work. His concern for aesthetics is evidenced by his careful and elegant
selections of color, composition, and form throughout his visual work. His
needs for more control over the design and structure of time in animated
sequences, has led him to some of his current research objectives. His work
in dynamic constraints developed largely from this need for aesthetic control.
In addition, Dr. Barr sees the benefits of studying the working process
of such artists as character animators, in order to understand how artists
control form for human expression. Dr. Barr has been a major force in teaching
his students concepts of visual aesthetics as well serving in his formal
role as a Professor of Computer Science.
The University of Illinois at Urbana-Champaign
The University of Illinois at Urbana-Champaign Center for Supercomputing,
is currently setting an excellent example through joint efforts between
artists and scientists. Donna Cox, an artist working at the Center, has
spoken about this fruitful interaction, what she calls Renaissance Teams.
These teams make use of specialists in art and science, where neither person
has to "backtrack" and learn the other field in order to work.
This is an excellent prototype for the future of Scientific Visualization,
an approach which serves both the mature scientist and artist. The artist
focuses on difficult visual decisions, while the scientist focuses on the
difficult scientific problems. It helps, of course, that Ms. Cox has a lengthy
experience with science, so that her aesthetic decisions are made with an
intelligent approach to the research. The interaction of specialists also
serves as an education of one about the other, thus enriching the two on
My Work with Dr. Weinberg at USC
Among my efforts to stimulate communication and collaboration between artists
and scientists, I have worked with Dr. Richard Weinberg of the University
of Southern California (USC) on the creation of an animated sequence depicting
two models of the Solar System, the Platonic and Ptolemaic, for the Omnimax
film, "The Seasons," directed by Ben Shedd. I designed the databases
and animation, and Dr.Weinberg converted the data into a format for display
in high resolution for filming in 70mm film format. Through my own rediscovery
of the models of solar system and their relationships, I arrived at a 12
day year, each day spanning 30 degrees of a 360 degree rotation about the
sun. In working with Dr. Weinberg, we separated the tasks of art/design
and computation, so that we each focused on our strengths. Support for this
film came from the Science Museum of Minnesota, among others.
Historical Models in related Media:
John Whitney, Sr., considered the father of computer graphics and animation,
was an experimental filmmaker who came to the use of computers from his
need to control graphic artwork below a film camera. He used a World War
II analog computer, an M-5 Antiaircraft Gun Director, augmented later by
an M-7, and hybridized it to make the first motion-control films. As Gene
Youngblood writes in Expanded Cinema (1971 - page 208), "The analogue
computer work gained Whitney a worldwide reputation, and in the spring of
1966 International Business Machines became the first major corporation
to take an "artist in residence" to explore the aesthetic potentials
of computer graphics. IBM awarded Whitney a continuing grant that has resulted
in several significant developments in the area of cybernetic art."
John Whitney, Sr.
Computer Art at Bell Labs
Kenneth Knowlton, computer scientist working in the sixties and seventies
at Bell Labs in Murray Hill, New Jersey, collaborated with a number of artists,
including Stan Vanderbeek and Lillian Schwartz. Vanderbeek worked with Knowlton
on a number of films, including Poem Fields, "whose micro-patterns
seem to permutate in a constant process of metamorphosis which could very
likely include a hundred thousand miniscule changes every second."
Gene Youngblood in Expanded Cinema (page 248) interviewed Vanderbeek about
his work with Knowlton at Bell Labs:
"The present state of design of graphics display systems," Vanderbeek
explains, "is to integrate small points of light turned on or off at
high speeds. A picture is 'resolved' from the mosaic points of light."
The artist seems to feel that this process bears some physiognomic similarities
to human perception. "The eye," he notes, "is a mosaic of
rods and cones." '
Other scientists, such as B. Julescz and C. Bosche, made very interesting
and aesthetic works as a result of experiments in the psychology of perception
and human vision. The works of these and other collaborative efforts over
the past two decades has led directly to the development of our current
computer imaging and display technologies and applications.
Bell Labs Sound and Speech Synthesis Research
While Bell Labs is famous for its computer graphics artists, it is also
famous for its Composer-in-Residence program, in existence for over two
decades. The Composer-in-Residence Program, as headed by Dr. Max Mathews,
was set up in the early 60's in an effort to make computers available for
cultural use, as well as an aid in the research effort. As Dr. Mathews,
in Computer Composers -- Comments and Case Histories stated,
"Encouragement was offered from a feeling of responsibility for bringing
new technology to the attention of those in the cultural community who can
make use of the new tools, and partly because increased understanding of
sounds may improve speech communication. Composers are very creative and
have very good ears for making and analyzing sounds. Much more could be
said about the impact of art on technology..."
In my interview with Dr. Mathews, now at Stanford University, I asked, "Did
the composers help to accelerate the development of the sound and speech
synthesis technology? "Yes," said Dr. Mathews. "A good example
is Jean-Claude Risset," who was both a physicist and composer. Risset
made the first catalog of sounds with a computer, including brass sounds
such as a trumpet, flutes, woodwinds, pianos, bells, and "a host of
percussion instruments plus numerous non-instrumental sounds. In addition,"
writes Mathews, "he has invented another musical dimension--spectral
pitch control--which an be used to produce tones which seem to be simultaneously
ascending and descending in pitch. He has become so expert in tone quality
that he can achieve a desired timbre with almost no experimentation. He
has made his expertise available to the world by making a catalogue of sounds.
I believe that this is the most important musical document of the century."
This concept of a sound catalog, and the models set forth by Dr. Risset,
is part of virtually every synthesizer on the market today.
I asked Dr. Mathews if the composers at Bell Labs had to be their own technicians
and scientists. He said that while many were able to develop their own technological
devices and tools, this was not required. The Laboratory was set up to provide
a supportive scientific environment. Since the composers were formally invited
into the Lab, computer time was set aside for their work, though sometimes
the hours were odd and in the middle of the night. They were invited to
make experiments in music as well as to compose pieces.
Recognition, Documentation, and Formal Integration
As discussed, artists and scientists have worked together for many, many
years, but now it is beginning to be a more formal collaboration. The examples
of current team-work at such places as the University of Illinois Center
for Supercomputing, imply a movement,a profound change that is now being
noticed. These formal collaborations are, however, still more the exception
than the rule. Even though artists have been and continue to be of great
influence, their positions in research environments are relatively informal.
This results from a general lack of articulation of the artists' contributions,
and lack of justification for support of the artist. Therefore, the influence
and contribution of the artist has been virtually undocumented. Leonardo,an
excellent publication on the subject of art and science, begins to address
this need, but it is not nearly enough. And while the work of Ms. Cox at
the University of Illinois is an excellent model and starting to be documented,
much remains elsewhere to be recognized and formalized. There is a great
need for an honest and expanded study of the contributions artists make
to science throughout the United States. An investigation should be undertaken,
a report written and published, and formal changes made in the granting
and support procedures in agencies, so that serious consideration is given
to the artist, reflecting this awareness. Arguments and justifications for
the formal integration of artists into scientific research should be made
at the highest levels, so that this becomes a standard, not an exotic and
unusual situation. Changes should be made so that artists are not expected
to work for free.The institutions that include artists will probably be
seen as visionary, forward thinking environments, and may in the end have
better chances for support, because their research may be better. This support
and reputation would attract the best artists in the field, bringing strong
ideas and the best energies to the research environment. The formalization
of the artist in Scientific Visualization in particular, is especially important
since the work draws so much from the visual artist who is already very
Not only should there be justification for Artist-Scientist Teams, but there
should be formal Artist-in-Residence Programs, where an artist is working
on their own work, as did the composers at Bell Labs. These people should
be paid like any other researcher, and should have access to funding from
the same sources as the scientists. Special grants should be made available
to artists from government agencies, as well as from organizations that
benefit from the work, such as ACM SIGGRAPH and large computer companies.
A portion of the funds for science should be designated for the artist's
research. Artists should be considered equal members of the research environment,
and their work should be recognized as such. Their work should be seen in
the context of the working environment, the influence it has on the sciences,
and should be published in the scientific journals when the content merits
it. The artists selected for residencies should be carefully screened, so
that their capabilities, interests and goals relate to the overall goals
of the research environment. It is very important, though, that the artist
be free to pursue their own research ideas, as this will bring an increased
possibility for advancement of the field.
Scientists can benefit from immersion in the art environment. They can learn
a great deal about their own perception and develop excellent communication
skills. They can work from these skills towards solving problems of not
only aesthetics and refined visual composition, but also user interface
design, and new applications as set forth by artists working in various
media. For example, multi-screen interactive digital-video performance,
computer controlled sculpture, or the integration of sound and image may
stimulate new ideas and directions for research. They may be learning more
about the psychology of perception and the working and thinking processes
of artists, so as to make better tools for artists, and other users of computers.
This could lead to many, many new applications and insights. It can also
be of great benefit to the art environment, helping both the artist and
scientist to develop new tools and reach new goals. Because this exploration
of tools and methods used in, and the methods for computer art and animation,
the work really is research. But because it takes place in the art environment,
it is not formally designated as such. Therefore, it is my feeling that
funds for the work of scientists and artists in art environments should
be available from organizations that support research elsewhere.
Just as an individual stimulates the research environment, so does the work
of a collective group of artists. The value of art shows in general, and
computer art shows in particular cannot be measured in dollars spent by
attendees. There are other rewards in life besides money! The artworks can
lead us to an improved quality of life, suggesting new ideas or improvements
in our working methods and living environments. The true value of art shows
like those at ACM SIGGRAPH and NCGA, lies in these intellectual rewards,
the seeds they plant in the minds of the viewing and participating audience.
It may be years later that the seeds take root and lead to major research
efforts or applications. But without the art shows, the seeds cannot be
planted. It is of critical importance that art shows such as these be supported,
for their impact is invaluable, measurable only over a long period of time.
Art shows should not have to be mounted at the expense of the artists (a
common reality). While some of these artists are also scientists with Ph.D.'s
who manage to produce their artwork supported by other activities, most
artists struggle very hard, without an equivalent amount of support. Because
of the nature of their work and their unusual backgrounds, they may not
fit into a normally funded slot. Therefore, I feel that it is important
to recognize when one is "taking" from the work and ideas of creative
artists, and do one's best to "give back." Through active support
for artists and art shows, we can continue this important cross-fertilization
and help the entire field to grow.
The Role of Education
To help the next generation of researchers, we need educational role models,
institutions leading the way, not afraid to step forward and set an example.
We need educational programs which actively bring together artists and scientists
at all levels. We need to rethink and restructure our curricula, requiring
math and science for all art students, and humanities and art for all science
students. We should create new courses which specifically discuss the relationship
between art and technology, and include, for example, a lecture series which
brings in role models for the students to see and hear. We should embrace
the concept of hybrid renaissance people, and for older people who are already
specialized, create courses which foster mutual understanding and collaboration.
We should help to improve visual literacy on the part of sciences, and technological
literacy on the part of the humanities. We can make use of existing programs,
expanding them where necessary. Art schools have already started this process,
now it is up to science institutions to level the balance.
Making use of Existing Art Programs
Art schools have the courses already developed for a good visual training
for the sciences. Existing art programs within universities are rich and
useful resources. Instead of re-inventing color theory, composition, and
media studies, the science schools should be working toward making use of
art departments, sending their students to classes in art history and art
production, and hiring artists from the art departments to help with teaching
as well as research projects which involve visualization. Why is it, though,
that so few science departments have made use of these resources? As in
research, the problem seems to be one of justifying the role of art in support
of science. Call it "Visualization Engineering" instead of "Art,"
said one scientist. Perhaps this is the solution, finding a new name for
the technologically literate artist. In any event, I hope that this problem
will be resolved as an articulated need for visual literacy on the part
of the science community becomes more and more of an issue. Art History
The change from "computer graphics" to "visualization,"
what I call the second stage of computer imaging, parallels the history
of visual art, wherein the desire to accurately represent they optical image
realistically gave way to surrealism, impressionism, and abstraction. This
developed as artists looked for a way of representing abstract visual concepts
and realities beyond the directly perceptible. As the level of sophistication
of computer visualization increases, and as researchers seek to make the
invisible visible, one can sense the direct application of the history of
art to Visualization. By studying both the history of art and contemporary
art, it is possible to gain insights into the aesthetic and conceptual manifestations
and uses of images in society. The accumulated knowledge of the fine arts
can be extremely useful to Scientific Visualization, a field which will
rely more and more on visual skills and ideas. A study of art history can
help to gain insights into visual form giving and unique ways of solving
problems which could enhance the scientific research environment in new
and unexpected ways.
Other Art Courses
Since so much of Scientific Visualization is based on visual communication,
a formal visual training beyond art history is necessary. If scientists
wish to take on the work of the artist themselves, they need better skills
for drawing, visual and temporal composition, experience with film and video
analysis and production, including animation, as well as 2 and 3 dimensional
From artists and art courses, scientists can learn about the psychology
of perception and its application to image making. For example, opposite
colors on the color wheel cause figure-ground disparities and seem to "jump."
In an enclosed space with little ambient light, darker colors imply a greater
distance from the viewer than do lighter colors (depth cueing). In a space
with a great deal of ambient light, such as outdoors in daylight, colors
fade into the grey of the atmosphere, that is, they lose saturation as a
function of distance. Artists know this from observation and analysis.
Scientists can learn about composition and visual balance from a graphic
design course. As an example, a symmetrical design appears static, whereas
a scene set assymetrically and at an angle, appears dynamic. Dynamic images
are usually more interesting to look at. In addition, studying various color
models, such as the color sphere of Philip Otto Runge (1810), the Albert
H. Munsell model (1898), and the CIELAB model, which is known to the science
community, can help in understanding color. A study of the work of such
masters as Albers, Itten, and Klee can supplement this study and be of immense
value in the application of color to form and composition. In film animation
one can learn from the work of Hans Richter, who developed a vocabulary
of temporal spatial articulation by eliminating form and color, and working
only with the movement of the screen itself before a film projector. Through
an architecture course, one can combine a formal training in plane and solid
geometry with the study of architects such as Frank Lloyd Wright, and begin
to understand our relationship to the space in which we live and work. In
general, art education can help scientists to understand their relationship
to the visual world and how to use it in their work.
Art Schools and Computers
Art schools not only teach traditional art history and production, but computer
graphics and animation as well. This should be seen as no surprise, as artists
have always worked with the tools of their age. Computers for graphics were
not readily available to artists before about 1980, and due to the steady
decrease in cost and increase in power, art schools are now providing courses
in computer art, programming and mathematics. Where in the past, artists
who wanted to learn about computers had to enroll in computer science programs
(Duane Palyka, of New York Institute of Technology is an excellent example),
this is less the case today. The new generation feels illiterate without
training in computers, and virtually all art schools worth their salt offer
computer graphics and programming courses. Perhaps not as demanding as full
fledged computer science programs, they are forging the way towards an integrated
artist-scientist training ground. Science institutions, while not yet embracing
the arts with the same conviction that the art schools embrace technology,
are beginning to express the need for visual literacy. At some point the
two fields will find that their curricula overlap, and that they are much
better off if they pool their students and resources.
California Institute of the Arts
More and more scientists, as well as people from other disciplines, are
applying to the Computer Animation Program at the California Institute of
the Arts (CalArts). We have students entering this program with backgrounds
that include biology, computer science, geology, oceanography, physics,
chemistry, as well as the more traditional art backgrounds of literature,
painting, and photography. All entering students must have a good portfolio,
an original vision and a well developed ability to visually express themselves.
The artists entering the program without much computer experience are strongly
encouraged to study electronics and programming, in order to gain a deeper
understanding of the tools they use, so that they may have a larger technical
base from which to create. Most of these people find an internal motivation
for this, and therefore are not required to take the courses. Curiously,
more students take programming and electronics courses when the choice is
theirs to make. In the case of a computer scientist who is already far along
the technical path, I ask, "Why does this person want to study at a
fine art school?" In the case of one computer scientist who recently
applied, he answered, " Now that I know the techniques of computer
graphics, I want to learn about the meaning of images, how to compose them
and work with motion and structure in time. I want to work towards a humanistic
application of my knowledge, and to develop my knowledge of and skills for
communication through images... I want to realize my inner vision."
What we are seeing is the growth of a diverse and dynamic computer art program,
one which is formed largely by the arts moving closer to the sciences as
the sciences are moving closer to the arts. It is a field in terrific flux
and growth, as each new student brings a wealth of educational and personal
experience to their work and to the new field. Being both highly technical
as well as highly malleable, the uses of computers in art and animation
at CalArts borders closely on research, since the possibilities for serious
creative work is vast. There is a great deal of freedom for development
of computer applications at CalArts, perhaps more so than in more rigid
environments. Though there exist very real financial and hence, technical
limitations, the intellectual limitations are few. Perhaps this freedom
is what attracts the unusual multi-disciplinary student to computer art
programs like that at CalArts.
CalArts and Caltech
In addition to my work at CalArts, where I am Director of the Computer Animation
Laboratory, I hold a position as Visiting Associate in Computer Science
at Caltech, where I pursue my personal work and try to improve my understanding
of computing through auditing of classes. I am also active on several levels
in trying to catalyse increased interaction between the fields of art and
science. I continue to invite researchers from throughout the field, including
Caltech and the Jet propulsion Laboratory (JPL), to speak in my classes
at CalArts. I try to keep an ongoing verbal and visual dialog between CalArts
and Caltech. A few years ago, I mounted a gallery exhibit and live performance
of computer art, animation and music called "ArtsTech '85," in
order to bring together artists and scientists from the CalArts, Caltech,
and Jet Propulsion Lab (JPL) in a celebration of technology and art. This
show received major attention in several international journals, including
Novum, a European graphic art publication. The evolving relationship between
CalArts and Caltech has led directly to the employment of a number of my
students at Caltech and JPL, where their duties have included computer aided
illustration and design for scientific texts, as well as the creation of
computer animation for the simulation of Galileo, the un-manned space probe
to Jupiter. In addition, I have discussed the formation of joint educational
programs for students at CalArts and Caltech. A sample curriculum for institutions
which house computer science and art departments is described in general,
A program which would prepare a student for work in the field of Scientific
Visualization could be twofold, a Master of Fine Arts Degree, as well as
Master of Science Degree in Visualization. Students would focus in either
the arts or the sciences, or an extended program could join the two, yielding
a double degree. Faculty and curricula could be created entirely anew, or
through joint educational efforts formed from existing programs. In the
latter case, an equal number of students from both disciplines would be
accepted into the program so that neither the the sciences nor the humanities
departments would bear the major burden of resource allocation, as facilities
would be shared. Syllabi would be geared towards students and their special
backgrounds; artists would learn applied math, as well as general computer
science courses, including programming, electronics, Very Large Scale Integration
(VLSI), and algorithms; and for the scientists, drawing, film, video, animation,
as well as art history and aesthetics would be offered. It is very important
to have the very best teachers for this purpose, as attitudes impacting
the future of the field will surely be formed through these courses. In
addition, new courses in Scientific Visualization would be created and taught
by experts in the field.
The following college level courses would be spread out over a 4 year period:
Art students would take 1 per semester: Science students would take 1 per semester:
* Algebra ** Art History, incl. architecture
* Trigonometry ** Drawing and Painting
* Plane and solid geometry Sculpture
* Differential and Integral Calculus Photography
Topology ** Film and Video analysis and production
Chemistry ** Graphic Design
* Physics Electronic and Computer Music
* Electronics Performance Art
** Programming Languages: ** Interdisciplinary Art
C ** Computer Art
Lisp ** History of Technology in Art
Prologue * Geometry
other new languages
** Algorithms for Graphics
** VLSI and Simulation
** Artificial Intelligence and Expert Systems
Engineering Courses as related to Specialty interest
* = required if not taken in high school ** = required courses
Special Courses bringing together artists and scientists could include:
- Principles of Scientific Visualization
- Directed Research Projects in Visualization, artists working with scientists
- Projects in Applications of Computer Animation to Visualization
- Developing an Expert System for Visualization
- User Interface Design
99% of the population is affected by visual language and communications
media, and while visually sophisticated, most people are, including our
scientists, relatively unconscious of this visual language being used. Scientific
visualization leads scientists to the need for an understanding of this
visual language, as images become the major mode of data representation
and communication. Since the eyes take in vast amounts of data in parallel,
images can be thought of as a high-bandwidth data representations. Kinetic,
three dimensional models can describe complex behaviors which may be impossible
to understand or communicate any other way. Since so much of our invisible
natural world is spatial, the computed models which give these phenomena
form can communicate better and more widely, what before only a few people
could model in their minds. Great care should be taken to the display of
this data, as visual decisions can dramatically affect its comprehension.
Visual representation of information is a skill which anyone can study and
learn, either through example, or through formal study of the visual arts.
Visual thinking relates to not just the output stage of science. For many,
many years, science has made use of visual models or paradigms. Science
education should reflect this fact, teaching geometry and art in an effort
to help the next generation of scientists. The integration and application
of art to science and vice-versa, should be taught at all levels of our
educational systems, so that as our technology for communication develops,
our users are not left behind. An organized effort to merge the fields would
enhance the efforts and accelerate the growth of Scientific Visualization.
To facilitate and formalize the particularly considerable contribution of
the arts to science, we should create funded Artist-in-Residence Programs,
funded by the government and private companies, where artists would work
on their own research projects, the product of which could integrate very
well with overall technological research and development. This is based
partly on companies such as Apple Computer, which employs artists in various
capacities, including in user interface design teams, and partly on the
model of the Bell Laboratories sound and speech synthesis research effort.
Bell Labs has had a formal Composer-in-Residence program for over 25 years,
which has led directly to the sound and music technology available on the
consumer market today. In the field of Scientific Visualization, the artist's
understanding of visual perception and communication, and their contributions
to the identification of research problems, should be formally recognized
and integrated into the scientific environment. In addition, I encourage
the formation of Artist-Scientist Research Teams, where artists would aid
in the visual display of data collected from scientific research. This is
already taking place at the Center for Supercomputing at the University
of Illinois at Urbana-Champaign. I also support the creation of Scientist-in-Residence
Programs, where scientists would pursue their work in the context of an
art school, in order to gain as much insight into visual thinking and aesthetics
as possible. We should support Computer Art Shows and Exhibitions which
bring together artists and scientists for a cross-fertilization of ideas.
Finally, I encourage mutual curriculum development between art and science
institutions which could help to bring together the two worlds in a common
endeavor. It would yield a new generation of visually and culturally literate
scientists and technologically literate artists, and potentially providing
us with a unified individual with dual capabilities, the truly contemporary
Renaissance Person, well prepared for the emerging field of Scientific Visualization.
First written and delivered as a talk with the same title in 1987 at JPL