© X. Zheng et al., Published by EDP Sciences, 2022

1 The collision between artificial intelligence technology and art

In 1956, John McCarthy first used the term “Artificial Intelligence” (AI) at the Dartmouth Conference, marking the birth of the discipline of artificial intelligence. “Intelligence” is derived from the Latin word, which literally implies: gathering, collecting, aggregating, and selecting. Human intelligence is commonly considered to be the intellectual capacity of human beings in activities of understanding and reshaping the world, i.e., the individual's ability to investigate, judge and act purposefully to resolve issues [1]. According to Marshall McLuhan [2], any medium is an extension and expansion of human sensation; thus, the appearance of AI has become a more comprehensive extension of human intelligence. Besides, AI has also turned into a new science for researching and developing theories, methods, techniques and application systems to mimic, extend and expand intelligence [3]. Even though AI does not have a long history as a discipline, it has been discussed and explored much earlier in literature, art, philosophy, and history [4].

Leonardo da Vinci's “Automaton” invented in the 16 th Century can be regarded as the earliest integration of robots and arts [5]. With the electronic and computer revolution, some researchers suggested that this numerical advanced revolution should be matched with art and design. John von Neumann, the “father of the modern computer,” believed that although there is a distinct difference in structure between the computer and the human brain, the “von Neumann machine” could mimic how the human brain processes information. He held that the two systems are essentially the same, so the redundancy of brain to keep memories stable and consistent can be completely simulated by software [6]. The first computer “art” work was Electronic Abstractions by Ben F. Laposky in 1952 [7], produced with an oscilloscope. Ivan Sutherland introduced Computer Graphics in 1961, which has been studied in depth by many experts and slowly led to computer painting, digital media art and new media art [8].

The involvement of AI technology in visual arts can be traced back to 1973, when Harold Cohen invented the AARON painting robot [9]. In the 1990s, Leonel Moura created a painting robot that also utilized robotic devices to mimic traditional painting methods. Moura's “Robotic Action Painter” (RAP, Swarm of Robots) in 2006, which was permanently installed by the American Museum of Natural History, worked by mocking the group behavior of ants [10]. In 2014, Google created an AI program named DeepDream [11] for blending and generating images. In 2017, several scientists at Rutgers University collaborated with artists to generate a new art generation system via Generative Adversarial Networks (GAN). The system allows AI to learn deeply from established human artistic styles, moderately deviate and then emulate these classic works in art history by setting parameters [12]. Creative Adversarial Networks (CAN) (2017), released by Rutgers University and College of Charleston in the same year, performs better in “originality”. The main difference between them is that GAN can only imitate a specific style, while CAN “becomes creative by increasing the arousal potential of the generated art by deviating from the learned styles [13]”.

Various applications of AI art have emerged one after another since then. In 2017, MT Lab launched Andy, a painting robot that can compose illustration-style works. Figure 1 shows portrait of Edmond Belamy, an AI painting made by the French art team Obvious in 2018 which fetched up to $430,000 at auction. The Chinese-American artist Sougwen Chung offered viewers a unique experience of live performance presented by human-machine collaboration. The German artist Mario Klingemann revealed the process of AI-generated images via screen, creating an immersive and magical feeling [14]. With more AI art creations such as these put into practice, the combination of AI and visual art is becoming increasingly regular, and all kinds of digital media art and new media art are springing up. The arrival of AI has given birth to AI art and established the concept of AI aesthetics [15]. Generative (or intelligent) art, a branch of AI art, has also made a big splash recently.

The essential quality of art is creativity, and the expression of creativity lies in continuously discovering and constructing new relationships and connections [16]. Both of them are significant laws for creativity production as well as the basic logic behind AI art. The creativity of AI art depends on programs and algorithms which rely on clear logic. Furthermore, the basic logic refers to decomposability, processability, representability, and datability in art creation process. This is, in fact, similar to the basic idea of “dataism”. Its key points are that the universe comprises data flows, the value of any phenomenon or entity depends on its contribution to data processing, and the world is “shifting from a homo-centric to a data-centric view [17]”.

Fig. 1 Portrait of Edmond Belamy [Ref: Wikipedia https://en.wikipedia.org/wiki/Edmond_de_Belamy].

2 The convergence and interaction of AI and art

The American philosopher John Searle pays attention to the connection between cognitive abilities of computers and their psychological and philosophical meanings, thus separating them into “strong” and “weak” AI. He argues that the main value of weak AI is “to give us a very powerful tool,” while strong AI “can be literally said to understand and have other cognitive states [18]”. Weak AI concentrates on the instrumental properties of technology, while strong AI examines and judges the cognitive capabilities of artificial intelligence. The major tendencies of AI technology are as Figure 2. After more than 60 years of development, AI has crossed from learning to innovation. By fulfilling the closed-loop of human creation in an efficient, accurate and steady way, AI is now growing vibrantly and evolving into a leader in technological progress. As McLuhan states, “Rapidly, we approach the final phase of the extension of man—the technological simulation of consciousness [19]”.

In July 2017, to capture the strategic opportunity of AI development and speed up constructing an innovative country and world power in science and technology, China's State Council (2017) published and implemented the Development Planning for a New Generation of Artificial Intelligence. Since then, the government has vigorously promoted AI, and China has opened the “first year of AI”. In October 2017, Art Observation, a journal published by the Chinese National Academy of Arts, issued a feature on “Artificial Intelligence in Art [20]”. Almost at the same time, the Central Academy of Fine Arts (2017) launched a global education program with the theme of “Future Unknown”. Besides, Zhejiang University and Sichuan Fine Arts Institute have held special forums on “Art and Science,” making the integration of art and science a hot topic in the art world. In June 2019, Museum of Contemporary Art, Shanghai opened a “window of AI” for art. An exhibition titled “Basic World: Intelligent Art” kicked off the Art×AI series of cross-border exhibitions. With technologies like face recognition, face 3D reconstruction, style transfer, body sensing and data visualization, AI can autonomously comprehend art styles and designs in real-time, offering art a more vibrant life and more diverse extension [21].

In the era of intelligent Internet, AI is gradually integrating into every aspect of our life. Simultaneously, visual art has also taken on a new application paradigm because of the infusion of AI. The integration of AI and visual design has brought more efficient and convenient visual productivity for human society. AI representing science and art representing innovative design, are continuously combining, expanding, and fusing to jointly promote the progress of society and the development of civilization as a whole [22]. While acclaiming the artistic productivity of AI, it is equally essential to affirm the creativity of artists and designers [23].

According to artist Binkley [24], computers are not media art like paintings, sculptures, and installations. On the contrary, the uniqueness is its non-media nature as a kind of digital art, rather than materialized art. Based on the criterion of “media,” human art can be divided into three historical periods: manual art, media art, and intelligent art. the appearance of media art transforms the original “manual art” into a “tangible and intangible cultural heritage” of humanity. Likewise, the emergence of AI art marks a new phase of “heritagization” for “media art,” namely that “industrial heritage” and “digital heritage” have turned into new categories of human heritage. The comprehensive heritagization of human art shown in Figure 3 converts the research paradigm from a spatial dimension into the temporal one overall. The epoch of AI has created “human-like art,” fully transforming subjects of creating, ways of thinking, education, and values [25].

On the other hand, the powerful force of AI has raised grave concerns about whether it will displace artists' jobs. Fears of new technology can be dated back to at least the 19th century, when Luddite protesters destroyed mechanical weaving machines, and the fear of AI can be the same irrational reaction. According to AI researcher Aaron Hertzmann [26], there is no difference between a computer/AI program and a machine such as camera; computers do not create art, but their users do. Every technical means used by far, whether photography, film, or software algorithms, is basically a human tool, like a paintbrush and paint. Up to now, all “computer-generated art” has been the result of the artist's design or direct control. The human race is the master, and the computer is merely a complex tool for executing commands. From another perspective, computers and AI do enrich the means of creating art as an assisting tool, and alleviate basic heavy work in the traditional creation process as well. By the way, modifications in parameters of the AI program generate a kind of “uncertainty” and even develop a new sense of order in disorder, which can also be comprehended as a creative expression to some extent [27].

Fig. 2 The diagram of two major tendencies of AI technology.

Fig. 3 The diagram of the heritagization course of human art.

3 AI and large-scale art events: case study of Beijing Olympic winter games

The 24th Olympic Winter Games was held in Beijing, China. Under the principle of “romance, simplicity and beauty” (Zhang Yimou), the opening ceremony of the Games featured a series of technological means including AI algorithms, glasses-free 3D, augmented reality (AR) and image motion, which demonstrates a simple ethereal modern art style and presents a feast of technological aesthetics [28]. It provides a visual experience of “spacious but not empty, ethereal and romantic”, enormously enriching forms of stage performance through a new artistic expression.

In the center of the National Stadium, the venue for the ceremony, there was a large 8K ground screen covering a total area of 11,626 square meters, which was built with more than 40,000 LED modules [29]. It is the world's largest 3D stage, constituting a large space for immersive experience [30]. The interaction between actors and the LED ground screen became a highlight of the opening ceremony. In the performance of “Snowflake,” hundreds of children holding lightened “peace doves” danced and played in the center of the stage. Behind them were pieces of snowflakes moving and changing with the footsteps of the young actors, which turned out to be one of the most impressive moments in the audience's memory [31]. Figure 4 shows the stage of “Snowflake”.

Compared to non-real-time static recognition of traditional algorithms based on picture information, 3D Athlete Tracking (3DAT) used by this performance processes and investigates video stream data, so as to accurately track actors on the field and set up an independent action track one by one [32]. It is also the world's first attempt to apply AI visual technology to dynamically tracking over 600 people at the same time in such a spectacle.

The technical core of “Snowflake” primarily lies in resolving two issues: one is to identify the actors on stage and locate their position; the other is to guarantee real-time interaction. Tracking technology and real-time rendering are the two essential conditions to keep this technology operating properly. Collaborating with a professional team from Beijing Film Academy (BFA), Intel created an “AI-based real-time special effects system for large-scale performances,” intended for the Winter Olympics opening ceremony scenes. The former is supported by Intel's 3DAT technology, while BFA's real-time rendering supports the latter. The overall system consists of three sub-systems: camera information acquisition, data analysis and effects rendering, and its core algorithm is real-time human detecting and location tracking technology. The program leans on four cameras covering the whole venue to identify and locate little actors' movements on stage in real-time by means of AI and computer vision algorithms. Then, the signal data of the actors' key skeletal points and motion tracks collected by cameras are transmitted to Intel's edge computing system. Meanwhile, BFA's art effects team rendered real-time art effects according to the identified actor's position, and immediately fed them back to the LED controller for imaging. The whole process of real-time location, information collection, data analysis, and effect rendering takes less than 0.1 seconds, achieving an amazing dreamy effect of real-time interaction between the screen and the actors. The flowchart and interface of this AI-based system is as Figures 5–8.

The creative special effect inspired by Chinese ancient poem, “Waters of the Yellow River comes from the sky” shown in Figure 9, is also a digital interpretation of Chinese traditional art realized by AI technology that learns a lot from Chinese ink paintings and accordingly develops realistic visual effects in similar style [33]. On the vertical screen at the opening ceremony, a drop of ice-blue ink haloed and changed into a waterfall of the Yellow River. As the raging waves cascade down and rush to the ground screen, they guide the audience's line of sight along the momentum of water and make them feel like they are immersing in the actual water scene.

Chinese traditional art is a linear, temporal art determined by its cultural paradigm of “the harmony between man and nature” [34]. Just like musical melodies, Chinese calligraphy varies rhythmically with the movement of the brush, which is the soul of China's all kinds of plastic and expressive arts [35]. Therefore, “linearity” and “musicality” become the main features of Chinese traditional aesthetic system. The linear flow is exactly what the above poem uses to demonstrate the special mood of Chinese landscape painting, and thus to convey ultimate aesthetic experience to the audience. Wang Zhiou, visual effects director of the opening ceremony, explained, “Images of water flow on the screen are created by image processing algorithms. By ‘learning’ a large number of traditional Chinese ink paintings, the machine constructs a model of ink texture features and generates a stylized image of the landscape. To meet the demand for creative updates, we converted the 3D animation into point data the laser machine needed and achieved a quick effect iteration. This is the deconstruction and reconstruction to traditional Chinese ink painting style by digital means”. The interpretation of “ink art & digital technology” also reflects the design team's unique understanding of traditional culture, which combines both traditional aesthetics and the beauty of modern technology.

Water and ice, two elements of the Winter Olympics opening, form a subtle connection and an organic unity. The majestic waterfall flowing down from the sky paves the way for the subsequent solidification of ice and snow, and the appearance of the laser “Five Rings of Ice and Snow”. In the opening ceremony of the Beijing Winter Olympics, AI technology was adopted in many ways, integrating well with art and culture. The new media stage and visual art were perfectly blended, creating a multi-dimensional and highly infectious performance space. The unprecedented unification of virtuality and reality not only turns stage art much more sensual and ethereal, produces a strong sense of immersion for actors and the audience, but brings a visual feast full of Chinese romance to viewers. The entire performance attempts to illustrate the aesthetic style of minimalism of the Song dynasty, “to be vast and to be subtle,” and to emphasize the beauty of culture, art, spirit and technology.

In major events such as the Olympic games, AI technology enables creators to transmit cultural and spiritual core in larger spaces and broader dimensions, and also enriches types of visual effects. However, we should still be aware that technology is only a carrier of expression. Only by integrating technology into art, can we truly fulfill the goal of touching, influencing, and impressing people.

Fig. 4 The “Snowflake” performance during the 2022 Beijing Winter Olympics Opening Ceremony [Ref: XinhuaNet https://mp.weixin.qq.com/s?__biz=MzA4MjQxNjQzMA==&mid=2768709066&idx=1&sn=9dbc473762560cace39943a03aca411e&chksm=bf75838b88020a9dab6822894440f49c14a9258f1f1d8cb319729bf9a85a221b38ae2a2bd94a#rd].

Fig. 5 The workflow of “AI-based real-time special effects system for large-scale performances”.

Fig. 6 Performers create a dandelion shape display with LED lights during the 2022 Beijing Winter Olympics Opening Ceremony [Ref : HuffPost https://www.huffpost.com/entry/2022-winter-olympics-opening-ceremony-photos_n_61fd0dabe4b06abdc42ec853].

Fig. 7 The backend interface of “AI-based real-time special effects system for large-scale performances” [Ref: Intel China https://weibo.com/2637370927/LdXVoiqCv].

Fig. 8 The real-time 3D model of Olympic athletes built by 3DAT technology [Ref: Intel China https://zhuanlan.zhihu.com/p/466657044].

Fig. 9 The visual effect of “Waters of the Yellow River comes from the sky” at 2022 Beijing Winter Olympics Opening Ceremony [Ref: Xinhua Net https://mp.weixin.qq.com/s?__biz=MzA4MjQxNjQzMA==&mid=2768709066&idx=1&sn=9dbc473762560cace39943a03aca411e&chksm=bf75838b88020a9dab6822894440f49c14a9258f1f1d8cb319729bf9a85a221b38ae2a2bd94a#rd].

4 Advantages and limitations of AI application in art

The integration of technology and art has become an influential trend nowadays. New media offer new directions, paths, and languages for art creation. AI, as one of them, is being used more and more frequently in the creation of visual art. And its extensive application in visual design of large-scale events, such as the opening and closing ceremonies of the Beijing Winter Olympics, has promoted technology to integrate into art. The development of Metaverse has further prompted a comprehensive blend of the two realms [36]. AI has certain advantages over the human brain in terms of learning ability. It can do things that are difficult for humans to accomplish, such as digesting huge samples in a short period, and avoiding potential errors in repetitive and tedious work to reduce artists' loads to a certain extent.

Even so, the demand for large databases and high-quality hardware can make AI less practical in reality. Firstly, AI programs must learn from the sample data, no matter which algorithm they are based on. The better the sample data is, the more “intelligent” the AI program will be. But collecting samples is undoubtedly a huge effort. Take the work “Portrait of Edmond Belamy” as an example. It is based on a sample of 15,000 European portraits from the 14th to 20th centuries. In cases of “Snowflake” and “Waters of the Yellow River” mentioned above, it takes a considerable amount of manpower to collect and arrange samples, which seems hard for common artists to accomplish without teamwork. Secondly, the comprehensive performance of the hardware is also crucial. Computing power of the machine determines work efficiency to a certain extent, and its function of display is also highly required in visual art. As a result, it still costs relatively high to apply AI technology to visual art creation.

Even though people attempt to popularize AI in various fields, the technology is far from widespread in the field of art at this moment. DeepDream's action of opening source in 2014 did allow more people to use AI technology for creation, but the open source of only one or a few AI programs is not enough though. In the meantime, creating art simply by using established AI programs may limit artists' initiative to a certain extent. DeepDream is built on the basic logic of Convolutional Neural Network (CNN) [37,38]. Before running the program, creators must input an image into CNN, and then set the category of the image they need. CNN will compare the inaccuracy between the expected image and the output result to form a new one. Nonetheless, the fundamental working principle of the open-source software is determined beforehand, so that artists cannot entirely expressed their subjective ideas. The most essential part in art creation is to embody artists' subjectivity and creativity, rather than mechanically repeat by tools. Apparently, AI programs with set parameters cannot completely satisfy creators' needs. In addition, considering the substantial differences between disciplines of computer and art, there are relatively few talents who can cross over and build a link between them, or independently develop AI programs and apply them to art creation. The limitations of AI applications in visual art will remain for a period, meaning that it still needs some time to develop and iterate AI technology before its popularization in the field of art. Hopefully, the emergence of more trans-disciplinary talents will promote these problems to be solved.

It is undeniable that the appearance of new media presents more possibilities for artists and their art creations. Some scholars even consider that AI paintings are better than artists in terms of technique, because AI is more fearless and daring to innovate [39]. Nonetheless, AI doesn't possess intentional stance as human artists do, yet it's exactly the preliminary stage where art creation starts. The software programmed to generate art has no choice but to do as humans have commanded [40]. Therefore, it is fair to say that the success of artwork is more dependent on whether the selected medium fits the work's content, or how well artists display their creativity during the process, than on the intelligence degree of AI programs. AI has its unique advantages but also restrictions. Ultimately, AI-generated works are still combinations of selected databases of samples [41], lacking true imagination and creativity. With the development of AI technology, this issue may be solved in the future.

5 Conclusion

One of the most remarkable features of the Big Data era is quantitative thinking, meaning everything can be quantified, and AI is propelling the transformation of various industries as a quantitative medium. The development of art and design is increasingly focusing on the relationship between art and science, so AI technology is both an opportunity and a challenge for the development of art.

The rapid development of AI technology has stirred artistic imagination of human beings. In the field of art and design, the most incredible convenience AI brings to artists and designers is that it can replace tedious and inefficient repetitive work, as well as some trite conventions of art. AI is more like a catalyst in art and design, giving rise to the cross-fertilization of computer science and art and design, along with encouraging artistic creation. McLuhan highlights, “The effects of technology do not occur at the level of opinions or concepts, but alter sense ratios or patterns of perception steadily and without any resistance [42]”. The involvement of AI technology empowers art. As an “extension of senses,” it has expanded people's horizons, broadened the development path of art, enriched artistic expression, created special forms of presentation, and brought audiences an immersive interactive experience. Concurrently, it is beneficial in constructing diversified communication channels, promoting artwork, and breaking boundaries between industries.

Simultaneously, AI has reconstructed art in many ways. It not only dismantles the original aesthetic mechanism and brings about revolutionary challenges to traditional art, but also transforms the ecology of art and contributes to forming a new aesthetic system. Techniques including virtual reality, simulacra immersion, and fractal fragmentation profoundly affect aesthetic subjects' daily experience, interaction habits, aesthetic discourse and emotional experience, thus promoting the transformation of aesthetic forms, the release of aesthetic discourse as well as the reshaping of aesthetic values. Media such as languages and images are not merely separate entities, but also “others” that are culturally decoded in communication between aesthetic subjects, reflecting the refreshment of aesthetic experience. Furthermore, “the aestheticization of something nonmaterial not only impacts the construction of reality, but affects the existing mode of reality and our overall perception of reality [43]”.

To make a better future for the integration of AI and art, it is required to build up a virtuous interactive human-computer relationship and enhance art evaluation mechanism. In addition, critical issues like the extent of AI application in art creation, the essence of of art, the value of artworks and the subject of art creation remains to be reconsidered. Art creators and researchers must also make profounder reflection and discuss problems and solutions in the course of integrating AI and art. As Professor Daviel C. Dennett says, “AI has not yet solved any of our ancient riddles about the mind, but it has provided us with new ways of disciplining and extending philosophical imagination that we have only begun to exploit. [44]”.

References

All Figures

	Fig. 1 Portrait of Edmond Belamy [Ref: Wikipedia https://en.wikipedia.org/wiki/Edmond_de_Belamy].
In the text

	Fig. 2 The diagram of two major tendencies of AI technology.
In the text

	Fig. 3 The diagram of the heritagization course of human art.
In the text

	Fig. 4 The “Snowflake” performance during the 2022 Beijing Winter Olympics Opening Ceremony [Ref: XinhuaNet https://mp.weixin.qq.com/s?__biz=MzA4MjQxNjQzMA==&mid=2768709066&idx=1&sn=9dbc473762560cace39943a03aca411e&chksm=bf75838b88020a9dab6822894440f49c14a9258f1f1d8cb319729bf9a85a221b38ae2a2bd94a#rd].
In the text

	Fig. 5 The workflow of “AI-based real-time special effects system for large-scale performances”.
In the text

	Fig. 6 Performers create a dandelion shape display with LED lights during the 2022 Beijing Winter Olympics Opening Ceremony [Ref : HuffPost https://www.huffpost.com/entry/2022-winter-olympics-opening-ceremony-photos_n_61fd0dabe4b06abdc42ec853].
In the text

	Fig. 7 The backend interface of “AI-based real-time special effects system for large-scale performances” [Ref: Intel China https://weibo.com/2637370927/LdXVoiqCv].
In the text

	Fig. 8 The real-time 3D model of Olympic athletes built by 3DAT technology [Ref: Intel China https://zhuanlan.zhihu.com/p/466657044].
In the text

	Fig. 9 The visual effect of “Waters of the Yellow River comes from the sky” at 2022 Beijing Winter Olympics Opening Ceremony [Ref: Xinhua Net https://mp.weixin.qq.com/s?__biz=MzA4MjQxNjQzMA==&mid=2768709066&idx=1&sn=9dbc473762560cace39943a03aca411e&chksm=bf75838b88020a9dab6822894440f49c14a9258f1f1d8cb319729bf9a85a221b38ae2a2bd94a#rd].
In the text