Guided Image Weathering using Image-to-Image Translation Li-Yu Chen National Taiwan University Taipei, Taiwan jcly.rikiu@gmail.com I-Chao Shen The University of Tokyo Tokyo, Japan ichao.shen@ui.is.s.u-tokyo.ac.jp Bing-Yu Chen National Taiwan University Taipei, Taiwan robin@ntu.edu.tw Figure 1: Weathering sequence guided by the age map (shown in the 2nd row). In an age map, the cold color (blue) indicates the less weathered region and the hot color (red) indicates the more weathered region. As the age value increases, the synthesized texture has more weathered efects on it. ABSTRACT In this paper, we present a guided image weathering method that allows the user to generate the weathering process. The core of our method is a three-step method to generate textures at diferent time steps of the weathering process. The input texture is analyzed frst to obtain the weathering degree (age map) for each pixel, then we train a conditional adversarial network to generate texture patches with diverse weathering efects. Once the training is fnished, new weathering results can be generated by manipulating the age map, such as automatic interpolation and manually modifed by the user. CCS CONCEPTS · Computing methodologies → Image manipulation; Graph- ics systems and interfaces. KEYWORDS image weathering, image to image translation ACM Reference Format: Li-Yu Chen, I-Chao Shen, and Bing-Yu Chen. 2021. Guided Image Weather- ing using Image-to-Image Translation. In SIGGRAPH Asia 2021 Technical Communications (SA ’21 Technical Communications), December 14–17, 2021, Tokyo, Japan. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/ 3478512.3488603 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specifc permission and/or a fee. Request permissions from permissions@acm.org. SA ’21 Technical Communications, December 14–17, 2021, Tokyo, Japan © 2021 Association for Computing Machinery. ACM ISBN 978-1-4503-9073-6/21/12. . . $15.00 https://doi.org/10.1145/3478512.3488603 1 INTRODUCTION The realistic texture is used by a wide range of applications: virtual reality, game development, digital visual efects, etc. The quality of textures can impact a lot, with more realistic ones often bring more immersion for the user experience. Thus, capturing realistic texture from the appearance of natural material becomes a major research area in computer graphics. However, the appearance of natural material is not always the same, for example, in the area with temperature variation near freeze point, water freezes into ice and melts to water again, the expansion due to volume changes between two states gives enough pressure causing rock cracking. Other examples like paint peeling, metal rusting are also common phenomena around our daily life. The process of the natural environment infuencing materials is known as weathering efect. Weathering efect can be synthesized by simulating the physical process. The difcult part is that diferent materials usually have their type of weathering process, and constructing a simulation for each material may also not be an efective way. This makes weathering efect synthesis a challenging task. By observing the texture, we can fnd that weathered pixels often cover only a small part of the entire texture. That means calculating pixel distance inside texture gives a rough guess of weathering degree (age map) of the texture, then we can use the mapping as guidance to generate new textures. In this paper, we develop a three-step method for generating textures at diferent time steps of the weathering process. Firstly, we analyze the input texture and predict the age value for each pixel [Bellini et al. 2016]. Secondly, we randomly crop pair patches of age map and texture, and deal with them as an image-to-image translation problem, which is achieved by training a conditional adversarial network [Isola et al. 2017]. Lastly, by manipulating