Journal of Theoretical and Applied Information Technology 31 st March 2015. Vol.73 No.3 © 2005 - 2015 JATIT & LLS. All rights reserved . ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 456 RECONSTRUCTING, AUGMENTING AND MANIPULATING 3D SCENE FROM SEQUENCE OF IMAGES 1 Osama Hosam * , 2 Nadhir Ben Halima, 3 Sameh Dakroury, 4 Essam O. Abdel-Rahman 1,2,3,4 Asstt Prof., Faculty of Computer Science and Engineering, TAIBAH UNIVERSITY, KSA 1 Asstt Prof., The City for Scientific Research and Technology Applications, IRI, Alexandria, EGYPT 3 Asstt. Prof., Department of Engineering Mathematics, Faculty of Engineering, Cairo University, EGYPT * Correspondence Author E-mail: 1 mohandesosama@yahoo.com , 2 nadhir_bh@yahoo.fr , 3 sdakrory@hotmail.com , 4 essamothman1@yahoo.com ABSTRACT In this paper, the scene parameters are setup for scene reconstruction; camera position and orientation are considered. Consecutive images are used to create the depth map of the scene. After reconstructing the scene, we apply a new methodology to augment the scene by using LAB color space and K-means clustering. A general scheme for scene manipulation is also introduced. The new methodology has shown high accuracy in scene reconstruction and augmentation. Keywords: 3D Reconstruction, Scene Manipulation, Augmented Reality, Shape from Stereo, 3D Point Cloud Generation. 1. INTRODUCTION The detection of third dimension by using multiple images follows naturally from the physical behavior of human eyes in the vision of 3D objects. The two eyes take pairs of images (one for each eye) for the same view. Using this image pair (also called stereo image), the depth or the third dimension can be defined. Many approaches are available for detecting the third dimension of an image, namely, shape from shading [1], shape from texture [2], shape from motion [3], and shape from multiple images or pair of images (stereo) [4, 5, 6]. The main idea behind constructing 3D model from multiple images is the concept of motion parallax [8]. An observer looking outside of car glass window will notice that objects near the car will move faster than those far away. This shift in speed is called the motion parallax. The same concept can be applied in case of an aircraft taking multiple images. Higher objects will appear moving faster than lower objects. In this case, the shift in speed is called the x-parallax of the satellite image [9]. Therefore, given multiple images of the same view, objects will not appear in the same position in each image. Instead, objects position will be shifted in the image. This shift is called the disparity. Correspondence or matching can be used to find the disparity [4]. In this paper, we introduce an approach for reconstructing and manipulating the scene from multiple images. First, the reconstruction of 3D cloud points of the scene is shown. Second, the scene is reconstructed in colored format and finally, the 3D reconstructed scene is manipulated with interpolation approach. 2. SETTING UP THE RECONSTRUCTION PARAMETERS In this section, the preliminaries needed for 3D reconstruction are introduced. For example; how to do projections of the points; how to transform them from 2D to 3D and vice versa. Then how to build the colored point cloud from the original image and the provided depth map. Projections: Assuming the camera calibration (camera parameters are properly set), the third dimension can be inferred from pair of images (successive images in the scene sequence). As shown in Figure 1. Where P is the point in the scene and   and   are the projections of the point on the camera plan for the left and right images.