An Effective Methodology for Processing and Managing Massive Spacecraft Datasets Haydar Teymourlouei Department of Computer Science Bowie State University, Bowie, MD, USA Abstract - The emergence of enormous and complex datasets has made existing data processing methods more strenuous. The growth in datasets continues to increase vastly. Despite the Interpolation technologies out there to manipulate efficient large data searching methods, the task to search for datasets expeditiously is still an obstacle. However, this research offers a more effective method to quickly search a large dataset within a timely manner. For this method to work, we used a technique where we create a directory file to catalogue and to retrieve data. The directory file is where the file can be acquired based on its time and date. The proposed method is intended to alleviate the process of searching a data’s content entirely and to scale down the search time in order to find the data file. Keywords: Big data, data processing, data set, interpolation search, raw data, unsorted data 1 Introduction As technology is expanding, so is the rapid acceleration of complex and diverse types of data results in the emergence of a fast paced algorithm. The sheer amount of data generated that must be ingested, analyzed, and managed is of relevant importance and must be considered when attempting to propose useful tools. The speed in which data must be received and be processed must also be considered. The rise of information coming from new sources has taken a toll on IT . Therefore, data management is a much more difficult task using only the traditional methodologies. Data has attained the form of continuous data streams rather than finite stored data sets, posing barriers to users that wish to obtain results at a preferred time. Data prescribed in this manner displays no bounds or limitations; thus, a delay in the retrieval of data can be expected. With today’s overflowing datasets, data management and analysis challenges are on a rise. Large data is described as a substantial amount of data accumulated over time that makes it difficult to examine and to process using various algorithms. Data analysis is the process of understanding the meaning of the data we have acquired, catalogued, and exhibited in representation’s form such as a table or a line graph. Working with millions or even billions of datasets has become problematic for researchers. If we can sort, approach, allocate, and evaluate the datasets competently, we can alleviate the trouble in searching for data. Many researchers believe that using various indexing methods to search for data expedites the search mechanism [2]. 2 Data Processing The term data is typically described as information. Data processing is basically conversion of raw data into meaningful information through process. Data is first gathered, then it gets processed. Large datasets usually refer to voluminous data beyond the capabilities of the current database technology. Data is used to refer to the vast amounts of information in a standardized format. Generally, data can include numbers, letters, equations, images, dates, figures, maps, documents, media files, and much more information. Particularly, data processing is a distinctive step in the information processing cycle. In information processing, “data is acquired, entered, validated and processed, stored and outputted, either in response to queries or in the form of routine reports. Data processing refers to the act of recording or otherwise handling one or more sets of data” [5]. The processing of data requires to be displayed in an understandable and efficient form. The levels must be given consecutively in order from the pathway to the result where is its readable to the reader. Nonetheless, large datasets are transforming the way research is carried out, resulting in the emergence of a fast-paced algorithm. Data has to be effectively processed in order to convert raw data into meaningful information. See Figure below for details. Figure 1: Raw Data Conversion 94 Int'l Conf. Scientific Computing | CSC'15 |