This open access book was prepared as a Final Publication of the COST Action IC1406 “High-Performance Modelling and Simulation for Big Data Applications (cHiPSet)“ project. Long considered important pillars of the scientific method, Modelling and Simulation have evolved from traditional discrete numerical methods to complex data-intensive continuous analytical optimisations. Resolution, scale, and accuracy have become essential to predict and analyse natural and complex systems in science and engineering. When their level of abstraction raises to have a better discernment of the domain at hand, their representation gets increasingly demanding for computational and data resources. On the other hand, High Performance Computing typically entails the effective use of parallel and distributed processing units coupled with efficient storage, communication and visualisation systems to underpin complex data-intensive applications in distinct scientific and technical domains. It is then arguably required to have a seamless interaction of High Performance Computing with Modelling and Simulation in order to store, compute, analyse, and visualise large data sets in science and engineering. Funded by the European Commission, cHiPSet has provided a dynamic trans-European forum for their members and distinguished guests to openly discuss novel perspectives and topics of interests for these two communities. This cHiPSet compendium presents a set of selected case studies related to healthcare, biological data, computational advertising, multimedia, finance, bioinformatics, and telecommunications.

This book consists of eight chapters, five of which provide a summary of the tutorials and workshops organised as part of the cHiPSet Summer School: High-Performance Modelling and Simulation for Big Data Applications Cost Action on “New Trends in Modelling and Simulation in HPC Systems,” which was held in Bucharest (Romania) on September 21–23, 2016. As such it offers a solid foundation for the development of new-generation data-intensive intelligent systems. Modelling and simulation (MS) in the big data era is widely considered the essential tool in science and engineering to substantiate the prediction and analysis of complex systems and natural phenomena. MS offers suitable abstractions to manage the complexity of analysing big data in various scientific and engineering domains. Unfortunately, big data problems are not always easily amenable to efficient MS over HPC (high performance computing). Further, MS communities may lack the detailed expertise required to exploit the full potential of HPC solutions, and HPC architects may not be fully aware of specific MS requirements. The main goal of the Summer School was to improve the participants’ practical skills and knowledge of the novel HPC-driven models and technologies for big data applications. The trainers, who are also the authors of this book, explained how to design, construct, and utilise the complex MS tools that capture many of the HPC modelling needs, from scalability to fault tolerance and beyond. In the final three chapters, the book presents the first outcomes of the school: new ideas and novel results of the research on security aspects in clouds, first prototypes of the complex virtual models of data in big data streams and a data-intensive computing framework for opportunistic networks. It is a valuable reference resource for those wanting to start working in HPC and big data systems, as well as for advanced researchers and practitioners.

Serving as a flagship driver towards advance research in the area of Big Data platforms and applications, this book provides a platform for the dissemination of advanced topics of theory, research efforts and analysis, and implementation oriented on methods, techniques and performance evaluation. In 23 chapters, several important formulations of the architecture design, optimization techniques, advanced analytics methods, biological, medical and social media applications are presented. These chapters discuss the research of members from the ICT COST Action IC1406 High-Performance Modelling and Simulation for Big Data Applications (cHiPSet). This volume is ideal as a reference for students, researchers and industry practitioners working in or interested in joining interdisciplinary works in the areas of intelligent decision systems using emergent distributed computing paradigms. It will also allow newcomers to grasp the key concerns and their potential solutions.

This book provides a review of advanced topics relating to the theory, research, analysis and implementation in the context of big data platforms and their applications, with a focus on methods, techniques, and performance evaluation. The explosive growth in the volume, speed, and variety of data being produced every day requires a continuous increase in the processing speeds of servers and of entire network infrastructures, as well as new resource management models. This poses significant challenges (and provides striking development opportunities) for data intensive and high-performance computing, i.e., how to efficiently turn extremely large datasets into valuable information and meaningful knowledge. The task of context data management is further complicated by the variety of sources such data derives from, resulting in different data formats, with varying storage, transformation, delivery, and archiving requirements. At the same time rapid responses are needed for real-time applications. With the emergence of cloud infrastructures, achieving highly scalable data management in such contexts is a critical problem, as the overall application performance is highly dependent on the properties of the data management service.

Systems Simulation and Modelling for Cloud Computing and Big Data Applications provides readers with the most current approaches to solving problems through the use of models and simulations, presenting SSM based approaches to performance testing and benchmarking that offer significant advantages. For example, multiple big data and cloud application developers and researchers can perform tests in a controllable and repeatable manner. Inspired by the need to analyze the performance of different big data processing and cloud frameworks, researchers have introduced several benchmarks, including BigDataBench, BigBench, HiBench, PigMix, CloudSuite and GridMix, which are all covered in this book. Despite the substantial progress, the research community still needs a holistic, comprehensive big data SSM to use in almost every scientific and engineering discipline involving multidisciplinary research. SSM develops frameworks that are applicable across disciplines to develop benchmarking tools that are useful in solutions development. Examines the methodology and requirements of benchmarking big data and cloud computing tools, advances in big data frameworks and benchmarks for large-scale data analytics, and frameworks for benchmarking and predictive analytics in big data deployment Discusses applications using big data benchmarks, such as BigDataBench, BigBench, HiBench, MapReduce, HPCC, ECL, HOBBIT, GridMix and PigMix, and applications using big data frameworks, such as Hadoop, Spark, Samza, Flink and SQL frameworks Covers development of big data benchmarks to evaluate workloads in state-of-the-practice heterogeneous hardware platforms, advances in modeling and simulation tools for performance evaluation, security problems and scalable cloud computing environments

The objective of this book is to introduce the basic concepts of big data computing and then to describe the total solution of big data problems using HPCC, an open-source computing platform. The book comprises 15 chapters broken into three parts. The first part, Big Data Technologies, includes introductions to big data concepts and techniques; big data analytics; and visualization and learning techniques. The second part, LexisNexis Risk Solution to Big Data, focuses on specific technologies and techniques developed at LexisNexis to solve critical problems that use big data analytics. It covers the open source High Performance Computing Cluster (HPCC Systems®) platform and its architecture, as well as parallel data languages ECL and KEL, developed to effectively solve big data problems. The third part, Big Data Applications, describes various data intensive applications solved on HPCC Systems. It includes applications such as cyber security, social network analytics including fraud, Ebola spread modeling using big data analytics, unsupervised learning, and image classification. The book is intended for a wide variety of people including researchers, scientists, programmers, engineers, designers, developers, educators, and students. This book can also be beneficial for business managers, entrepreneurs, and investors.

This book constitutes the referred proceedings of two workshops held at the 32nd ACM International Conference on Supercomputing, ACM ICS 2018, in Beijing, China, in June 2018. This volume presents the papers that have been accepted for the following workshops: Second International Workshop on High Performance Computing for Advanced Modeling and Simulation in Nuclear Energy and Environmental Science, HPCMS 2018, and First International Workshop on HPC Supported Data Analytics for Edge Computing, HiDEC 2018. The 20 full papers presented during HPCMS 2018 and HiDEC 2018 were carefully reviewed and selected from numerous submissions. The papers reflect such topics as computing methodologies; parallel algorithms; simulation types and techniques; machine learning.