This book offers a much needed critical introduction to data mining and ‘big data’. Supported by multiple case studies and examples, the authors provide: Digestible overviews of key terms and concepts relevant to using social media data in quantitative research. A critical review of data mining and ‘big data’ from a complexity science perspective, including its future potential and limitations A practical exploration of the challenges of putting together and managing a ‘big data’ database An evaluation of the core mathematical and conceptual frameworks, grounded in a case-based computational modeling perspective, which form the foundations of all data mining techniques Part of The SAGE Quantitative Research Kit, this book will give you the know-how and confidence needed to succeed on your quantitative research journey.

This volume conveys some of the surprises, puzzles and success stories in high-dimensional and complex data analysis and related fields. Its peer-reviewed contributions showcase recent advances in variable selection, estimation and prediction strategies for a host of useful models, as well as essential new developments in the field. The continued and rapid advancement of modern technology now allows scientists to collect data of increasingly unprecedented size and complexity. Examples include epigenomic data, genomic data, proteomic data, high-resolution image data, high-frequency financial data, functional and longitudinal data, and network data. Simultaneous variable selection and estimation is one of the key statistical problems involved in analyzing such big and complex data. The purpose of this book is to stimulate research and foster interaction between researchers in the area of high-dimensional data analysis. More concretely, its goals are to: 1) highlight and expand the breadth of existing methods in big data and high-dimensional data analysis and their potential for the advancement of both the mathematical and statistical sciences; 2) identify important directions for future research in the theory of regularization methods, in algorithmic development, and in methodologies for different application areas; and 3) facilitate collaboration between theoretical and subject-specific researchers.

The amount of new information is constantly increasing, faster than our ability to fully interpret and utilize it to improve human experiences. Addressing this asymmetry requires novel and revolutionary scientific methods and effective human and artificial intelligence interfaces. By lifting the concept of time from a positive real number to a 2D complex time (kime), this book uncovers a connection between artificial intelligence (AI), data science, and quantum mechanics. It proposes a new mathematical foundation for data science based on raising the 4D spacetime to a higher dimension where longitudinal data (e.g., time-series) are represented as manifolds (e.g., kime-surfaces). This new framework enables the development of innovative data science analytical methods for model-based and model-free scientific inference, derived computed phenotyping, and statistical forecasting. The book provides a transdisciplinary bridge and a pragmatic mechanism to translate quantum mechanical principles, such as particles and wavefunctions, into data science concepts, such as datum and inference-functions. It includes many open mathematical problems that still need to be solved, technological challenges that need to be tackled, and computational statistics algorithms that have to be fully developed and validated. Spacekime analytics provide mechanisms to effectively handle, process, and interpret large, heterogeneous, and continuously-tracked digital information from multiple sources. The authors propose computational methods, probability model-based techniques, and analytical strategies to estimate, approximate, or simulate the complex time phases (kime directions). This allows transforming time-varying data, such as time-series observations, into higher-dimensional manifolds representing complex-valued and kime-indexed surfaces (kime-surfaces). The book includes many illustrations of model-based and model-free spacekime analytic techniques applied to economic forecasting, identification of functional brain activation, and high-dimensional cohort phenotyping. Specific case-study examples include unsupervised clustering using the Michigan Consumer Sentiment Index (MCSI), model-based inference using functional magnetic resonance imaging (fMRI) data, and model-free inference using the UK Biobank data archive. The material includes mathematical, inferential, computational, and philosophical topics such as Heisenberg uncertainty principle and alternative approaches to large sample theory, where a few spacetime observations can be amplified by a series of derived, estimated, or simulated kime-phases. The authors extend Newton-Leibniz calculus of integration and differentiation to the spacekime manifold and discuss possible solutions to some of the "problems of time". The coverage also includes 5D spacekime formulations of classical 4D spacetime mathematical equations describing natural laws of physics, as well as, statistical articulation of spacekime analytics in a Bayesian inference framework. The steady increase of the volume and complexity of observed and recorded digital information drives the urgent need to develop novel data analytical strategies. Spacekime analytics represents one new data-analytic approach, which provides a mechanism to understand compound phenomena that are observed as multiplex longitudinal processes and computationally tracked by proxy measures. This book may be of interest to academic scholars, graduate students, postdoctoral fellows, artificial intelligence and machine learning engineers, biostatisticians, econometricians, and data analysts. Some of the material may also resonate with philosophers, futurists, astrophysicists, space industry technicians, biomedical researchers, health practitioners, and the general public.

The amount and the complexity of the data gathered by current enterprises are increasing at an exponential rate. Consequently, the analysis of Big Data is nowadays a central challenge in Computer Science, especially for complex data. For example, given a satellite image database containing tens of Terabytes, how can we find regions aiming at identifying native rainforests, deforestation or reforestation? Can it be made automatically? Based on the work discussed in this book, the answers to both questions are a sound “yes”, and the results can be obtained in just minutes. In fact, results that used to require days or weeks of hard work from human specialists can now be obtained in minutes with high precision. Data Mining in Large Sets of Complex Data discusses new algorithms that take steps forward from traditional data mining (especially for clustering) by considering large, complex datasets. Usually, other works focus in one aspect, either data size or complexity. This work considers both: it enables mining complex data from high impact applications, such as breast cancer diagnosis, region classification in satellite images, assistance to climate change forecast, recommendation systems for the Web and social networks; the data are large in the Terabyte-scale, not in Giga as usual; and very accurate results are found in just minutes. Thus, it provides a crucial and well timed contribution for allowing the creation of real time applications that deal with Big Data of high complexity in which mining on the fly can make an immeasurable difference, such as supporting cancer diagnosis or detecting deforestation.

This book ties together big data, data mining, and analytics to explain how readers can leverage them to transform their business strategy. Illustrating basic approaches of business intelligence to data and text mining, the book guides readers through the process of extracting valuable knowledge from the varieties of data currently being generated in the brick and mortar and Internet environments. It considers the broad spectrum of analytics approaches for decision making, including dashboards, OLAP cubes, data mining, and text mining.

The field of data mining has made significant and far-reaching advances over the past three decades. Because of its potential power for solving complex problems, data mining has been successfully applied to diverse areas such as business, engineering, social media, and biological science. Many of these applications search for patterns in complex structural information. In biomedicine for example, modeling complex biological systems requires linking knowledge across many levels of science, from genes to disease. Further, the data characteristics of the problems have also grown from static to dynamic and spatiotemporal, complete to incomplete, and centralized to distributed, and grow in their scope and size (this is known as big data). The effective integration of big data for decision-making also requires privacy preservation. The contributions to this monograph summarize the advances of data mining in the respective fields. This volume consists of nine chapters that address subjects ranging from mining data from opinion, spatiotemporal databases, discriminative subgraph patterns, path knowledge discovery, social media, and privacy issues to the subject of computation reduction via binary matrix factorization.