BIG DATA 2015-16

IEEE 2015: FastRAQ: A Fast Approach to Range-Aggregate Queries in Big Data Environments

ABSTRACT: Range-aggregate queries are to apply a certain aggregate function on all tuples within given query ranges. Existing approaches to range-aggregate queries are insufficient to quickly provide accurate results in big data environments. In this paper, we propose FastRAQ—a fast approach to range-aggregate queries in big data environments. FastRAQ first divides big data into independent partitions with a balanced partitioning algorithm, and then generates a local estimation sketch for each partition. When a range-aggregate query request arrives, FastRAQ obtains the result directly by  summarizing local estimates from all partitions. FastRAQ has Oð1Þ time complexity for data updates and Oð N P_BÞ time complexity for range-aggregate queries, where N is the number of distinct tuples for all dimensions, P is the partition number, and B is the bucket number in the histogram. We implement the FastRAQ approach on the Linux platform, and evaluate its performance with about 10 billions data records. Experimental results demonstrate that FastRAQ provides range-aggregate query results within a time period two orders of magnitude lower than that of Hive, while the relative error is less than 3 percent within the given confidence interval.

IEEE 2015: Deploying Large-Scale Datasets on-Demand in the Cloud: Treats and Tricks on Data Distribution

ABSTRACT: Public clouds have democratised the access to analytics for virtually any institution in the world. Virtual machines (VMs) can be provisioned on demand to crunch data after uploading into the VMs. While this task is trivial for a few tens of VMs, it becomes increasingly complex and time consuming when the scale grows to hundreds or thousands of VMs crunching tens or hundreds of TB. Moreover, the elapsed time comes at a price: the cost of provisioning VMs in the cloud and keeping them waiting to load the data. In this paper we present a big data provisioning service that incorporates hierarchical and peer-to-peer data distribution techniques to speed-up data loading into the VMs used for data processing. The system dynamically mutates the sources of the data for the VMs to speed-up data loading. We tested this solution with 1000 VMs and 100 TB of data, reducing time by at least 30 percent over current state of the art techniques. This dynamic topology mechanism is tightly coupled with classic declarative machine configuration techniques (the system takes a single high-level declarative configuration file and configures both software and data loading). Together, these two techniques simplify the deployment of big data in the cloud for end users who may not be experts in infrastructure management.

IEEE 2015: Cccloud: Context-Aware And Credible Cloud Service Selection Based On Subjective Assessment And Objective Assessment

ABSTRACT: Due to the diversity and dynamic nature of cloud services, it is usually hard for potential cloud consumers to select the most suitable cloud service. This paper proposes CCCloud: a context-aware and credible cloud service selection model based on the comparison and aggregation of subjective assessments extracted from ordinary cloud consumers and objective assessments from quantitative performance testing parties. We propose a novel approach to evaluate cloud users’ credibility, which not only can accurately evaluate how truthfully they assess cloud services, but also resist user collusion. In addition, in our model, objective assessments are used as benchmarks to filter out potentially biased subjective assessments, and then objective assessments and subjective assessments are aggregated to evaluate the overall performance of a cloud service. Furthermore, our model takes the contexts of objective assessments and subjective assessments into account. By calculating the similarity between different contexts, the benchmark level of objective assessments is dynamically adjusted according to  context similarity, and the aggregated final scores of alternative cloud services are weighted by the similarity between the contexts of a potential cloud consumer and every testing party. This makes our cloud service selection model reflect potential cloud consumers’ customized requirements more effectively. Finally, our proposed model is evaluated through the experiments conducted under different conditions. The experimental results demonstrate that our model significantly outperforms the existing work, especially in the resistance of user collusion.

IEEE 2015: Processing Cassandra Datasets with Hadoop-Streaming Based Approaches

ABSTRACT: The progressive transition in the nature of bothscientific and industrial datasets has been the driving force behind the development and research interests in the NoSQL model. Loosely structured data poses a challenge to traditional data store systems, and when working with the NoSQL model, these systems are often considered impractical and costly. As the quantity and quality of unstructured data grows, so does the demand for a processing pipeline that is capable of seamlessly combining the NoSQL storage model and a “Big Data” processing platform such as MapReduce. Although MapReduce is the paradigm of choice for data-intensive computing, Java-based frameworks such as Hadoop require users to write MapReduce code in Java while Hadoop Streaming module allows users to define non- Java executables as map and reduce operations. When confronted with legacy C/C++ applications and other non-Java executables, there arises a further need to allow NoSQL data stores access to the features of Hadoop Streaming. We present approaches in solving the challenge of integrating NoSQL data stores with MapReduce under non-Java application scenarios, along with advantages and disadvantages of each approach. We compare Hadoop Streaming alongside our own streaming framework, MARISSA, to show performance implications of coupling NoSQL data stores like Cassandra with MapReduce frameworks that normally rely on file-system based data stores. Our experiments also include Hadoop-C*, which is a setup where a Hadoop cluster is co-located with a Cassandra cluster in order to process data using Hadoop with non-java executables.

IEEE 2015: i2MapReduce: Incremental MapReduce for Mining Evolving Big Data

ABSTRACT: As new data and updates are constantly arriving, the results of data mining applications become stale and obsolete over time. Incremental processing is a promising approach to refreshing mining results. It utilizes previously saved states to avoid the expense of re-computation from scratch. In this paper, we propose i2MapReduce, a novel incremental processing extension to MapReduce, the most widely used framework for mining big data. Compared with the state-of-the- art work on Incoop, i2MapReduce (i) performs key-value pair level incremental processing rather than task level re-computation, (ii) supports not only one-step computation but also more sophisticated iterative computation, which is widely used in data mining applications, and (iii) incorporates a set of novel techniques to reduce I/O overhead for accessing preserved fine-grain computation states.We evaluate i2MapReduce using a one-step algorithm and four iterative algorithms with diverse computation characteristics. Experimental results on Amazon EC2 show significant performance improvements of i2MapReduce compared to both plain and iterative MapReduce performing re-computation.

IEEE 2015: In-Memory Big Data Management and Processing: A Survey

ABSTRACT: Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing.

IEEE 2015: Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors

ABSTRACT: Extracting latent low-dimensional structure from high-dimensional data is of paramount importance in timely inference tasks encountered with “Big Data” analytics. However, increasingly noisy, heterogeneous, and incomplete datasets, as well as the need for real-time processing of streaming data, pose major challenges to this end. In this context, the present paper permeates benefits from rank minimization to scalable imputation of missing data, via tracking low-dimensional subspaces and unraveling latent (possibly multi-way) structure from incomplete streaming data. For low-rank matrix data, a subspace estimator is proposed based on an exponentially weighted least-squares criterion regularized with the nuclear norm. After recasting the nonseparable nuclear norm into a form amenable to online optimization, real-time algorithms with complementary strengths are developed, and their convergence is established under simplifying technical assumptions. In a stationary setting, the asymptotic estimates obtained offer the well-documented performance guarantees of the batch nuclear-norm regularized estimator. Under the same unifying framework, a novel online (adaptive) algorithm is developed to obtain multi-way decompositions of low-rank tensors with missing entries and perform imputation as a byproduct. Simulated tests with both synthetic as well as real Internet and cardiac magnetic resonance imagery (MRI) data confirm the efficacy of the proposed algorithms, and their superior performance relative to state-of-the-art alternatives.

IEEE 2015: Stochastic Decision Making for Adaptive Crowdsourcing in Medical Big-Data Platforms

ABSTRACT: This paper proposes two novel algorithms for adaptive crowdsourcing in 60-GHz medical imaging big-data platforms, namely, a max-weight scheduling algorithm for medical cloud platforms and a stochastic decision-making algorithm for distributed power-and-latency-aware dynamic buffer management in medical devices. In the first algorithm, medical cloud platforms perform a joint queue-backlog and rate-aware scheduling decisions for matching deployed access points (APs) and medical users where APs are eventually connected to medical clouds. In the second algorithm, each scheduled medical device computes the amounts of power allocation to upload its own medical data to medical big-data clouds with stochastic decision making considering joint energy-efficiency and buffer stability optimization. Through extensive simulations, the proposed algorithms are shown to achieve the desired results.

 IEEE 2015: Privacy-Preserving Ciphertext Multi-Sharing Control for Big Data Storage

ABSTRACT: The need of secure big data storage service is more desirable than ever to date. The basic requirement of the service is to guarantee the confidentiality of the data. However, the anonymity of the service clients, one of the most essential aspects of privacy, should be considered simultaneously. Moreover, the service also should provide practical and finegrained encrypted data sharing such that a data owner is allowed to share a ciphertext of data among others under some specified conditions. This paper, for the first time, proposes a privacy preserving ciphertext multi-sharing mechanism to achieve the above properties. It combines the merits of proxy re-encryption with anonymous technique in which a ciphertext can be securely and conditionally shared multiple times without leaking both the knowledge of underlying message and the identity information of ciphertext senders/recipients. Furthermore, the paper shows that the new primitive is secure against chosen-ciphertext attacks in the standard model.