7.30 | Amber: A Debuggable Dataflow System Based on the Actor Model

演讲简介

A long-running analytic task on big data often leaves a developer in the dark without providing valuable feedback about the status of the execution. In addition, a failed job that needs to restart from scratch can waste earlier computing resources. An effective method to address these issues is to allow the developer to debug the task during its execution, which is unfortunately not supported by existing big data solutions. In this talk we present a system we are developing called Amber to support interactive debugging during the execution of a workflow task. After starting the execution, the developer can pause the job at will, investigate the states of the cluster, modify the job, and resume the computation. She can also set conditional breakpoints to pause the execution when certain conditions are satisfied. Amber is based on the actor model, a distributed computing paradigm that provides concurrent units of computation using actors. We give a full specification of Amber, and implement it on top of the Orleans system. Our experiments show its high usability of interactive debugging on computing clusters, with performance comparable to Spark. Amber is developed in the Texera project with the goal of supporting big data analytics using GUI-based workflows without much programming effort.

关于讲者

Chen Li is a professor in the Department of Computer Science at UC Irvine. He received his Ph.D. degree in Computer Science from Stanford University, and his M.S. and B.S. in Computer Science from Tsinghua University, China, respectively. His research interests are in the field of data management, including data-intensive computing, query processing and optimization, visualization, and text analytics. His current focus is building open source systems for data management and analytics. He was a recipient of an NSF CAREER award and several test-of-time publication awards, a part-time visiting research scientist at Google, PC co-chair of VLDB 2015, and an ACM distinguished member. He took a roller coaster ride by doing a company to commercialize university research.