Loading…
Welcome to the Earth Science Information Partners (ESIP) 2018 Summer Meeting! The 2018 theme is Realizing the Socioeconomic Value of Data. The theme is based on one of the goals in the 2015 - 2020 ESIP Strategic Plan, which provides a framework for ESIP’s activities over the next three years.

If you haven’t already, register here!
_____________
View analytic
Wednesday, July 18 • 4:00pm - 5:30pm
TaskAPI - A Scalable Computing Platform for Large Scientific Data Systems

Sign up or log in to save this to your schedule and see who's attending!

TaskAPI is a workflow platform and DSL (Domain Specific Language) that provides automatic horizontal and vertical scaling of multi-language data-intensive scientific software systems using a functionally declarative workflow paradigm. TaskAPI is capable of quickly wrapping legacy systems, provides structured guidance for best-practices in continued or new development via its JSON DSL, and automatically provides system components with a unified, straightforward API for centralized logging, job and task killing, and configurable property use.

TaskAPI was developed to serve as the backbone for the reengineered US ASOS Ingest software system and exists as its own distributable package for use by other large polyglot systems.

This session will begin by providing a broad overview (surface skim) of the TaskAPI platform, including motivation, capabilities, current and potential use cases, and design and performance characteristics.

The summary will lead into a more detailed look at the TaskAPI structure, including the DSL setup, workflow branching, task types, multi-language parallelization techniques in Java, C, and Fortran, and current and planned language support (Python via Jep, Clojure, Scala via drivers), and other features (Kafka messaging queues).

After we thoroughly explain the system and its capabilities, we will deep dive into a live example of TaskAPI as it was implemented in ASOS, examining real-life challenges we faced and how to think about and implement best use practices.

Finally, we will assist session attendees and participants in determining if this system could serve their own projects, provide assistance in TaskAPI download and setup, and solicit feature requests and needs.

Speakers & Moderators
avatar for Ryan Berkheimer

Ryan Berkheimer

Software Research, GST at NOAA NCEI


Wednesday July 18, 2018 4:00pm - 5:30pm
Room 1
Feedback form isn't open yet.

Attendees (6)