Intro

On a fundamental level distributed computing is a system whose components are connected through communication networks and are coordinated through the exchange of messages. Metcalfe's law states that the potential benefits derived from a distributed system is proportional to the square of number of participants. There are many types of distributed system topologies that govern how participants operate, as shown in the figure below:

Illustration of centralized, decentralized, and distributed systems

The largest distributed system is the Internet itself, which is made up of many types of globally connected devices. However, most Internet applications are not decentralized or even distributed. Most applications are build in the client-server model, shown in the figure below, requiring the end-to-end support of the TCP/IP protocol and communicating with a central internet-connected system. These constrains have lead to, in many cases, a network messages traversing the Internet to some remote and potentially distant datacenter and back that should have stayed in the building, or even the room.

Client-server: Client transfers data to a central point where decisions are made.

The application of AI in real-time applications associated with the Internet of Things (IoT), Cyber-Physical Systems (CPS), Machine to Machine (M2M) technologies, and Industrial Internet stand to improve decision velocity and accuracy for a wide-range of data-driven applications. The increasing growth of devices, data sources, and data have made current practices of moving data directly from end-devices to remote and potentially distant cloud computing services insufficient. Edge computing techniques are required to extract data from these networks of connected devices.

Edge Computing

What is edge computing?

In the edge computing paradigm data might be processed, or decisions might be made, anywhere throughout the network. Fundamentally, the field of edge computing is the study of when you move data and when to move functionality. A comparison of client-server and edge computing models are shown in the figure below:

Edge Model: End-devices and intermediate nodes not only generate and transmit data, they also make decisions.

Why is edge computing needed?

  • There exist technical constraints (speed of light, computational, storage, throughput, power, etc.) on the movement and processing of data remotely.
  • There exist a data-value paradox, where information that is required to model data value has an indeterminate collection rate and duration.
  • This is to say that the value of data is not known until the cost of transmission and processing have been incurred.
  • There exist privacy, legal, and operational constrains in the exposing of protected data to remote systems.

    • Edge Computing is the migration of computational functionality to sources of data generation. The importance of edge computing increases with the size and complexity of devices and resulting data. In addition, the coordination of global edge-to-edge communications, shared resources, high-level application scheduling, monitoring, measurement, and Quality of Service (QoS) enforcement will be critical to address the rapid growth of connected devices and associated data.


The Platform

We present a new distributed agent-based platform designed to address the challenges of edge computing. This actor-model platform implementation is designed to manage large numbers of geographically distributed services, comprised from heterogeneous resources and communication protocols, in support of low-latency real-time streaming applications. As part of this platform, an application description language was developed and implemented. Using the application description language a number of high-order management modules have been implemented including solutions for resource and workload comparison, performance observation, scheduling, and provisioning.

Cresco functions across the entire application stack, allowing the rapid development of high-level edge computing applications, across diverse operating environments.

Edge computing across the application stack.

Distributed Data and Infrastructure Layer

Cresco is based on agents that run across a wide range of devices, from embedded systems to the largest servers. On a Raspberry Pi the agent operating in agent-only uses a negligible computational resources and operates in less than 25MB of ram. The same agent code on a server can manage over 100k agents when functioning as a global controller. The agent provides a platform for multi-protocol communication and a launch point for plugins, such as event processor, container, executor, and other modules.

Determine node resources, monitor utilization, schedule tasks.

Data Transport and Integration Service Bus Layer

Cresco provides communication paths between hosts, including multi-hop messaging throughout a distributed infrastructure. These communications provide both framework control messages and application-level data transport.

Multi-agent, multi-platform, and multi-protocol system.

Semantic Layer

Cresco provides methods to encode data streams and event with semantic meaning. For example, the ability to query all events of a specific type or payload of a specific type across the data layer. These subscriptions and events can be chained together to form very complex computational networks.

Multi-agent, multi-platform, and multi-protocol system.

Application Layer

Application stacks might make use of Cresco for edge functions, or the entire application might be build on the platform. In some cases Cresco might be used to manage or monitor existing applications. Custom plugins can be developed to expand Cresco functions, or existing plugins can be used to monitor the application stack, launch containers or applications, or interface with higher-level APIs and control systems.

Low-level visibility, communications, and processing across distributed nodes.

About

Origin

The needs that drove this project started back in 2012 when one of the project founders was working on an early Software Defined Networking (SDN) grant funded by the National Science Foundation (NSF). The problem at that time was that we needed ways to compute on, correlate, and generate stream of events from distributed data sources, where the data generation was too rapid to centrally transmit or store. Through subsequent local infrastructure, national testbed monitoring, international research network exchange measurement, and internal projects, components were developed along the way that eventually coalesced to form Cresco.

Contacts

Cresco is currently maintained by Cody Bumgardner's research lab.

If you use Cresco for a paper please cite:

@inproceedings{bumgardner2016cresco,
title={Cresco: A distributed agent-based edge computing framework},
author={Bumgardner, VK Cody and Marek, Victor W and Hickey, Caylin D},
booktitle={2016 12th International Conference on Network and Service Management (CNSM)},
pages={400--405},year={2016},organization={IEEE}}

Why the name Cresco?

  • Proto-Indo-European *ker ("to grow")
  • Old Armenian սերիմ (serim, "be born") and սերեմ (serem, "bring forth")
  • Latin creare ("become visible, multiply, augment")

    • Papers and Media

    • Cresco Handout
    • Constellation: A secure self-optimizing framework for genomic processing. "e-Health Networking, Applications and Services (Healthcom), 2016 IEEE 18th International Conference on. IEEE, 2016.
    • "An Edge-Focused Model for Distributed Streaming Data Applications", International Workshop on Pervasive Flow of Things (PerFoT'18). IEEE, 2018
    • Edge Computing in Smart Cities. Poster session presented at the 2017 Smart Cities Connect Conference and Expo, Austin, TX, USA, June 2017
    • "Edge-enabled Distributed Network Measurement", Second International Workshop on Smart Edge Computing and Networking (SmartEdge'18). IEEE, 2018
    • Edge Computing in Defense Management. Poster session presented at the 2017 Defense Innovation Challenges, Tampa, FL, USA, October 2017.
    • Edge framework for resource-aware distributed applications. Poster session presented at the 2018 Defense TechConnect Fall Summit, Tampa, FL, USA, October 2018.
    • "Cresco: A distributed agent-based edge computing framework." Network and Service Management (CNSM), 2016 12th International Conference on. IEEE, 2016.
    • "Edge Framework for a Resource-aware Distributed Applications". Poster session presented TNC18 (Geant), Trondheim, Norway, June 2018.