The MarketDesk cloud infrastructure is a distributed multi-pods architecture, with each pod following a multi-tiers design. The multi-pods architecture allows for a global distribution following regional and allows full redundancy across the globe.

Each pod is currently run in the AWS cloud but could be run in any facility.

Multi-pods Architecture

  • Pods can be deployed in various regions to provision for full redundancy.
  • Regional pods are kept in communication with other region’s pod for global data distribution.
  • A single pod can also be deployed across different physical locations to remove single point of failure.
  • Regional content remains in the region’s pod(s) so there is no latency distribution cost for that region.
  • Intra-pods communication is limited to only the real-time content being requested region to region (cross ocean bandwidth optimization).

Pod Multi-tiers Architecture

A pod consists of 5 tiers implementing the system’s functionality.

Content Acquisition Tier

The content acquisition tier implements all the data input facilities.

  • Feed handlers (FIX, MKF, JSON, and XML).
  • FTP servers.
  • Contribution servers (JSON over SSL or secure web socket connections).
  • Excel RTD handlers (direct contribution via excel RTD).

Middleware Tier

MarketDesk developed its proprietary messaging middleware for internal usage. This middleware is optimized for real time, high velocity, fully redundant to allow the pod’s services intra communication.

Here is some but not all major functionality:

  • Physical location abstraction.
  • Fully redundancy and self-healing. No single point of failure.
  • Self-describing messaging: content agnostic.
  • Native master/multiple slaves design for crucial service.
  • Native real time load balancing across multi-sources.
  • Secure communication.
  • Data access auditing.

Content Distribution Tier

The content distribution tier implements all facilities to massively distribute market data to many remote end points. We utilize horizontal scaling so in a modern cloud, the system scalability is infinite.

  • Real time feed distribution servers (SSL or WSS JSON, XML, FIX, and MKF format).
  • Web-socket servers.
  • Excel RTD servers.
  • WEB REST API servers.
  • FTP servers.

Web Application Tier

This tier has all the tools to support our website and web application.

  • HTTPS servers / load balancers.
  • Mobile servers for IOS and Android apps.

Administration Tier

This implements all the administration services needed by all other tiers.

  • Databases
  • Permission servers
  • Application file servers and network file server
  • Active monitoring systems
  • Reporting tools and email alerts

Technology Stack

Backend Systems:

All MarketDesk code has been developed and is maintained in New York.

  • Linux based system on multicores servers.
  • C++ multithreaded design.
  • Proprietary real time distributed system.

Web systems:

MarketDesk is a major supporter of open source software. Our core web/mobile application stack is powered by:

  • Apache (Web server)
  • Rivet (Web Application Framework)
  • HTML5 (WebSocket, CSS, AngularJS, React)
  • Swift (iOS)
  • Apache Cordova (Hybrid app container)

API

MarketDesk offers a range of protocols to contribute and access data.

  • Contribution wire protocol: SSL / JSON messaging protocol.
  • Feed wire protocol: SSL and WSS connection with FIX, XML, MKF, JSON messaging protocol.
  • WEB URL API: HTTPS connection, CSV, XML and JSON data format, Python wrapper.

Cloud Platform

MarketDesk was built on Amazon Web Service leveraging Cloud native tools that provides Scalability, Security and BCDR.  Our main mission is to easily scale and deploy our services in a secured manner to our customers across the Globe. Using some of the AWS services helps us just do that.

  • AWS Cloud Security Hub
  • AWS Guard Duty, Inspector, IAM Access Analyzer
  • Identity and access management, detective controls, infrastructure protection, data protection, compliance
  • ELB, EC2, ELB, EFS, S3, Route 53, Cloud Watch, RDS