NexTech 2021 Congress
October 03, 2021 to October 07, 2021 - Barcelona, Spain

  • UBICOMM 2021, The Fifteenth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies
  • ADVCOMP 2021, The Fifteenth International Conference on Advanced Engineering Computing and Applications in Sciences
  • SEMAPRO 2021, The Fifteenth International Conference on Advances in Semantic Processing
  • AMBIENT 2021, The Eleventh International Conference on Ambient Computing, Applications, Services and Technologies
  • EMERGING 2021, The Thirteenth International Conference on Emerging Networks and Systems Intelligence
  • DATA ANALYTICS 2021, The Tenth International Conference on Data Analytics
  • GLOBAL HEALTH 2021, The Tenth International Conference on Global Health Challenges
  • CYBER 2021, The Sixth International Conference on Cyber-Technologies and Cyber-Systems

SoftNet 2021 Congress
October 03, 2021 to October 07, 2021 - Barcelona, Spain

  • ICSEA 2021, The Sixteenth International Conference on Software Engineering Advances
  • ICSNC 2021, The Sixteenth International Conference on Systems and Networks Communications
  • CENTRIC 2021, The Fourteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services
  • VALID 2021, The Thirteenth International Conference on Advances in System Testing and Validation Lifecycle
  • SIMUL 2021, The Thirteenth International Conference on Advances in System Simulation
  • SOTICS 2021, The Eleventh International Conference on Social Media Technologies, Communication, and Informatics
  • INNOV 2021, The Tenth International Conference on Communications, Computation, Networks and Technologies
  • HEALTHINFO 2021, The Sixth International Conference on Informatics and Assistive Technologies for Health-Care, Medical Support and Wellbeing

NetWare 2021 Congress
November 14, 2021 to November 18, 2021 - Athens, Greece

  • SENSORCOMM 2021, The Fifteenth International Conference on Sensor Technologies and Applications
  • SENSORDEVICES 2021, The Twelfth International Conference on Sensor Device Technologies and Applications
  • SECURWARE 2021, The Fifteenth International Conference on Emerging Security Information, Systems and Technologies
  • AFIN 2021, The Thirteenth International Conference on Advances in Future Internet
  • CENICS 2021, The Fourteenth International Conference on Advances in Circuits, Electronics and Micro-electronics
  • ICQNM 2021, The Fifteenth International Conference on Quantum, Nano/Bio, and Micro Technologies
  • FASSI 2021, The Seventh International Conference on Fundamentals and Advances in Software Systems Integration
  • GREEN 2021, The Sixth International Conference on Green Communications, Computing and Technologies

TrendNews 2021 Congress
November 14, 2021 to November 18, 2021 - Athens, Greece

  • CORETA 2021, Advances on Core Technologies and Applications
  • DIGITAL 2021, Advances on Societal Digital Transformation

 


ThinkMind // International Journal On Advances in Internet Technology, volume 11, numbers 3 and 4, 2018 // View article inttech_v11_n34_2018_2


DXNet: Scalable Messaging for Multi-Threaded Java-Applications Processing Big Data in Clouds

Authors:
Kevin Beineke
Stefan Nothaas
Michael Schoettner

Keywords: Message passing; Ethernet networks; InfiniBand; Java; Cloud computing.

Abstract:
Many big data and large-scale cloud applications are written in Java or are built using Java-based frameworks. Typically, application instances are running in a data center on many virtual machines which requires scalable and efficient network communication. In this paper, we present the practical experience of designing an extensible Java network subsystem, called DXNet, providing fast object de-/serialization, automatic connection management and zero-copy messaging. Additionally, we present a Java.nio-based transport for DXNet, called EthDXNet, to efficiently utilize Ethernet networks. The proposed design uses a zero-copy send and receive approach for asynchronous messages and requests/responses. DXNet is optimized for small messages (< 100 bytes) in order to support graph-based applications, but also works well with larger messages (e.g., 8 MB). DXNet is available on GitHub, and its modular design is open for different transport implementations currently supporting Ethernet and InfiniBand. EthDXNet is based on Java.nio socket channels complemented by application-level flow control to achieve low latency and high throughput for 10 GBit/s and faster Ethernet networks. Furthermore, a scalable automatic connection management and a low-overhead interest handling provide efficient network communication for dozens of servers, even for small messages (< 100 bytes) and an all-to-all communication pattern. The evaluation with micro-benchmarks and the Yahoo! Cloud Serving Benchmark (YCSB) shows the efficiency and scalability with up to 64 virtual machines in the Microsoft Azure cloud. Furthermore, DXNet achieves request-response latencies sub 10 µs (round trip) including object de-/serialization, as well as a maximum throughput of more than 9 GByte/s on a private cluster (using InfiniBand).

Pages: 117 to 135

Copyright: Copyright (c) to authors, 2018. Used with permission.

Publication date: December 30, 2018

Published in: journal

ISSN: 1942-2652

SERVICES CONTACT
2010 - 2017 © ThinkMind. All rights reserved.
Read Terms of Service and Privacy Policy.