Archived-DMM
| DMM Facts |
|
Project Lead: George Zhao
Repository: git clone https://gerrit.fd.io/r/dmm |
What is DMM
DMM (Dual Mode, Multi-protocol, Multi-instance) is to implement a transport agnostic framework for network applications that can
- Work with both user space and kernel space network stacks
- Use different network protocol stacks based on their functional and performance requirements (QOS)
- Work with multiple instances of a transport protocol stack
Use and engage or adopt a new protocol stack dynamically as applicable.
History
Emerging applications are bringing extremely high-performance requirements to their network systems. Eg. AR/VR, IoT etc. Many of these applications also have unique demands for QOS/SLA. Some application need a low latency network, some need high reliability, etc. Though such performance targets should be required for the complete communication system, the transport layer protocols play a key role and encounter a relatively higher challenge, because traditionally the TCP-based transport layer exploits the “best-effort” principle and inherently provides no performance guarantees. However, as internet applications rapidly grow and diversify, an all-powerful or one-size-fits-all protocol or algorithm becomes less feasible. Thus the traditional single-instance TCP-based network stack faces many challenges when serving applications with different QoS/SLA requirements simultaneously on the same platform.
Moving the networking stack out of the kernel is an occurring trend in both industry implementations and academic literature. Technologies like DPDK (and others) are improving performance of the network stack, bypassing the kernel and avoiding context-switching and data copies, as well as providing a complete set of packet-processing acceleration libraries.
Keeping the above trends in mind the DMM/nStack provides a framework where, system operators can plug in dedicated types of networking stack instances according to performance and/or functional requirements from the user space applications. The application need not consider changes to the transport layer API. A lightweight nStack management daemon is responsible for maintaining the stack instances and the app/socket-to-stack-mappings, which are provided via the orchestration/management interface.
DMM Architecture
The DMM framework provides posix socket APIs to the application. A protocol stack could be plugged into the DMM. DMM will choose the most suitable stack according to the RD policy.
Quick Start
With this quick start, users can have a simple understanding of DMM.
Following the quick start, it will set environment, build DMM and test apps automatically on Vagrant.
The test topology will typically look like this. It uses DMM + kernel stack now.
Make sure vagrant and virtual box have been installed first.
1. Download dmm code
# git clone https://gerrit.fd.io/r/dmm
2. Setup Vagrant VMs
# cd dmm/resources/extras
If users want to change the DISTRO, NICS, VMCPU or VMRAM settings, modified the env.sh and run source env.sh.
# vagrant up
This steps will take a while. Until vagrant automatically set the environment and compile dpdk, glog and DMM.
3. Log in to server/client in two different terminals.
# vagrant ssh dmm-test-server
and
# vagrant ssh dmm-test-client
4. There will be two configuration files in /dmm/config/app-test.
module_config.json file is the stack module configuration file to use different protocol stacks (Currently only kernel stack supported, later we will support other stacks).
{
"default_stack_name": "kernel", /*when rd can't be find maybe choose the defualt one*/
"module_list": [
{
"stack_name": "kernel", /*stack name*/
"function_name": "kernel_stack_register", /*function name*/
"libname": "./", /*library name, if loadtype is static, this maybe
null, else must give a library name*/
"loadtype": "static", /*library load type: static or dynamic*/
"deploytype": "1", /*deploy model type:model type1, model type2,
model type3. Indicating single or multi process
deployment. Used during shared memory initialization.*/
"maxfd": "1024", /*the max fd supported*/
"minfd": "0", /*the min fd supported*/
"priorty": "1", /*priorty when executing, reserv*/
"stackid": "0", /*stack id, this must be ordered and not be repeated*/
},
}
rd_config.json file describes rd policies. DMM provides dynamic mapping between apps (connections/sockets) and candidate networking stack instances. And this flexibility is achieved by the rd. nStack provides dynamic mapping between apps (connections/sockets) and candidate networking stack instances. And this flexibility is achieved by the rd. You can modify these files if required.
{ "ip_route": [
{
"subnet": "192.165.1.1/16",
"type": "nstack-kernel",
/* output interface type,
nstack-kernel: indicate this ip should go through kernel protocol
nstack-dpdk: go through stackx protocol*/
},
{
"subnet": "172.16.25.125/16",
"type": "nstack-kernel",
}
],
"prot_route": [
{
"proto_type": "11",
"type": "nstack-kernel",
},
]
}
To know more about RD check in DMM_DeveloperManual.md [find this document in DMM repository https://gerrit.fd.io/r/dmm]
Before testing apps, users can update configuration files as needed.
5.Users can test sample client/server app in /dmm/release/bin.
Copy two configuration files from /dmm/config/app-test to /dmm/release/bin
After building the DMM, sample test apps will be generated:
kc_common, ks_epoll, ks_select, vc_common, vs_epoll, vs_select.
The use of ks_epoll,ks_select,vs_epoll and vs_select are the same.
Examples:
With Kernel stack(Without DMM):
server:
#./ks_epoll -p 20000 -d 172.16.25.125 -a 10000 -s 172.16.25.126 -l 200 -t 5000000 -i 0 -f 1 -r 20000 -n 1 -w 10 -u 10000 -e 10 -x 1
client:
#./kc_common -p 20000 -d 172.16.25.126 -a 10000 -s 172.16.25.125 -l 200 -t 5000000 -i 0 -f 1 -r 20000 -n 1 -w 10 -u 10000 -e 10 -x 1
With DMM nStack(Stack is choosed by congfigure file):
Note:
Currently we only support nstack
server:
#./vs_epoll -p 20000 -d 172.16.25.125 -a 10000 -s 172.16.25.126 -l 200 -t 5000000 -i 0 -f 1 -r 20000 -n 1 -w 10 -u 10000 -e 10 -x 1
client:
#./vc_common -p 20000 -d 172.16.25.126 -a 10000 -s 172.16.25.125 -l 200 -t 5000000 -i 0 -f 1 -r 20000 -n 1 -w 10 -u 10000 -e 10 -x 1
If users want to test their own apps, copy program executable in app-test directory. Run your program using LD_PRELOAD from app-test directory
# sudo LD_PRELOAD=../../release/lib64/libnStackAPI.so ./your_app
6. Post testing cleanup. After completing testing, issue the following command to clean up the vagrant VM's:
# vagrant destroy -f
Get Involved
- Bi-Weekly DMM Meeting.
- Join the DMM Mailing List.
- Join fdio-dmm IRC channel.
- Browse the code.
- 18.04 Release Plan
Reference
If you are using Chrome, you can install extension "Markdown Reader" to read the MD file.
[1] Enabling “Protocol Routing”: Revisiting Transport Layer Protocol Design in Internet Communications, By Dr. Kai Zheng November 2017
[2] Empower Diverse Open Transport Layer Protocol in the Cloud Networking, By Qing Chang At FD.io ONS2018 mini summit
[3] Video presentation, By Qing Chang At FD.io Mini-Summit at KubeCon + CloudNativeCon EU 2018
[4] DMM Developer Manual
[5] How to build DMM
[6] How to test APP linked with DMM
