Difference between revisions of "VPP/Python API"

From fd.io
< VPP
Jump to: navigation, search
m (Step by Step (Ubuntu 14.04))
m (API definition jsons are now separated by core and plugin, appending filename to base path /usr/share/vpp/api/core/ does not yield the correct path.)
 
(8 intermediate revisions by 4 users not shown)
Line 16: Line 16:
  
 
== Installation ==
 
== Installation ==
The main VPP build will build the C library (libpneum.so). If VPP is installed via the Linux package system that library will be available on the system. To run within a build directory, set LD_LIBRARY_PATH to point to the location of libpneum.so.
+
The main VPP build will build the C library (<code>libvppapiclient.so</code>). If VPP is installed via the Linux package system that library will be available on the system. To run within a build directory, set LD_LIBRARY_PATH to point to the location of <code>libvppapiclient.so</code>.
  
 
To build the VPP_PAPI Python package (and shared library):
 
To build the VPP_PAPI Python package (and shared library):
Line 39: Line 39:
 
Type "help", "copyright", "credits" or "license" for more information.
 
Type "help", "copyright", "credits" or "license" for more information.
 
>>> import vpp_papi
 
>>> import vpp_papi
/usr/local/lib/python2.7/dist-packages/vpp_papi-1.3-py2.7-linux-x86_64.egg/vpp_papi/__init__.py:1: UserWarning: Module vpp_papi was already imported from /usr/local/lib/python2.7/dist-packages/vpp_papi-1.3-py2.7-linux-x86_64.egg/vpp_papi/__init__.pyc, but /home/alagalah/git/work/fdio/vpp is being added to sys.path
 
 
>>>  
 
>>>  
 
  </nowiki>
 
  </nowiki>
  
If you run from the development directory in a virtualenv environment, you might have to set LD_LIBRARY_PATH to e.g. build-root/install-vpp_debug-native/vpp-api/lib64/
+
If you run from the development directory in a virtualenv environment, you might have to set LD_LIBRARY_PATH to e.g. <code>build-root/install-vpp_debug-native/vpp-api/lib64/</code>
  
The Python package can also be installed from the vpp-python-api RPM. It is then installed in the default Python library directory.
+
The Python package can also be installed from the vpp-python-api RPM/DEB. It is then installed in the system default Python library directory.
  
== Step by Step (Ubuntu 14.04) ==
+
== Step by Step ==
 +
<syntaxhighlight lang="bash" line='line'>
 +
# install preresquisites
 +
sudo apt-get install python-virtualenv
  
These steps assume you have a checked out VPP tree at ~/vpp and that you have successfully got to the stage of "make build" succeeding
+
export VPP=~vpp/
 +
cd $VPP
  
They also assume you are logged in as a user "ubuntu" with the home directory "/home/ubuntu".
+
# build vpp
 +
make bootstrap build
  
First install the virtualenv package:
+
# create virtualenv
 +
virtualenv virtualenv
  
  '''sudo apt-get install python-virtualenv'''
+
# (optional) install python packages
 +
# ipaddress is used by some scripts
 +
virtualenv/bin/pip install ipaddress
 +
# nice to have to get the tab completion and other CLI niceties
 +
virtualenv/bin/pip install scapy
  
Now change to the VPP directory and create the virtualenv there:
+
# install vpp python api
 +
pushd $VPP/src/vpp-api/python/
 +
$VPP/virtualenv/bin/python setup.py install
 +
popd
  
  '''cd ~/vpp'''
+
# Now set the LD_LIBRARY_PATH such that it points to the directory containing libvppapiclient.so
  '''virtualenv virtualenv'''
+
export LD_LIBRARY_PATH=`find $VPP -name "libvppapiclient.so" -exec dirname {} \; | grep install-vpp | head -n 1`
  
Now make the API:
+
# You will now need two windows :
 +
# one for vpp, and the other for python
  
  '''make build-vpp-api'''
+
# VPP
 +
cd $VPP
 +
make run
  
Install the ipaddress package (used by some of the scripts)
+
# python
 +
# (as root, as vpp.connect() requires root privileges)
 +
# Note that sudo cannot not preserve LD_LIBRARY_PATH
 +
cd $VPP
  
  '''virtualenv/bin/pip install ipaddress'''
+
# you can run a script
 +
sudo -E LD_LIBRARY_PATH=$LD_LIBRARY_PATH $VPP/virtualenv/bin/python vpp-api/python/tests/vpp_hello_world.py.py
  
Now install the python API into the virtualenv
+
# or get a python prompt
 +
sudo -E LD_LIBRARY_PATH=$LD_LIBRARY_PATH $VPP/virtualenv/bin/python
 +
</syntaxhighlight>
  
  '''pushd src/vpp-api/python/'''
+
VPP python's hello world
  '''~/vpp/virtualenv/bin/python setup.py install'''
+
<syntaxhighlight lang="python" line='line'>
  '''popd'''
+
#!/bin/env python
 +
'''
 +
vpp_hello_world.py
 +
'''
 +
from __future__ import print_function
  
Now set the LD_LIBRARY_PATH such that it points to the directory containing libpneum.so:
+
import os
 +
import fnmatch
  
  '''export LD_LIBRARY_PATH=/home/ubuntu/vpp/build-root/install-vpp_debug-native/vpp-api/lib64/'''
+
from vpp_papi import VPP
  
Or, in a more automatic fashion:
+
# first, construct a vpp instance from vpp json api files
 +
# this will be a header for all python vpp scripts
  
  '''export LD_LIBRARY_PATH=`pwd`/`find . -name "libpneum.so" -exec dirname {} \; | grep lib64 | head -n 1`'''
+
# directory containing all the json api files.
 +
# if vpp is installed on the system, these will be in /usr/share/vpp/api/
 +
vpp_json_dir = os.environ['VPP'] + '/build-root/install-vpp_debug-native/vpp/share/vpp/api/core'
  
 +
# construct a list of all the json api files
 +
jsonfiles = []
 +
for root, dirnames, filenames in os.walk(vpp_json_dir):
 +
    for filename in fnmatch.filter(filenames, '*.api.json'):
 +
        jsonfiles.append(os.path.join(vpp_json_dir, filename))
  
 +
if not jsonfiles:
 +
    print('Error: no json api files found')
 +
    exit(-1)
  
In another window, edit a file ~/vpp/startup.conf such that it looks like follows:
+
# use all those files to create vpp.
 +
# Note that there will be no vpp method available before vpp.connect()
 +
vpp = VPP(jsonfiles)
 +
r = vpp.connect('papi-example')
 +
print(r)
 +
# None
  
  unix { interactive exec /home/ubuntu/vpp/vpp.cmd cli-listen 127.0.0.1:5002 }  api-segment { uid ubuntu gid ubuntu }
+
# You're all set.
  dpdk {
+
# You can check the list of available methods by calling dir(vpp)
    no-pci
+
    vdev eth_af_packet0,iface=eth0
+
  }
+
  
The critical part above is "api-segment" part which configures the uid/gid so that your API client did not need the sudo privileges.
+
# show vpp version
 +
rv = vpp.api.show_version()
 +
print('VPP version =', rv.version.decode().rstrip('\0x00'))
 +
# VPP version = 17.04-rc0~192-gc5fccc0c
  
Make sure to export the environment variable with the path:
+
# disconnect from vpp
 +
r = vpp.disconnect()
 +
print(r)
 +
# 0
  
  '''export STARTUP_CONF=startup.conf'''
+
exit(r)
 
+
</syntaxhighlight>
Now touch the ~/vpp/vpp.cmd, later on you can put some VPP startup commands there
+
 
+
  '''touch /home/ubuntu/vpp/vpp.cmd'''
+
 
+
now in this window you should be able to issue "make run" and get to a running instance of VPP.
+
 
+
  '''make run'''
+
 
+
 
+
Go back the "python window"
+
 
+
Install scapy
+
 
+
  '''virtualenv/bin/pip install scapy'''
+
 
+
Launch scapy to get the tab completion and other CLI niceties:
+
 
+
  '''virtualenv/bin/scapy'''
+
 
+
The next actions are inside the scapy prompt:
+
 
+
  >>> '''import vpp_papi'''
+
 
+
Connect to VPP
+
 
+
  >>> '''vpp_papi.connect("foo")'''
+
  0L
+
  >>>
+
 
+
Issue a "show version" request
+
 
+
  >>> '''vpp_papi.show_version()'''
+
  show_version_reply(vl_msg_id=164, context=1, retval=0, program='vpe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', version='16.12-rc0~208-g5899fde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', build_date='Wed Oct 12 19:49:12 UTC 2016\x00\x00\x00\x00', build_directory='/home/ubuntu/vpp\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
+
  >>>
+
 
+
 
+
Disconnect from VPP
+
 
+
  >>> '''vpp_papi.disconnect()'''
+
  0L
+
  >>>
+
 
+
NB: currently, if you forget to connect() first, the python prompt will segfault. So, don't forget to connect! :)
+
 
+
 
+
Now you can run a script using Python API:
+
 
+
  ubuntu@vpp-tdd:~/vpp$ '''virtualenv/bin/python vpp-api/python/tests/test_papi.py'''
+
  13
+
  VPP BIN: /home/ubuntu/vpp/vpp-api/python/tests/../../../build-root/install-vpp_debug-native/vpp/bin/vpp
+
  Started VPP
+
  Connecting API
+
  Summary stats vnet_summary_stats_reply(vl_msg_id=55, context=1, retval=0, total_pkts=(0, 0), total_bytes=(0, 0), vector_rate=0.0)
+
  Packets: 0
+
  Packets: 0
+
  .Connecting API
+
  Dump/details T [sw_interface_details(vl_msg_id=12, context=2, sw_if_index=0, sup_sw_if_index=0, l2_address_length=0, l2_address='\x00\x00\x00\x00\x00\x00\x00\x00',    interface_name='local0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', admin_up_down=0, link_up_down=0, link_duplex=0, link_speed=0, link_mtu=0, sub_id=0, sub_dot1ad=0, sub_number_of_tags=0, sub_outer_vlan_id=0, sub_inner_vlan_id=0, sub_exact_match=0, sub_default=0, sub_outer_vlan_id_any=0, sub_inner_vlan_id_any=0, vtr_op=0, vtr_push_dot1q=0, vtr_tag1=0, vtr_tag2=0), sw_interface_details(vl_msg_id=12, context=2, sw_if_index=1, sup_sw_if_index=1, l2_address_length=6, l2_address='\x02\xfe\x03\xc99d\x00\x00', interface_name='af_packet0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', admin_up_down=0, link_up_down=0, link_duplex=2, link_speed=8, link_mtu=9216, sub_id=0, sub_dot1ad=0, sub_number_of_tags=0, sub_outer_vlan_id=0, sub_inner_vlan_id=0, sub_exact_match=0, sub_default=0, sub_outer_vlan_id_any=0, sub_inner_vlan_id_any=0, vtr_op=0, vtr_push_dot1q=0, vtr_tag1=0, vtr_tag2=0)]
+
  .Connecting API
+
  T show_version_reply(vl_msg_id=164, context=3, retval=0, program='vpe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', version='16.12-rc0~208-g5899fde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', build_date='Wed Oct 12 19:49:12 UTC 2016\x00\x00\x00\x00', build_directory='/home/ubuntu/vpp\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
+
  .Connecting API
+
  create_loopback_reply(vl_msg_id=80, context=4, retval=0, sw_if_index=2)
+
  sw_interface_add_del_address_reply(vl_msg_id=17, context=5, retval=0)
+
  want_stats_reply(vl_msg_id=50, context=7, retval=0)
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=0, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=1, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=2, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=3, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=4, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=5, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=6, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=7, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=8, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=0, is_combined=1, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=1, is_combined=1, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=0, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=1, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=2, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=3, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=4, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=5, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=6, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=7, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=8, is_combined=0, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=0, is_combined=1, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  Interface counters vnet_interface_counters(vl_msg_id=51, vnet_counter_type=1, is_combined=1, first_sw_if_index=0, count=3, data=('\x00\x00\x00',))
+
  IPv6 FIB counters vnet_ip6_fib_counters(vl_msg_id=53, vrf_id=0, count=2, c=[(18375249429625044992L, 1, 128, 1, 72), (18375249429625044992L, 22, 128, 1, 156)])
+
  want_stats_reply(vl_msg_id=50, context=8, retval=0)
+
  .
+
  ----------------------------------------------------------------------
+
  Ran 4 tests in 11.067s
+
  OK
+
  ubuntu@vpp-tdd:~/vpp$
+
  
 
== Implementation ==
 
== Implementation ==
  
 +
<b>Note: This is out of date and needs updating; <code>pneum</code> has been replaced by <code>libvppapiclient</code>; the semantics are otherwise mostly the same.</b>
  
 
The vpp/api/vpe.api file specifies a set of messages that can be exchanged between VPP and the API client. The semantics of those messages are somewhat up to interpretation and convention.
 
The vpp/api/vpe.api file specifies a set of messages that can be exchanged between VPP and the API client. The semantics of those messages are somewhat up to interpretation and convention.
  
 
The language binding is implemented simply by exposing four C calls to Python. Those are:
 
The language binding is implemented simply by exposing four C calls to Python. Those are:
 +
<syntaxhighlight lang="C">
 
   int pneum_connect(char *name);
 
   int pneum_connect(char *name);
 
   int pneum_disconnect(void);
 
   int pneum_disconnect(void);
 
   int pneum_read(char **data, int *l);
 
   int pneum_read(char **data, int *l);
 
   int pneum_write(char *data, int len);
 
   int pneum_write(char *data, int len);
 +
</syntaxhighlight>
  
 
In addition there is a Python message handler callback called by the C RX pthread. All message handling and parsing is done in Python.
 
In addition there is a Python message handler callback called by the C RX pthread. All message handling and parsing is done in Python.
Line 213: Line 169:
  
 
[[File:Screen Shot 2016-09-21 at 12.02.05.png||300px|VPP PAPI Packaging]]
 
[[File:Screen Shot 2016-09-21 at 12.02.05.png||300px|VPP PAPI Packaging]]
 +
== Examples ==
  
 
=== Example: Dumping interface table ===
 
=== Example: Dumping interface table ===
The example is in "vpp-api-client/python/examply.py" and there are unit tests in "vpp-api-client/python/test_papi.py".
 
  
<nowiki>
+
<syntaxhighlight lang="python" line='line'>
 
#!/usr/bin/env python
 
#!/usr/bin/env python
  
import vpp_papi
+
from __future__ import print_function
  
r = vpp_papi.connect("test_papi")
+
import os
 +
import fnmatch
  
t = vpp_papi.show_version()
+
from vpp_papi import VPP  
print('VPP version:', t.version.decode())
+
  
t = vpp_papi.sw_interface_dump(0, b'ignored')
 
  
if t:
+
vpp_json_dir = '/usr/share/vpp/api/core/'
    print('List of interfaces')
+
    for interface in t:
+
        if interface.vl_msg_id == vpp_papi.vpe.VL_API_SW_INTERFACE_DETAILS:
+
            print(interface.interface_name.decode())
+
r = vpp_papi.disconnect()
+
</nowiki>
+
  
== Examples ==
+
jsonfiles = []
=== Example: Receive statistics ===
+
for root, dirnames, filenames in os.walk(vpp_json_dir):
<nowiki>
+
    for filename in fnmatch.filter(filenames, '*.api.json'):
 +
        jsonfiles.append(os.path.join(vpp_json_dir, filename))
  
#!/usr/bin/env python
+
if not jsonfiles:
 +
    print('Error: no json api files found')
 +
    exit(-1)
  
import struct
+
vpp = VPP(jsonfiles)
import time
+
  
import vpp_papi
+
r = vpp.connect("test_papi")
 +
print(r)
  
def papi_event_handler(result):
+
for intf in vpp.api.sw_interface_dump():
     if result.vl_msg_id == vpp_papi.VL_API_VNET_INTERFACE_COUNTERS:
+
     print(intf.interface_name.decode())
        format = '>' + str(int(len(result.data) / 8)) + 'Q'
+
        counters = struct.unpack(format, result.data)
+
        print('Counters:', counters)
+
        return
+
  
    print('Unknown message id:', result.vl_msg_id)
+
exit(vpp.disconnect())
 +
</syntaxhighlight>
  
r = vpp_papi.connect("test_papi")
+
=== Example: Receive statistics ===
vpp_papi.register_event_callback(papi_event_handler)
+
r = vpp_papi.want_stats(True, 123)
+
  
#
+
<syntaxhighlight lang="python" line='line'>
# Wait for some stats
+
#!/usr/bin/env python
#
+
 
 +
from vpp_papi import VPP
 +
import os                                                                                                                                                                                               
 +
import sys
 +
import fnmatch
 +
import time                                                                                                                                                                                             
 +
def papi_event_handler(msgname, result):
 +
  print(msgname)                                                                                                                                                                                           
 +
  print(result)
 +
 
 +
vpp_json_dir = os.environ['VPP'] + '/build-root/install-vpp_debug-native/vpp/share/vpp/api/core'                 
 +
                                                                                     
 +
# construct a list of all the json api files
 +
jsonfiles = []                                                                                                                                                                                           
 +
for root, dirnames, filenames in os.walk(vpp_json_dir):
 +
    for filename in fnmatch.filter(filenames, '*.api.json'):
 +
        jsonfiles.append(os.path.join(vpp_json_dir, filename))                                                                                                                                           
 +
if not jsonfiles:
 +
    print('Error: no json api files found')
 +
    exit(-1)                                                                                                                                                                                             
 +
# use all those files to create vpp.
 +
vpp = VPP(jsonfiles)
 +
r = vpp.connect("test_papi")                                                                                                                                                                             
 +
print(r)
 +
 
 +
async=True                                                                                                                                                                                               
 +
r=vpp.register_event_callback(papi_event_handler)
 +
pid=os.getpid()
 +
sw_ifs = [1,2,3,4,5,6,7]                                                                                                                                                                                 
 +
r = vpp.api.want_per_interface_simple_stats(enable_disable=True, sw_ifs=sw_ifs, num=len(sw_ifs), pid=pid)
 +
print(r)
 +
# Wait for some stats                                                                                                                                                                                  
 
time.sleep(60)
 
time.sleep(60)
r = vpp_papi.want_stats(False, pid)
+
r = vpp.api.want_per_interface_simple_stats(enable_disable=False)
r = vpp_papi.disconnect()
+
r = vpp.disconnect()
  
</nowiki>
+
</syntaxhighlight>
  
 
== API generation ==
 
== API generation ==

Latest revision as of 15:25, 18 April 2019

Python binding for the VPP API

The vpp-papi module in vpp-api/python/ provides a Python binding to the VPP API. The Python bindings to the API is auto-generated from JSON API definitions. These JSON definitions must be passed to the VPP class init method. Both individual components and plugins provide API definitions. The JSON files are also generated, from .api files. In a binary installation the JSON API definitions are installed under /usr/share/vpp/api/

Currently there are three classes of VPP API methods:

  1. Simple request / reply. For example the show_version() call the SHOW_VERSION message is the request and the SHOW_VERSION_REPLY is the answer back. By convention replies are named ending with _REPLY.
  2. Dump functions. For example sw_interface_dump() send the SW_INTERFACE_DUMP message and receive a set of messages back. In this example SW_INTERFACE_DETAILS and SW_INTERFACE_SET_FLAGS are (typically) received. The CONTROL_PING/CONTROL_PING_REPLY is used as a method to signal to the client that the last message has been received. By convention the request message have names ending with _DUMP and the replies have names ending in _DETAILS.
  3. Register for events. For example want_stats() sends a WANT_STATS message, get a WANT_STATS_REPLY message back, and the client will then asynchronously receive VNET_INTERFACE_COUNTERS messages.

The API is by default blocking although there is possible to get asynchronous behaviour by setting the function argument async=True.

Each call uses the arguments as specified in the API definitions file (e.g. vpe.api). The "client_index" and "context" fields are handled by the module itself. A call returns a named tuple or a list of named tuples.

Installation

The main VPP build will build the C library (libvppapiclient.so). If VPP is installed via the Linux package system that library will be available on the system. To run within a build directory, set LD_LIBRARY_PATH to point to the location of libvppapiclient.so.

To build the VPP_PAPI Python package (and shared library): This step maybe unnecessary if you have already done it (generally one time)

 
make build
 

The Python package can either be installed in the Python system directory or in a Virtualenv environment. To install the Python component:

cd src/vpp-api/python
sudo python setup.py install
 

To test:

alagalah@thing1:vpp (master)*$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import vpp_papi
>>> 
 

If you run from the development directory in a virtualenv environment, you might have to set LD_LIBRARY_PATH to e.g. build-root/install-vpp_debug-native/vpp-api/lib64/

The Python package can also be installed from the vpp-python-api RPM/DEB. It is then installed in the system default Python library directory.

Step by Step

  1. # install preresquisites
  2. sudo apt-get install python-virtualenv
  3.  
  4. export VPP=~vpp/
  5. cd $VPP
  6.  
  7. # build vpp
  8. make bootstrap build
  9.  
  10. # create virtualenv
  11. virtualenv virtualenv
  12.  
  13. # (optional) install python packages
  14. # ipaddress is used by some scripts
  15. virtualenv/bin/pip install ipaddress
  16. # nice to have to get the tab completion and other CLI niceties
  17. virtualenv/bin/pip install scapy
  18.  
  19. # install vpp python api
  20. pushd $VPP/src/vpp-api/python/
  21. $VPP/virtualenv/bin/python setup.py install
  22. popd
  23.  
  24. # Now set the LD_LIBRARY_PATH such that it points to the directory containing libvppapiclient.so
  25. export LD_LIBRARY_PATH=`find $VPP -name "libvppapiclient.so" -exec dirname {} \; | grep install-vpp | head -n 1`
  26.  
  27. # You will now need two windows :
  28. # one for vpp, and the other for python
  29.  
  30. # VPP
  31. cd $VPP
  32. make run
  33.  
  34. # python
  35. # (as root, as vpp.connect() requires root privileges)
  36. # Note that sudo cannot not preserve LD_LIBRARY_PATH
  37. cd $VPP
  38.  
  39. # you can run a script 
  40. sudo -E LD_LIBRARY_PATH=$LD_LIBRARY_PATH $VPP/virtualenv/bin/python vpp-api/python/tests/vpp_hello_world.py.py
  41.  
  42. # or get a python prompt
  43. sudo -E LD_LIBRARY_PATH=$LD_LIBRARY_PATH $VPP/virtualenv/bin/python

VPP python's hello world

  1. #!/bin/env python
  2. '''
  3. vpp_hello_world.py
  4. '''
  5. from __future__ import print_function
  6.  
  7. import os
  8. import fnmatch
  9.  
  10. from vpp_papi import VPP 
  11.  
  12. # first, construct a vpp instance from vpp json api files
  13. # this will be a header for all python vpp scripts
  14.  
  15. # directory containing all the json api files.
  16. # if vpp is installed on the system, these will be in /usr/share/vpp/api/
  17. vpp_json_dir = os.environ['VPP'] + '/build-root/install-vpp_debug-native/vpp/share/vpp/api/core'
  18.  
  19. # construct a list of all the json api files
  20. jsonfiles = []
  21. for root, dirnames, filenames in os.walk(vpp_json_dir):
  22.     for filename in fnmatch.filter(filenames, '*.api.json'):
  23.         jsonfiles.append(os.path.join(vpp_json_dir, filename))
  24.  
  25. if not jsonfiles:
  26.     print('Error: no json api files found')
  27.     exit(-1)
  28.  
  29. # use all those files to create vpp.
  30. # Note that there will be no vpp method available before vpp.connect()
  31. vpp = VPP(jsonfiles)
  32. r = vpp.connect('papi-example')
  33. print(r)
  34. # None
  35.  
  36. # You're all set.
  37. # You can check the list of available methods by calling dir(vpp)
  38.  
  39. # show vpp version
  40. rv = vpp.api.show_version()
  41. print('VPP version =', rv.version.decode().rstrip('\0x00'))
  42. # VPP version = 17.04-rc0~192-gc5fccc0c
  43.  
  44. # disconnect from vpp
  45. r = vpp.disconnect()
  46. print(r)
  47. # 0
  48.  
  49. exit(r)

Implementation

Note: This is out of date and needs updating; pneum has been replaced by libvppapiclient; the semantics are otherwise mostly the same.

The vpp/api/vpe.api file specifies a set of messages that can be exchanged between VPP and the API client. The semantics of those messages are somewhat up to interpretation and convention.

The language binding is implemented simply by exposing four C calls to Python. Those are:

  int pneum_connect(char *name);
  int pneum_disconnect(void);
  int pneum_read(char **data, int *l);
  int pneum_write(char *data, int len);

In addition there is a Python message handler callback called by the C RX pthread. All message handling and parsing is done in Python.

Architecture

Architecture.png

Packaging:

VPP PAPI Packaging

Examples

Example: Dumping interface table

  1. #!/usr/bin/env python
  2.  
  3. from __future__ import print_function
  4.  
  5. import os
  6. import fnmatch
  7.  
  8. from vpp_papi import VPP 
  9.  
  10.  
  11. vpp_json_dir = '/usr/share/vpp/api/core/'
  12.  
  13. jsonfiles = []
  14. for root, dirnames, filenames in os.walk(vpp_json_dir):
  15.     for filename in fnmatch.filter(filenames, '*.api.json'):
  16.         jsonfiles.append(os.path.join(vpp_json_dir, filename))
  17.  
  18. if not jsonfiles:
  19.     print('Error: no json api files found')
  20.     exit(-1)
  21.  
  22. vpp = VPP(jsonfiles)
  23.  
  24. r = vpp.connect("test_papi")
  25. print(r)
  26.  
  27. for intf in vpp.api.sw_interface_dump():
  28.     print(intf.interface_name.decode())
  29.  
  30. exit(vpp.disconnect())

Example: Receive statistics

  1. #!/usr/bin/env python
  2.  
  3. from vpp_papi import VPP
  4. import os                                                                                                                                                                                                 
  5. import sys
  6. import fnmatch
  7. import time                                                                                                                                                                                               
  8. def papi_event_handler(msgname, result):
  9.   print(msgname)                                                                                                                                                                                            
  10.   print(result)
  11.  
  12. vpp_json_dir = os.environ['VPP'] + '/build-root/install-vpp_debug-native/vpp/share/vpp/api/core'                   
  13.  
  14. # construct a list of all the json api files
  15. jsonfiles = []                                                                                                                                                                                            
  16. for root, dirnames, filenames in os.walk(vpp_json_dir):
  17.     for filename in fnmatch.filter(filenames, '*.api.json'):
  18.         jsonfiles.append(os.path.join(vpp_json_dir, filename))                                                                                                                                            
  19. if not jsonfiles:
  20.     print('Error: no json api files found')
  21.     exit(-1)                                                                                                                                                                                              
  22. # use all those files to create vpp.
  23. vpp = VPP(jsonfiles)
  24. r = vpp.connect("test_papi")                                                                                                                                                                              
  25. print(r)
  26.  
  27. async=True                                                                                                                                                                                                
  28. r=vpp.register_event_callback(papi_event_handler)
  29. pid=os.getpid()
  30. sw_ifs = [1,2,3,4,5,6,7]                                                                                                                                                                                  
  31. r = vpp.api.want_per_interface_simple_stats(enable_disable=True, sw_ifs=sw_ifs, num=len(sw_ifs), pid=pid)
  32. print(r)
  33. # Wait for some stats                                                                                                                                                                                    
  34. time.sleep(60)
  35. r = vpp.api.want_per_interface_simple_stats(enable_disable=False)
  36. r = vpp.disconnect()

API generation

The Python binding is automatically generated from the API definitions. See figure below.

Assumptions

  • A common context field is used as a transaction id, for the client to be able to match replies with requests. Not all messages have context and the API handles that, as long as the CONTROL_PING is used to embed them.
  • The API generates code that will send a CONTROL_PING for _DUMP/_DETAIL message exchanges. It will not do so for CALL/CALL_REPLY style calls, so it is important that those conventions are followed.
  • Some messages, e.g. VNET_INTERFACE_COUNTERS are variable sized, with an unspecified u8 data[0] field and a something like a u32 count or u32 nitems field telling the message specific handler the size of the message. There is no way to automatically generate code to handle this, so the Python API returns these to the caller as a byte string. These can then be handled by message specific code like:
 if result.vl_msg_id == vpp_papi.VL_API_VNET_INTERFACE_COUNTERS:
        format = '>' + str(int(len(result.data) / 8)) + 'Q'
        counters = struct.unpack(format, result.data)
 

Papi.png

Future Improvements / TODOs

Performance: Python essentially runs single threaded. The RX thread will hold the Global Interpreter Lock during callback. Current performance is about 1500 messages/second. An implementation in C gets about 450000 messages/second in comparison.

API: Use Python Async I/O?

Exception / error handling

Handle messages like GET_NODE_GRAPH where the reply is a reference to shared memory.