Difference between revisions of "VPP/Python API"

Revision as of 08:27, 1 June 2016

Python binding for the VPP API

The vpp-papi.py module in vpp-api/python/ provides a Python 3 binding to the VPP API. The API is completely auto-generated and the framework should be easily extendable to generate bindings for other languages.

The vpp/api/vpe.api file specifies a set of messages that can be exchanged between VPP and the API client. The semantics of those messages are somewhat up to interpretation and convention.

The language binding is implemented simply by exposing four C calls to Python. Those are:

 int pneum_connect(char *name);
 int pneum_disconnect(void);
 int pneum_read(char **data, int *l);
 int pneum_write(char *data, int len);

In addition there is a Python message handler callback called by the C RX pthread. All message handling and parsing is done in Python.

Currently there are three classes of VPP API methods:

1. Simple request / reply. For example the show_version() call the SHOW_VERSION message is the request and the SHOW_VERSION_REPLY is the answer back. By convention replies are named ending with _REPLY.
2. Dump functions. For example sw_interface_dump() send the SW_INTERFACE_DUMP message and receive a set of messages back. In this example SW_INTERFACE_DETAILS and SW_INTERFACE_SET_FLAGS are (typically) received. The CONTROL_PING/CONTROL_PING_REPLY is used as a method to signal to the client that the last message has been received. By convention the request message have names ending with _DUMP and the replies have names ending in _DETAILS.
3. Register for events. For example want_stats() sends a WANT_STATS message, get a WANT_STATS_REPLY message back, and the client will then asynchronously receive VNET_INTERFACE_COUNTERS messages.

The API is by default blocking although there is possible to get asynchronous behaviour by setting the function argument async=True.

Each call uses the arguments as specified in "vpe.api". The "client_index" and "context" fields are handled by the module itself. A call returns a named tuple or a list of named tuples.

Installation

The main VPP build will build the C library (libpneum.so). If VPP is installed via the Linux package system that library will be available on the system. To run within a build directory, set LD_LIBRARY_PATH to point to the location of libpneum.so.

To build the vpp-api component (and shared library) separately:

make -Cbuild-root PLATFORM=vpp TAG=vpp_debug vpp-api-install
 

To install the Python component:

cd vpp-api/python
sudo python setup.py install
 

Architecture

Example: Dumping interface table

The example is in "vpp-api-client/python/examply.py" and there are unit tests in "vpp-api-client/python/test_papi.py".

#!/usr/bin/env python

import vpp_papi

r = vpp_papi.connect("test_papi")

t = vpp_papi.show_version()
print('VPP version:', t.version.decode())

t = vpp_papi.sw_interface_dump(0, b'ignored')

if t:
    print('List of interfaces')
    for interface in t:
        if interface.vlmsgid == vpp_papi.VL_API_SW_INTERFACE_DETAILS:
            print(interface.interfacename.decode())
r = vpp_papi.disconnect()
 

Examples

Example: Receive statistics

#!/usr/bin/env python3

import struct
import time

import vpp_papi

def papi_event_handler(result):
    if result.vlmsgid == vpp_papi.VL_API_VNET_INTERFACE_COUNTERS:
        format = '>' + str(int(len(result.data) / 8)) + 'Q'
        counters = struct.unpack(format, result.data)
        print('Counters:', counters)
        return

    print('Unknown message id:', result.vlmsgid)

r = vpp_papi.connect("test_papi")
vpp_papi.register_event_callback(papi_event_handler)
r = vpp_papi.want_stats(True, 123)

#
# Wait for some stats
#
time.sleep(60)
r = vpp_papi.want_stats(False, pid)
r = vpp_papi.disconnect()

 

API generation

The Python binding is automatically generated from the API definition in vpp/api/vpe.api. See figure below.

Assumptions

• A common context field is used as a transaction id, for the client to be able to match replies with requests. Not all messages have context and the API handles that, as long as the CONTROL_PING is used to embed them.

• The API generates code that will send a CONTROL_PING for _DUMP/_DETAIL message exchanges. It will not do so for CALL/CALL_REPLY style calls, so it is important that those conventions are followed.

• Some messages, e.g. VNET_INTERFACE_COUNTERS are variable sized, with an unspecified u8 data[0] field and a something like a u32 count or u32 nitems field telling the message specific handler the size of the message. There is no way to automatically generate code to handle this, so the Python API returns these to the caller as a byte string. These can then be handled by message specific code like:

 if result.vlmsgid == vpp_papi.VL_API_VNET_INTERFACE_COUNTERS:
        format = '>' + str(int(len(result.data) / 8)) + 'Q'
        counters = struct.unpack(format, result.data)
 

Future Improvements / TODOs

Performance: Python essentially runs single threaded. The RX thread will hold the Global Interpreter Lock during callback. Current performance is about 1500 messages/second. An implementation in C gets about 450000 messages/second in comparison.

API: Use Python Async I/O?

Exception / error handling

Python packaging

Handle messages like GET_NODE_GRAPH where the reply is a reference to shared memory.