What is Congestion?
Congestion occurs when servers are subjected to client queries faster than they can be fulfilled. Subsequently, the servers begin accumulating a backlog of pending queries. The longer the high rate of traffic continues the farther behind the servers fall. Depending on the client implementations, those that fail to get leases either give up or simply continue to retry forever. In the former case, the server may eventually recover. The latter case is vicious cycle from which the server is unable to escape.
In a well-planned deployment, the number and capacity of servers is matched to the maximum client loads expected. As long as capacity is matched to load, congestion does not occur. If the load is routinely too heavy, then the deployment needs to be re-evaluated. Congestion typically occurs when there is a network event that causes overly large numbers of clients to simultaneously need leases such as recovery after a network outage.
Congestion Handling Overview
Kea 1.5.0 introduces a new feature referred to as Congestion Handling. The goal of Congestion Handling is to help the servers mitigate the peak in traffic by fulfilling as many of the most relevant requests as possible until it subsides.
Prior to Kea 1.5.0, Kea DHCP servers read inbound packets directly from the interface sockets in the main application thread. This meant that packets waiting to be processed were held in socket buffers themselves. Once these buffers fill any new packets are discarded. Under swamped conditions the servers can end up processing client packets that may no longer be relevant, or worse are redundant. In other words, the packets waiting in the FIFO socket buffers become increasingly stale.
Congestion Handling offers the ability to configure the server to use a separate thread to read packets from the interface socket buffers. As the thread reads packets from the buffers they are added to an internal "packet
queue". The server's main application thread processes packets from this queue rather than the socket buffers. By structuring it this way, we've introduced a configurable layer which can make decisions on which packets to process, how to store them, and the order in which they are processed by the server.
The default packet queue implementation for both Kea DHCPv4 and DHCPv6 servers is a simple ring buffer. Once it reaches capacity, new packets get added to the back of queue by discarding packets from the front of queue. Rather than always discarding the newest packets, we now always discard the oldest packets. The capacity of the buffer (i.e. the maximum number of packets the buffer can contain) is configurable.
Custom Packet Queues
It is possible to replace the default packet queue implementation with a custom implementation by registering it with your Kea server via a hook library. The steps for doing this are listed below:
- Develop a derivation of the interface isc::dhcp::PacketQueue
- Registering and un-registering your implementation via Hook library
- Configure your Kea server to use your derivation
(If you are not familiar with writing Kea hook libraries, you may wish to read Hooks Developer's Guide before continuing).
Developing isc::dhcp::PacketQueue Derivations
The Basics
Your custom packet queue must derive from the class template, isc::dhcp::PacketQueue. The class is almost entirely abstract and deliberately brief to provide developers wide latitude in the internals of their solutions.
The template argument, PacketTypePtr, is expected to be either isc::dhcp::Pkt4Ptr or isc::dhcp::Pkt6Ptr, depending upon which protocol the implementation will handle. Please note that the while following text and examples largely focus on DHCPv4 out of convenience as the concepts are identical for DHCPv6. For completeness there are code snippets at the end of this chapter for DHCPv6.
The two primary functions of interest are:
- isc::dhcp::PacketQueue::enqueuePacket() - This function is invoked by the receiver thread each time a packet has been read from an interface socket buffer and should be added to the queue. It is passed a pointer to the unpacked client packet (isc::dhcp::Pkt4Ptr or isc::dhcp::Pkt6Ptr), and a reference to the isc::dhcp::SocketInfo describing the interface socket from which the packet was read. Your derivation is free to use whatever logic you deem appropriate to decide if a given packet should be added to the queue or dropped. The socket information is passed along to be used (or not) in your decision making. The simplest derivation would add every packet, every time.
- isc::dhcp::PacketQueue::dequeuePacket() - This function is invoked by the server's main thread whenever the receiver thread indicates that packets are ready. Which packet is dequeued and returned is entirely up to your derivation.
The remaining functions that you'll need to implement are self-explanatory.
How your actual "queue" is implemented is entirely up to you. Kea's default implementation using a ring buffer based on Boost's boost::circular_buffer (please refer to isc::dhcp::PacketQueueRing, isc::dhcp::PacketQueueRing4 and isc::dhcp::PacketQueueRing6). The most critical aspects to remember when developing your implementation are:
- It MUST be thread safe since queuing and dequeuing packets are done by separate threads. (You might considering using std::mutex and std::lock_guard).
- Its efficiency (or lack thereof) will have a direct impact on server performance. You will have to consider the dynamics of your deployment to determine where the trade-off lies between the volume of packets responded to and preferring to respond to some subset of those packets.
Defining a Factory
isc::dhcp::IfaceMgr using two derivations of isc::dhcp::PacketQueueMgr (one for DHCPv4 and one for DHCPv6), to register queue implementations and instantiate the appropriate queue type based the current configuration. In order to register your queue implementation your hook library must provide a factory function that will be used to create packet queues. This function will be invoked by the server during the configuration process to instantiate the appropriate queue type. For DHCPv4, the factory should be as follows:
boost::shared_ptr< const Element > ConstElementPtr
and for DHCPv6:
The factory's only argument is an isc::data::ConstElementPtr. This is will be an isc::data::MapElement instance containing the contents of the configuration element "dhcp-queue-control" from the Kea server's configuration. It will always have the following two values:
- "enable-queue" - used by isc::dhcp::IfaceMgr to know whether congestion handling is enabled. Your implementation need not do anything with this value.
- "queue-type" - name of the registered queue implementation to use. It is used by isc::dhcp::IfaceMgr to invoke the appropriate queue factory. Your implementation must pass this value through to the isc::dhcp::PacketQueue constructor.
Beyond that you may add whatever additional values you may require. In other words, the content is arbitrary so long as it is valid JSON. It is up to your factory implementation to examine the contents and use them to construct a queue instance.
@subsection packet-queue-derivation-example An Example
Let's suppose you wish to develop a queue for DHCPv4 and your implementation requires two configurable parameters: capacity and threshold. Your class declaration might look something like this:
public:
static const std::string QUEUE_TYPE;
YourPacketQueue4(const std::string& queue_type, size_t capacity, size_t threshold)
}
:
};
read_packet FORK if not parent: break YIELD answer=DNSLookup(packet, this) response=DNSAnswer(answer) YIELD send(response) At each "YIELD" point, the coroutine initiates an asynchronous operation, then pauses and turns over control to some other task on the ASIO service queue. When the operation completes, the coroutine resumes. The DNSLookup and DNSAnswer define callback methods used by a DNS Server to communicate with the module that called it. They are abstract-only classes whose concrete implementations are supplied by the calling module. The DNSLookup callback always runs asynchronously. Concrete implementations must be sure to call the server 's "resume" method when it is finished. In an authoritative server, the DNSLookup implementation would examine the query, look up the answer, then call "resume"(See the diagram in doc/auth_process.jpg). In a recursive server, the DNSLookup implementation would initiate a DNSQuery, which in turn would be responsible for calling the server 's "resume" method(See the diagram in doc/recursive_process.jpg). A DNSQuery object is intended to handle resolution of a query over the network when the local authoritative data sources or cache are not sufficient. The plan is that it will make use of subsidiary DNSFetch calls to get data from particular authoritative servers, and when it has gotten a complete answer, it calls "resume". In current form, however, DNSQuery is much simpler packet
boost::shared_ptr< Pkt4 > Pkt4Ptr
A pointer to Pkt4 object.
Interface for managing a queue of inbound DHCP packets.
virtual PacketTypePtr dequeuePacket()=0
Dequeues the next packet from the queue.
virtual void enqueuePacket(PacketTypePtr packet, const SocketInfo &source)=0
Adds a packet to the queue.
boost::shared_ptr< PacketQueue< Pkt4Ptr > > PacketQueue4Ptr
Defines pointer to the DHCPv4 queue interface used at the application level.
boost::shared_ptr< Pkt4 > Pkt4Ptr
A pointer to Pkt4 object.
Defines the logger used by the top-level component of kea-lfc.
Your factory implementation would then look something like this:
const std::string QUEUE_TYPE = "Your-Q4";
size_t capacity;
try {
} catch (const std::exception& ex) {
" 'capacity' parameter is missing/invalid: " << ex.what());
}
size_t threshold;
try {
} catch (const std::exception& ex) {
" 'threshold' parameter is missing/invalid: " << ex.what());
}
return (queue);
}
static std::string getString(isc::data::ConstElementPtr scope, const std::string &name)
Returns a string parameter from a scope.
static int64_t getInteger(isc::data::ConstElementPtr scope, const std::string &name)
Returns an integer parameter from a scope.
Invalid queue parameter exception.
#define isc_throw(type, stream)
A shortcut macro to insert known values into exception arguments.
Kea's configuration parser cannot know your parameter requirements and thus can only flag JSON syntax errors. Thus it is important for your factory to validate your parameters according to your requirements and throw meaningful exceptions when they are not met. This allows users to know what to correct.
Registering Your Implementation
All hook libraries must provide a load() and unload() function. Your hook library should register you queue factory during load() and un-register it during unload(). Picking up with the our example, those functions might look something like this:
int load(LibraryHandle& ) {
try {
registerPacketQueueFactory(YourPacketQueue4::QUEUE_TYPE,
YourPacketQueue::factory);
} catch (const std::exception& ex) {
.arg(ex.what());
return (1);
}
return (0);
}
unregisterPacketQueueFactory(YourPacketQueue4::QUEUE_TYPE);
return (0);
}
int load(LibraryHandle &)
This function is called when the library is loaded.
int unload()
This function is called when the library is unloaded.
PacketQueueMgr4Ptr getPacketQueueMgr4()
Fetches the DHCPv4 packet queue manager.
static IfaceMgr & instance()
IfaceMgr is a singleton class.
#define LOG_ERROR(LOGGER, MESSAGE)
Macro to conveniently test error output and log it.
#define LOG_INFO(LOGGER, MESSAGE)
Macro to conveniently test info output and log it.
Configuring Kea to use YourPacketQueue4
You're almost there. You developed your implementation, you've unit tested it (You did unit test it right?). Now you just have to tell Kea to load it and use it. Continuing with the example, your kea-dhcp4 configuration would need to look something like this:
{
"Dhcp4":
{
...
"hooks-libraries": [
{
# Loading your hook library!
"library": "/somepath/lib/libyour_packet_queue.so"
}
# any other hook libs
],
...
"dhcp-queue-control": {
"enable-queue": true,
"queue-type": "Your-Q4",
"capacity" : 100,
"threshold" : 75
},
...
}
DHCPv6 Example Snippets
For completeness, this section includes the example from above implemented for DHCPv6.
DHCPv6 Class declaration:
public:
static const std::string QUEUE_TYPE;
YourPacketQueue6(const std::string& queue_type, size_t capacity, size_t threshold)
:
isc::dhcp::PacketQueue<
isc::dhcp::Pkt6Ptr>(queue_type) {
}
:
};
boost::shared_ptr< PacketQueue< Pkt6Ptr > > PacketQueue6Ptr
Defines pointer to the DHCPv6 queue interface used at the application level.
boost::shared_ptr< Pkt6 > Pkt6Ptr
A pointer to Pkt6 packet.
DHCPv6 Factory implementation:
const std::string QUEUE_TYPE = "Your-Q6";
size_t capacity;
try {
} catch (const std::exception& ex) {
" 'capacity' parameter is missing/invalid: " << ex.what());
}
size_t threshold;
try {
} catch (const std::exception& ex) {
" 'threshold' parameter is missing/invalid: " << ex.what());
}
return (queue);
}
DHCPv6 Hook load/unload functions
int load(LibraryHandle& ) {
try {
registerPacketQueueFactory(YourPacketQueue6::QUEUE_TYPE,
YourPacketQueue::factory);
} catch (const std::exception& ex) {
.arg(ex.what());
return (1);
}
return (0);
}
unregisterPacketQueueFactory(YourPacketQueue6::QUEUE_TYPE);
return (0);
}
PacketQueueMgr6Ptr getPacketQueueMgr6()
Fetches the DHCPv6 packet queue manager.
Server configuration for kea-dhcp6:
{
"Dhcp6":
{
...
"hooks-libraries": [
{
# Loading your hook library!
"library": "/somepath/lib/libyour_packet_queue.so"
}
# any other hook libs
],
...
"dhcp-queue-control": {
"enable-queue": true,
"queue-type": "Your-Q6",
"capacity" : 100,
"threshold" : 75
},
...
}
1.13.2