libipq/IPQ.notes.txt

------------------------------------------------------------------------------------
IPv4 Queuing Documentation
------------------------------------------------------------------------------------

Note: this file is temporary until the documentation is complete.

Upgrade information:
	* If upgrading from the queue device (v0.90.4 or below), you will need to
	  delete the old shared library, usually found in
	  /usr/local/lib/iptables/libipt_QUEUE.so
	
TODO List:
	* Non-blocking i/o for userspace api
	* Buffered verdicts
	* Reschedule processing if userspace busy
	* Better session reliability
	* Testsuite scripts, fix/improve tools
	* Documentation
	* Multiple queues per protocol?
	* Performance analysis
	* Userspace language bindings


Overview:
The following diagram is a conceptual view of how the queue operates:

 +---------+
 |  QUEUE  |
 +---------+
 |         |
 |  +---+  | --> dequeue() --> nf_reinject() [stack]
 |  | V |  |
 |  +---+  |
 |         |
 |  +---+  |
 |  | W |  |
 |  +---+  |
 |         |
 |  +---+  | 
 |  | V |  |
 |  +---+  |
 |         |
 |  +---+  |
 |  | V |  | <-- set_verdict() [user]
 |  +---+  |
 |         |
 |  +---+  | 
 |  | W |  | 
 |  +---+  |
 |         |
 |  +---+  |
 |  | N |  | --> notify_user() [user]
 |  +---+  |
 |         |
 +---------+ <-- set_mode()  [user]
      ^
      |
   enqueue() 
      ^
      | 
   nf_queue() [stack]
  

The queue is processed via a kernel thread, which is woken up upon enqueue()
set_mode() and set_verdict().

As the queue is modal, and netlink is connectionless, a reasonable amount of
state needs to be maintained.
 
Packet states:
N = new packet (default initial state)
W = user notfied, waiting for verdict
V = verdict set (usually by user)

Queue states (settable by user):
* HOLD (default initial state)
enqueue packets
do not notify user
do not accept verdicts
do not dequeue packets

* NORMAL
enqueue packets
notify user of new packets (may copy entire packet)
accept verdicts from user (may include modified packet)
dequeue packets

* FLUSH (returns to HOLD when queue is empty, unless terminating)
do not enqueue packets
do not not notify user
set verdicts on all packets to NF_DROP
dequeue all packets for dropping

Note that for HOLD & NORMAL queue states, new packets are dropped if the 
queue is full.

Known bugs:
- Userspace app gets unknown message from kernel if it sends an invalid
  message type (should get an NLMSG_ERROR).
  
Documentation notes:
libipq:
- Queue is held after flush completes, user must either start copying 
  or shutdown or the queue will fill up.
  
- If you get a IPQ_ERR_RTRUNC message, your local receive
  buffer is probably too small.  Netlink has no way of detecting
  this, and thinks the message was delivered (technically, it was,
  to your *socket* receive buffer though).  Thus you need to respond
  with an NF_DROP for the packet and use a bigger buffer.
  
- If you modify a packet, you must recalculate checksums as 
  appropriate before sending it back.

- The code wont stop you from doing this, but try not to set NF_QUEUE
  verdict on packets.