# Architecture

For now we will keep things as easy as possible, not caring about the performance.

This means we will use only ::tokio and spawn one job per connection


# Future

I randomly came across this can easily explains most problems with multithreaded
scaling and work queues:  
https://tokio.rs/blog/2019-10-scheduler

What we want to do is:
* one thread per core
* thread pinning
* ebpf BBR packet pacing
  * need to support non-ebpf BBR for mobile ios, too
* connection sharding
  * directly in ebpf?
  * based on connection id
  * no cross-core talk
* implement connection id change
  * any enpoint can ask the other to change the connection id
    * once ACK is received, takes effect, old connection id dropped?
  * lets us rebalance the work on cores, without synchronization
  * can just use one work queue for core, without work stealing

## Problems

### How to integrate application thread pool with libFenrir?

The application/service will obviously have to do some work on its own

This means that it will have either:
* its own thread pool
* our same threadpool

Handling the connection on one thread, only to have the work done elsewhere
will incurr in cross-cpu talk, cache invalidations, synchronization techniques, etc

So we should find a way to either:
* use the same threadpool as the application
* sync work threads with the application threadpool

Do remember that all of this is not just Rust, but it will have a C interface,
which means that it will be used by other languages, too.

This alone probably means that we should run our own tokio runtime