Task

Internal Buffer (Task)

An internal buffer inside a server task decouples the arrival rate of work requests from the processing rate. Callers push requests into the buffer and return immediately; the server pulls and services them at its own pace. Without a buffer, a slow server forces every caller to block for the entire service time.

Why put the buffer inside the task instead of in front of it?

Because an internal buffer lets the server apply arbitrary policy per dequeue: priority, batching, coalescing duplicate requests, dropping stale work. An external buffer is opaque to the server. The cost is that the server now manages two responsibilities (storage + service), which is why the administrator pattern factors it back out once the buffer logic gets complex.

CS343, no concurrency vs some concurrency (Buhr §9.3.1)

The whole §9.3.1 argument starts before the internal buffer: even with buffer size 1, moving work out of the member and into the post-accept block is what creates client/server overlap.

        No Concurrency                        Some Concurrency
_Task server1 {                        _Task server2 {
public:                                public:
    void mem1(...) { S1 }                void mem1(...) { S1.copy-in }
    void mem2(...) { S2 }                int  mem2(...) { S2.copy-out }
    void main() {                        void main() {
        ...                                  ...
        _Accept( mem1 );                     _Accept( mem1 ) { S1.work }
        or _Accept( mem2 );                  or _Accept( mem2 ) { S2.work };
    }                                    }
}                                      }

• server1, all work happens inside mem1/mem2, so the client is blocked for the entire S1/S2 duration. No concurrency.
• server2, member does only S?.copy-in / copy-out; the heavy S?.work runs in the after-accept block on the server’s thread, after the client has returned. Small overlap per request ⇒ real concurrency.

From size-1 to size-N: the internal buffer

server2 is effectively a buffer of size 1 between client and server. Generalize: queue multiple requests so clients drop work even faster. But problems emerge:

• Mean production and consumption rates must match, or the buffer is always full or always empty.
• Task’s mutex means no calls accepted while server is working ⇒ clients can’t drop requests. Server must periodically accept between buffer operations (awkward).
• If clients need replies, a buffer doesn’t help unless you can reorder requests.

Real fix: add a worker thread. Server becomes a pure receiver/dispatcher, worker does S?.work. That’s the customer / manager / employee structure of the administrator pattern.

Sketch

_Task BufferedServer {
    Request buf[N]; int front = 0, back = 0, count = 0;
    void main() {
        for ( ;; ) {
            _When( count < N ) _Accept( submit ) {        // caller drops request
            } or _When( count > 0 ) _Accept( service ) {  // worker pulls
            } or _Accept( ~BufferedServer ) { break; };
        }
    }
  public:
    void submit( Request r ) { buf[back] = r; back = (back+1)%N; count++; }
    Request service() { Request r = buf[front]; front = (front+1)%N; count--; return r; }
};

Bounded vs unbounded

• Bounded ⇒ submit blocks when full, giving back-pressure to producers.
• Unbounded ⇒ submit never blocks, but memory is unbounded under overload.

Buhr’s admin variants

• Timer worker feeds periodic requests into the buffer.
• Courier worker drains the buffer and forwards to downstream servers (keeps admin non-blocking).

Related

• Administrator Pattern
• Bounded Buffer
• Task