Extracted from several tidbits, but if I were to venture a summary of the main point, it would be this: The combination of preemptive multitasking and general shared state leads to unmanageable complexity in the software development process, and developers may prefer to keep some sense of their own to avoid such unmanageable complexity. Preemptive scheduling is good for truly parallel tasks, but explicit multitasking cooperation is preferred when mutable state is shared across multiple concurrent threads.

Despite cooperative multitasking, your code can still be complex, it just has the opportunity to remain manageable with some complexity. When control transfer is made clear to a code reader there are at least some visible signs that things may be going off track. Not explicitly marking each new stage as a potential landmine: “If this operation is not atomic, what happens at the end?” Then the space between each command becomes an endless spatial black hole, and terrible Heisenbugs emerge

For more than a year, though, work on Heka (a high-performance data, logging, and metrics processing engine) has mostly been developed using GO. One of the nice things about Go is that the language itself has some very useful concurrency primitives. But the concurrency performance of Go needs to be seen through the lens of encouraging code that supports local reasoning.

Not all the facts are good. All goroutines access the same shared memory space, and the state is variable by default, but Go’s scheduler does not guarantee accuracy in context selection. In a single-core setup, Go’s runtime falls into the “implicit collaboration” category, selecting 4 in the list of asynchronous program models often mentioned in Glyph. When Goroutine can run in parallel on multi-core systems, you never know.

Go can’t protect you, but that doesn’t mean you can’t take steps to protect yourself. By using some of the primitives provided by Go during code writing, you can minimize the abnormal behavior associated with preemptive scheduling. Take a look at the Go interface in the Glyph example “Account conversion” snippet below (ignore floating point numbers that don’t easily end up storing fixed-point decimals)

func Transfer(amount float64, payer, payee *Account, 
    server SomeServerType) error { 
    if payer.Balance() < amount { 
        return errors.New("Insufficient funds") 
    } 
    log.Printf("%s has sufficient funds", payer) 
    payee.Deposit(amount) 
    log.Printf("%s received payment", payee) 
    payer.Withdraw(amount) 
    log.Printf("%s made payment", payer) 
    server.UpdateBalances(payer, payee) // Assume this is magic and always works. 
    return nil 
} 
Copy the code

This is obviously not safe if called from multiple Goroutines, as they may concurrently get the same result from deposit scheduling and then request more cancelled deposit variables together. It is best that dangerous parts of code are not executed by multiple Goroutines. This feature is implemented in one way:

type transfer struct { 
    payer *Account 
    payee *Account 
    amount float64 
} 
var xferChan = make(chan *transfer) 
var errChan = make(chan error) 
func init() { 
    go transferLoop() 
} 
func transferLoop() { 
    for xfer := range xferChan { 
        if xfer.payer.Balance < xfer.amount { 
            errChan <- errors.New("Insufficient funds") 
            continue 
        } 
        log.Printf("%s has sufficient funds", xfer.payer) 
        xfer.payee.Deposit(xfer.amount) 
        log.Printf("%s received payment", xfer.payee) 
        xfer.payer.Withdraw(xfer.amount) 
        log.Printf("%s made payment", xfer.payer) 
        errChan <- nil 
    } 
} 
func Transfer(amount float64, payer, payee *Account, 
    server SomeServerType) error { 
    xfer := &transfer{ 
        payer: payer, 
        payee: payee, 
        amount: amount, 
    } 
    xferChan <- xfer 
    err := <-errChan 
    if err == nil  { 
        server.UpdateBalances(payer, payee) // Still magic. 
    } 
    return err 
} 
Copy the code

There’s more code here, but we eliminate the concurrency problem by implementing a trivial event loop. When the code is first executed, it activates a Goroutine run loop. Forward requests are passed to a newly created channel for this purpose. The result is returned to the outside of the loop via an error channel. Because channels are not buffered, they are locked, and through the Transfer function they are continuously serviced through a single running event loop no matter how many concurrent forward requests come in.

The code above looks a little awkward, maybe. A mutex might be a better choice for such a simple scenario, but what I’m trying to prove is that you can apply the isolation state operation to a GO routine. It works well enough for most requirements, if slightly awkward, and it works with even the simplest account structure:

type Account struct { 
    balance float64 
} 
func (a *Account) Balance() float64 { 
    return a.balance 
} 
func (a *Account) Deposit(amount float64) { 
    log.Printf("depositing: %f", amount) 
    a.balance += amount 
} 
func (a *Account) Withdraw(amount float64) { 
    log.Printf("withdrawing: %f", amount) 
    a.balance -= amount 
} 
Copy the code

But such a clumsy account implementation can seem naive. It may be more useful to allow the account structure itself to provide some protection by not allowing any withdrawals to occur that are larger than the current balance. So what happens if we make the withdrawal function look like this?

func (a *Account) Withdraw(amount float64) { 
    if amount > a.balance { 
        log.Println("Insufficient funds") 
        return 
    } 
    log.Printf("withdrawing: %f", amount) 
    a.balance -= amount 
} 
Copy the code

Unfortunately, this code suffers from the same problem as our original Transfer implementation. Concurrent execution or unfortunate context switches mean we can end up with negative balance. Fortunately, the internal event loop concept applies equally well here, if not better, because the Event loop Goroutine is well coupled to each individual account structure instance. Here’s an example to illustrate the point:

type Account struct { 
    balance float64 
    deltaChan chan float64 
    balanceChan chan float64 
    errChan chan error 
} 
func NewAccount(balance float64) (a *Account) { 
    a = &Account{ 
        balance:     balance, 
        deltaChan:   make(chan float64), 
        balanceChan: make(chan float64), 
        errChan:     make(chan error), 
    } 
    go a.run() 
    return 
} 
func (a *Account) Balance() float64 { 
    return <-a.balanceChan 
} 
func (a *Account) Deposit(amount float64) error { 
    a.deltaChan <- amount 
    return <-a.errChan 
} 
func (a *Account) Withdraw(amount float64) error { 
    a.deltaChan <- -amount 
    return <-a.errChan 
} 
func (a *Account) applyDelta(amount float64) error { 
    newBalance := a.balance + amount 
    if newBalance < 0 { 
        return errors.New("Insufficient funds") 
    } 
    a.balance = newBalance 
    return nil 
} 
func (a *Account) run() { 
    var delta float64 
    for { 
        select { 
        case delta = <-a.deltaChan: 
            a.errChan <- a.applyDelta(delta) 
        case a.balanceChan <- a.balance: 
            // Do nothing, we've accomplished our goal w/ the channel put. } } }Copy the code

This API is slightly different; both the Deposit and Withdraw methods now return an error. Instead of processing their requests directly, they put the account balance adjustment into the deltaChan, which is accessed in the event loop when the Run method runs. Similarly, the Balance method keeps requesting data in the event loop through blocking until it receives a value through balanceChan.

The important thing to note in the above code is that all direct access to and modification of data within the structure is triggered by an event loop

within

At the heart of this pattern is Heke’s design. When Heka starts, it reads the configuration file and starts each plug-in in its own GO routine. With clock signals, shutdown notifications, and other control signals, data is channelled into the plug-in. This encourages plug-in authors to implement plug-in functionality using an event-loop type architecture like the one in the above example.

Again, GO doesn’t protect yourself. It is entirely possible to write a Heka plug-in (or any architecture for that matter) that remains loosely coupled to its internal data management and subject matter at issue. But there are a few small caveats, as well as Go’s controversial probe free application, where you can write code whose behavior is predictable, even in preemptive scheduling facade code.

Sane Concurrency with Go

Link: http://www.oschina.net/translate/sane-concurrency-with-go