2017-05-02 23:33:51 +00:00
|
|
|
package connection
|
|
|
|
|
|
|
|
import (
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
"context"
|
2017-07-04 18:29:11 +00:00
|
|
|
"fmt"
|
2018-06-08 21:54:31 +00:00
|
|
|
"git.openprivacy.ca/openprivacy/libricochet-go/channels"
|
|
|
|
"git.openprivacy.ca/openprivacy/libricochet-go/utils"
|
|
|
|
"git.openprivacy.ca/openprivacy/libricochet-go/wire/control"
|
2020-02-10 19:30:32 +00:00
|
|
|
"git.openprivacy.ca/openprivacy/log"
|
2018-09-22 20:12:08 +00:00
|
|
|
"github.com/golang/protobuf/proto"
|
2017-05-02 23:33:51 +00:00
|
|
|
"io"
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
"sync"
|
2017-05-02 23:33:51 +00:00
|
|
|
)
|
|
|
|
|
|
|
|
// Connection encapsulates the state required to maintain a connection to
|
|
|
|
// a ricochet service.
|
|
|
|
type Connection struct {
|
|
|
|
utils.RicochetNetwork
|
|
|
|
|
|
|
|
channelManager *ChannelManager
|
2018-01-14 16:45:30 +00:00
|
|
|
ctrlChannel ControlChannel
|
2017-05-02 23:33:51 +00:00
|
|
|
|
|
|
|
// Ricochet Network Loop
|
|
|
|
packetChannel chan utils.RicochetData
|
|
|
|
errorChannel chan error
|
|
|
|
|
|
|
|
breakChannel chan bool
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
breakResultChannel chan error
|
2017-05-02 23:33:51 +00:00
|
|
|
|
|
|
|
unlockChannel chan bool
|
|
|
|
unlockResponseChannel chan bool
|
|
|
|
|
|
|
|
messageBuilder utils.MessageBuilder
|
|
|
|
|
2019-11-08 00:11:14 +00:00
|
|
|
closed bool
|
|
|
|
closingLock sync.Mutex
|
|
|
|
closing bool
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
// This mutex is exclusively for preventing races during blocking
|
|
|
|
// interactions with Process; specifically Do and Break. Don't use
|
|
|
|
// it for anything else. See those functions for an explanation.
|
|
|
|
processBlockMutex sync.Mutex
|
|
|
|
|
2019-01-11 20:28:50 +00:00
|
|
|
conn io.ReadWriteCloser
|
2018-01-02 17:23:20 +00:00
|
|
|
IsInbound bool
|
2018-01-15 18:07:54 +00:00
|
|
|
am AuthorizationManager
|
2018-01-02 17:23:20 +00:00
|
|
|
RemoteHostname string
|
2018-01-01 18:06:58 +00:00
|
|
|
SupportChannels []string
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
func (rc *Connection) init() {
|
|
|
|
|
|
|
|
rc.packetChannel = make(chan utils.RicochetData)
|
|
|
|
rc.errorChannel = make(chan error)
|
|
|
|
|
|
|
|
rc.breakChannel = make(chan bool)
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
rc.breakResultChannel = make(chan error)
|
2017-05-02 23:33:51 +00:00
|
|
|
|
|
|
|
rc.unlockChannel = make(chan bool)
|
|
|
|
rc.unlockResponseChannel = make(chan bool)
|
|
|
|
|
2018-01-15 18:07:54 +00:00
|
|
|
rc.am.Init()
|
|
|
|
rc.am.AddAuthorization("none")
|
2017-05-02 23:33:51 +00:00
|
|
|
go rc.start()
|
|
|
|
}
|
|
|
|
|
|
|
|
// NewInboundConnection creates a new Connection struct
|
|
|
|
// modelling an Inbound Connection
|
|
|
|
func NewInboundConnection(conn io.ReadWriteCloser) *Connection {
|
|
|
|
rc := new(Connection)
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.conn = conn
|
2017-05-02 23:33:51 +00:00
|
|
|
rc.IsInbound = true
|
|
|
|
rc.init()
|
|
|
|
rc.channelManager = NewServerChannelManager()
|
2018-01-14 16:45:30 +00:00
|
|
|
rc.ctrlChannel.Init(rc.channelManager)
|
2017-05-02 23:33:51 +00:00
|
|
|
return rc
|
|
|
|
}
|
|
|
|
|
|
|
|
// NewOutboundConnection creates a new Connection struct
|
|
|
|
// modelling an Inbound Connection
|
|
|
|
func NewOutboundConnection(conn io.ReadWriteCloser, remoteHostname string) *Connection {
|
|
|
|
rc := new(Connection)
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.conn = conn
|
2017-05-02 23:33:51 +00:00
|
|
|
rc.IsInbound = false
|
|
|
|
rc.init()
|
|
|
|
rc.RemoteHostname = remoteHostname
|
|
|
|
rc.channelManager = NewClientChannelManager()
|
2018-01-14 16:45:30 +00:00
|
|
|
rc.ctrlChannel.Init(rc.channelManager)
|
2017-05-02 23:33:51 +00:00
|
|
|
return rc
|
|
|
|
}
|
|
|
|
|
|
|
|
// start
|
|
|
|
func (rc *Connection) start() {
|
|
|
|
for {
|
2019-01-11 20:28:50 +00:00
|
|
|
packet, err := rc.RecvRicochetPacket(rc.conn)
|
2017-05-02 23:33:51 +00:00
|
|
|
if err != nil {
|
|
|
|
rc.errorChannel <- err
|
|
|
|
return
|
|
|
|
}
|
|
|
|
rc.packetChannel <- packet
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
// Do allows any function utilizing Connection to be run safely, if you're
|
|
|
|
// careful. All operations which require access (directly or indirectly) to
|
|
|
|
// Connection while Process is running need to use Do. Calls to Do without
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
// Process running will block unless the connection is closed, which is
|
|
|
|
// returned as ConnectionClosedError.
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
//
|
|
|
|
// Like a mutex, Do cannot be called recursively. This will deadlock. As
|
|
|
|
// a result, no API in this library that can be reached from the application
|
|
|
|
// should use Do, with few exceptions. This would make the API impossible
|
|
|
|
// to use safely in many cases.
|
|
|
|
//
|
|
|
|
// Do is safe to call from methods of connection.Handler and channel.Handler
|
|
|
|
// that are called by Process.
|
2017-05-02 23:33:51 +00:00
|
|
|
func (rc *Connection) Do(do func() error) error {
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
// There's a complicated little dance here to prevent a race when the
|
|
|
|
// Process call is returning for a connection error. The problem is
|
|
|
|
// that if Do simply checked rc.closed and then tried to send, it's
|
|
|
|
// possible for Process to change rc.closed and stop reading before the
|
|
|
|
// send statement is executed, creating a deadlock.
|
|
|
|
//
|
|
|
|
// To prevent this, all of the functions that block on Process should
|
|
|
|
// do so by acquiring processBlockMutex, aborting if rc.closed is true,
|
|
|
|
// performing their blocking channel operations, and then releasing the
|
|
|
|
// mutex.
|
|
|
|
//
|
|
|
|
// This works because Process will always use a separate goroutine to
|
|
|
|
// acquire processBlockMutex before changing rc.closed, and the mutex
|
|
|
|
// guarantees that no blocking channel operation can happen during or
|
|
|
|
// after the value is changed. Since these operations block the Process
|
|
|
|
// loop, the behavior of multiple concurrent calls to Do/Break doesn't
|
|
|
|
// change: they just end up blocking on the mutex before blocking on the
|
|
|
|
// channel.
|
|
|
|
rc.processBlockMutex.Lock()
|
|
|
|
defer rc.processBlockMutex.Unlock()
|
|
|
|
if rc.closed {
|
|
|
|
return utils.ConnectionClosedError
|
|
|
|
}
|
|
|
|
|
2017-05-02 23:33:51 +00:00
|
|
|
// Force process to soft-break so we can lock
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("request unlocking of process loop for do()")
|
2017-05-02 23:33:51 +00:00
|
|
|
rc.unlockChannel <- true
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("process loop is unlocked for do()")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
defer func() {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("giving up lock process loop after do() ")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
rc.unlockResponseChannel <- true
|
|
|
|
}()
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
|
|
|
|
// Process sets rc.closing when it's trying to acquire the mutex and
|
|
|
|
// close down the connection. Behave as if the connection was already
|
|
|
|
// closed.
|
|
|
|
if rc.closing {
|
|
|
|
return utils.ConnectionClosedError
|
|
|
|
}
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
return do()
|
|
|
|
}
|
|
|
|
|
|
|
|
// DoContext behaves in the same way as Do, but also respects the provided
|
|
|
|
// context when blocked, and passes the context to the callback function.
|
|
|
|
//
|
|
|
|
// DoContext should be used when any call to Do may need to be cancelled
|
|
|
|
// or timed out.
|
|
|
|
func (rc *Connection) DoContext(ctx context.Context, do func(context.Context) error) error {
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
// .. see above
|
|
|
|
rc.processBlockMutex.Lock()
|
|
|
|
defer rc.processBlockMutex.Unlock()
|
|
|
|
if rc.closed {
|
|
|
|
return utils.ConnectionClosedError
|
|
|
|
}
|
|
|
|
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
// Force process to soft-break so we can lock
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("request unlocking of process loop for do()")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
select {
|
|
|
|
case rc.unlockChannel <- true:
|
|
|
|
break
|
|
|
|
case <-ctx.Done():
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("giving up on unlocking process loop for do() because context cancelled")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
return ctx.Err()
|
|
|
|
}
|
|
|
|
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("process loop is unlocked for do()")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
defer func() {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("giving up lock process loop after do() ")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
rc.unlockResponseChannel <- true
|
|
|
|
}()
|
|
|
|
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
if rc.closing {
|
|
|
|
return utils.ConnectionClosedError
|
|
|
|
}
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
return do(ctx)
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// RequestOpenChannel sends an OpenChannel message to the remote client.
|
2017-09-16 13:46:28 +00:00
|
|
|
// An error is returned only if the requirements for opening this channel
|
|
|
|
// are not met on the local side (a nil error return does not mean the
|
|
|
|
// channel was opened successfully, because channels open asynchronously).
|
2017-09-17 21:01:22 +00:00
|
|
|
func (rc *Connection) RequestOpenChannel(ctype string, handler channels.Handler) (*channels.Channel, error) {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("requesting open channel of type %s", ctype))
|
2018-01-16 16:53:34 +00:00
|
|
|
channel, err := rc.buildChannel(handler, rc.channelManager.OpenChannelRequest)
|
|
|
|
if err == nil {
|
|
|
|
response, err := handler.OpenOutbound(channel)
|
|
|
|
return rc.handleChannelOpening(channel, response, err)
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
}
|
2018-01-16 16:53:34 +00:00
|
|
|
return nil, err
|
|
|
|
}
|
2017-05-02 23:33:51 +00:00
|
|
|
|
2018-01-16 16:53:34 +00:00
|
|
|
func (rc *Connection) handleChannelOpening(channel *channels.Channel, response []byte, err error) (*channels.Channel, error) {
|
|
|
|
if err == nil {
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, 0, response)
|
2018-01-16 16:53:34 +00:00
|
|
|
return channel, nil
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
}
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("failed to request open channel: %v", err))
|
2018-01-16 16:53:34 +00:00
|
|
|
rc.channelManager.RemoveChannel(channel.ID)
|
|
|
|
return nil, err
|
|
|
|
}
|
2017-05-02 23:33:51 +00:00
|
|
|
|
2018-01-16 16:53:34 +00:00
|
|
|
func (rc *Connection) buildChannel(handler channels.Handler, openChannelFunc func(handler channels.Handler) (*channels.Channel, error)) (*channels.Channel, error) {
|
|
|
|
err := rc.am.Authorized(handler.RequiresAuthentication())
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
if err == nil {
|
2018-01-16 16:53:34 +00:00
|
|
|
channel, err := openChannelFunc(handler)
|
|
|
|
if err == nil {
|
|
|
|
channel.SendMessage = func(message []byte) {
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, channel.ID, message)
|
2018-01-16 16:53:34 +00:00
|
|
|
}
|
|
|
|
channel.DelegateAuthorization = func() {
|
|
|
|
rc.am.AddAuthorization(handler.Type())
|
|
|
|
}
|
|
|
|
channel.CloseChannel = func() {
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, channel.ID, []byte{})
|
2018-01-16 16:53:34 +00:00
|
|
|
rc.channelManager.RemoveChannel(channel.ID)
|
|
|
|
}
|
2019-01-22 20:21:56 +00:00
|
|
|
channel.DelegateEncryption = func(key [32]byte) {
|
|
|
|
rc.SetEncryptionKey(key)
|
|
|
|
}
|
2018-01-16 16:53:34 +00:00
|
|
|
return channel, nil
|
|
|
|
}
|
2018-01-16 17:13:46 +00:00
|
|
|
return nil, err
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
}
|
2018-01-16 16:53:34 +00:00
|
|
|
return nil, err
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// processUserCallback should be used to wrap any calls into handlers or
|
|
|
|
// application code from the Process goroutine. It handles calls to Do
|
|
|
|
// from within that code to prevent deadlocks.
|
|
|
|
func (rc *Connection) processUserCallback(cb func()) {
|
|
|
|
done := make(chan struct{})
|
|
|
|
go func() {
|
|
|
|
defer close(done)
|
|
|
|
cb()
|
|
|
|
}()
|
|
|
|
for {
|
|
|
|
select {
|
|
|
|
case <-done:
|
|
|
|
return
|
|
|
|
case <-rc.unlockChannel:
|
|
|
|
<-rc.unlockResponseChannel
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
}
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Process receives socket and protocol events for the connection. Methods
|
|
|
|
// of the application-provided `handler` will be called from this goroutine
|
|
|
|
// for all events.
|
|
|
|
//
|
|
|
|
// Process must be running in order to handle any events on the connection,
|
|
|
|
// including connection close.
|
|
|
|
//
|
|
|
|
// Process blocks until the connection is closed or until Break() is called.
|
2018-11-30 21:04:38 +00:00
|
|
|
// Infof the connection is closed, a non-nil error is returned.
|
2017-05-02 23:33:51 +00:00
|
|
|
func (rc *Connection) Process(handler Handler) error {
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
if rc.closed {
|
|
|
|
return utils.ConnectionClosedError
|
|
|
|
}
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("entering process loop")
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
rc.processUserCallback(func() { handler.OnReady(rc) })
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
|
|
|
|
// There are exactly two ways out of this loop: a signal on breakChannel
|
|
|
|
// caused by a call to Break, or a connection-fatal error on errorChannel.
|
|
|
|
//
|
|
|
|
// In the Break case, no particular care is necessary; it is the caller's
|
|
|
|
// responsibility to make sure there aren't e.g. concurrent calls to Do.
|
|
|
|
//
|
|
|
|
// Because connection errors can happen spontaneously, they must carefully
|
|
|
|
// prevent concurrent calls to Break or Do that could deadlock when Process
|
|
|
|
// returns.
|
|
|
|
for {
|
2017-05-02 23:33:51 +00:00
|
|
|
|
|
|
|
var packet utils.RicochetData
|
|
|
|
select {
|
|
|
|
case <-rc.unlockChannel:
|
|
|
|
<-rc.unlockResponseChannel
|
|
|
|
continue
|
|
|
|
case <-rc.breakChannel:
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("process has ended after break")
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
rc.breakResultChannel <- nil
|
|
|
|
return nil
|
2017-05-02 23:33:51 +00:00
|
|
|
case packet = <-rc.packetChannel:
|
|
|
|
break
|
|
|
|
case err := <-rc.errorChannel:
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.conn.Close()
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
rc.closing = true
|
|
|
|
|
|
|
|
// In order to safely close down concurrent calls to Do or Break,
|
|
|
|
// processBlockMutex must be held before setting rc.closed. That cannot
|
|
|
|
// happen in this goroutine, because one of those calls may already hold
|
|
|
|
// the mutex and be blocking on a channel send to this method. So the
|
|
|
|
// process here is to have a goroutine acquire the lock, set rc.closed, and
|
|
|
|
// signal back. Meanwhile, this one keeps handling unlockChannel and
|
|
|
|
// breakChannel.
|
|
|
|
closedChan := make(chan struct{})
|
|
|
|
go func() {
|
|
|
|
rc.processBlockMutex.Lock()
|
|
|
|
defer rc.processBlockMutex.Unlock()
|
2019-11-08 00:11:14 +00:00
|
|
|
rc.closingLock.Lock()
|
|
|
|
defer rc.closingLock.Unlock()
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
rc.closed = true
|
|
|
|
close(closedChan)
|
|
|
|
}()
|
|
|
|
|
|
|
|
// Keep accepting calls from Do or Break until closedChan signals that they're
|
|
|
|
// safely shut down.
|
|
|
|
clearLoop:
|
|
|
|
for {
|
|
|
|
select {
|
|
|
|
case <-rc.unlockChannel:
|
|
|
|
<-rc.unlockResponseChannel
|
|
|
|
case <-rc.breakChannel:
|
|
|
|
rc.breakResultChannel <- utils.ConnectionClosedError
|
|
|
|
case <-closedChan:
|
|
|
|
break clearLoop
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// This is the one case where processUserCallback isn't necessary, because
|
|
|
|
// all calls to Do immediately return ConnectionClosedError now.
|
|
|
|
handler.OnClosed(err)
|
2017-05-02 23:33:51 +00:00
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
if packet.Channel == 0 {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("received control packet on channel %d", packet.Channel))
|
2017-05-02 23:33:51 +00:00
|
|
|
res := new(Protocol_Data_Control.Packet)
|
|
|
|
err := proto.Unmarshal(packet.Data[:], res)
|
|
|
|
if err == nil {
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
// Wrap controlPacket in processUserCallback, since it calls out in many
|
|
|
|
// places, and wrapping the rest is harmless.
|
|
|
|
rc.processUserCallback(func() { rc.controlPacket(handler, res) })
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
} else {
|
|
|
|
// Let's check to see if we have defined this channel.
|
|
|
|
channel, found := rc.channelManager.GetChannel(packet.Channel)
|
|
|
|
if found {
|
|
|
|
if len(packet.Data) == 0 {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("removing channel %d", packet.Channel))
|
2017-05-02 23:33:51 +00:00
|
|
|
rc.channelManager.RemoveChannel(packet.Channel)
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
rc.processUserCallback(func() { channel.Handler.Closed(utils.ChannelClosedByPeerError) })
|
2017-05-02 23:33:51 +00:00
|
|
|
} else {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("received packet on %v channel %d", channel.Handler.Type(), packet.Channel))
|
2017-05-02 23:33:51 +00:00
|
|
|
// Send The Ricochet Packet to the Handler
|
Fix and document safety problems with Connection.Do
There were several issues with the Do function that made it nearly
impossible to write safe code.
First, Do cannot be called recursively -- it will deadlock. There is
actually no way to implement a safe and recursive Do (or mutex) in Go,
because there is no primitive that will identify the current goroutine.
RequestOpenChannel used Do internally, which made it impossible to open
channels safely in many circumstances. That has been removed, so all
calls to RequestOpenChannel must be changed to happen under Do now.
Do now has more documentation and a new rule: no code exposed through
API can use Do, unless it has sole custody of the connection (such as
ProcessAuthAsClient).
Related to that problem, Do was impossible to call from inside handlers
(or anything else on the process goroutine) -- it would again just
deadlock. This is resolved by wrapping calls into user code to continue
handling invocations of Do (and only those) while the handler is
executing.
There is a third issue with connection close, but it will be addressed
in a separate commit
And finally, because it's impossible to timeout or interrupt a call to
Do, I also added a DoContext method that takes a go Context, which is
also passed through to the called function.
2017-09-16 19:38:00 +00:00
|
|
|
rc.processUserCallback(func() { channel.Handler.Packet(packet.Data[:]) })
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
} else {
|
|
|
|
// When a non-zero packet is received for an unknown
|
|
|
|
// channel, the recipient responds by closing
|
2018-01-12 18:59:52 +00:00
|
|
|
// that channel.
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("received packet on unknown channel %d. closing.", packet.Channel))
|
2017-05-02 23:33:51 +00:00
|
|
|
if len(packet.Data) != 0 {
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, packet.Channel, []byte{})
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (rc *Connection) controlPacket(handler Handler, res *Protocol_Data_Control.Packet) {
|
|
|
|
|
|
|
|
if res.GetOpenChannel() != nil {
|
|
|
|
|
|
|
|
opm := res.GetOpenChannel()
|
|
|
|
chandler, err := handler.OnOpenChannelRequest(opm.GetChannelType())
|
|
|
|
if err == nil {
|
2018-01-16 16:53:34 +00:00
|
|
|
openChannel := func(chandler channels.Handler) (*channels.Channel, error) {
|
|
|
|
return rc.channelManager.OpenChannelRequestFromPeer(opm.GetChannelIdentifier(), chandler)
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
2018-01-16 16:53:34 +00:00
|
|
|
channel, err := rc.buildChannel(chandler, openChannel)
|
2019-01-26 22:04:28 +00:00
|
|
|
|
|
|
|
if err != nil {
|
|
|
|
log.Errorf("Failed to build channel from Control Packet: %v", err)
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
2018-01-16 16:53:34 +00:00
|
|
|
response, err := chandler.OpenInbound(channel, opm)
|
|
|
|
_, err = rc.handleChannelOpening(channel, response, err)
|
|
|
|
if err != nil {
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, 0, []byte{})
|
2018-01-16 16:53:34 +00:00
|
|
|
}
|
|
|
|
return
|
2018-01-15 18:07:54 +00:00
|
|
|
}
|
2017-05-02 23:33:51 +00:00
|
|
|
|
2018-01-15 18:07:54 +00:00
|
|
|
errorText := "GenericError"
|
|
|
|
switch err {
|
|
|
|
case utils.UnknownChannelTypeError:
|
|
|
|
errorText = "UnknownTypeError"
|
|
|
|
case utils.UnauthorizedChannelTypeError:
|
|
|
|
errorText = "UnauthorizedTypeError"
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
2018-01-15 18:07:54 +00:00
|
|
|
// Send Error Packet
|
|
|
|
response := rc.messageBuilder.RejectOpenChannel(opm.GetChannelIdentifier(), errorText)
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("sending reject open channel for %v: %v", opm.GetChannelIdentifier(), errorText))
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, 0, response)
|
2018-01-15 18:07:54 +00:00
|
|
|
|
2017-05-02 23:33:51 +00:00
|
|
|
} else if res.GetChannelResult() != nil {
|
2018-01-14 16:45:30 +00:00
|
|
|
rc.ctrlChannel.ProcessChannelResult(res.GetChannelResult())
|
2017-05-02 23:33:51 +00:00
|
|
|
} else if res.GetKeepAlive() != nil {
|
|
|
|
// XXX Though not currently part of the protocol
|
|
|
|
// We should likely put these calls behind
|
|
|
|
// authentication.
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("received keep alive packet")
|
2018-01-14 16:45:30 +00:00
|
|
|
respond, data := rc.ctrlChannel.ProcessKeepAlive(res.GetKeepAlive())
|
2018-01-13 16:43:49 +00:00
|
|
|
if respond {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("sending keep alive response")
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, 0, data)
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
} else if res.GetEnableFeatures() != nil {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("received enable features packet")
|
2018-01-14 16:45:30 +00:00
|
|
|
data := rc.ctrlChannel.ProcessEnableFeatures(handler, res.GetEnableFeatures())
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("sending featured enabled: %v", data))
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, 0, data)
|
2017-05-02 23:33:51 +00:00
|
|
|
} else if res.GetFeaturesEnabled() != nil {
|
2018-01-02 17:23:20 +00:00
|
|
|
rc.SupportChannels = res.GetFeaturesEnabled().GetFeature()
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln(fmt.Sprintf("connection supports: %v", rc.SupportChannels))
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-01-05 22:16:52 +00:00
|
|
|
// EnableFeatures sends an EnableFeatures messages which includes the
|
|
|
|
// list of `features`
|
2018-01-01 18:06:58 +00:00
|
|
|
func (rc *Connection) EnableFeatures(features []string) {
|
|
|
|
messageBuilder := new(utils.MessageBuilder)
|
|
|
|
raw := messageBuilder.EnableFeatures(features)
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.SendRicochetPacket(rc.conn, 0, raw)
|
2018-01-01 18:06:58 +00:00
|
|
|
}
|
|
|
|
|
2017-05-02 23:33:51 +00:00
|
|
|
// Break causes Process() to return, but does not close the underlying connection
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
// Break returns an error if it would not be valid to call Process() again for
|
|
|
|
// the connection now. Currently, the only such error is ConnectionClosedError.
|
|
|
|
func (rc *Connection) Break() error {
|
|
|
|
// See Do() for an explanation of the concurrency here; it's complicated.
|
|
|
|
// The summary is that this mutex prevents races on connection close that
|
|
|
|
// could lead to deadlocks in Block().
|
|
|
|
rc.processBlockMutex.Lock()
|
|
|
|
defer rc.processBlockMutex.Unlock()
|
|
|
|
if rc.closed {
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("ignoring break because connection is already closed")
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
return utils.ConnectionClosedError
|
|
|
|
}
|
2018-12-03 21:26:15 +00:00
|
|
|
log.Debugln("breaking out of process loop")
|
2017-05-02 23:33:51 +00:00
|
|
|
rc.breakChannel <- true
|
Prevent deadlocks with Do or Break and closing connections
After the last round of fixes in Do, there was still one major issue
making it impossible to use Do or Break safely: closing connections.
Connections can be closed spontaneously, and this causes Process to
return. At that moment any ongoing call to Do or Break has deadlocked;
nothing will ever read those channels again. To prevent this, we have to
ensure that Do or Break won't try to send to Process' channels once it
has stopped reading from them. Doing that without other races is tricky.
The solution here, briefly, is to acquire a mutex in Do and Break before
checking the `closed` boolean, and hold that mutex for the entire
operation (including any blocking channel operations). When Process is
closing down the connection, it uses a separate goroutine to acquire the
same mutex and change the boolean, while still handling channel reads.
Once the boolean has changed, the mutex guarantees that nothing will try
to send to these channels again.
I've tried to document the problems and solutions in the code, because
it is subtle in some places and this is definitely critical code.
2017-09-16 21:45:07 +00:00
|
|
|
return <-rc.breakResultChannel // Wait for Process to End
|
2017-05-02 23:33:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Channel is a convienciance method for returning a given channel to the caller
|
|
|
|
// of Process() - TODO - this is kind of ugly.
|
|
|
|
func (rc *Connection) Channel(ctype string, way channels.Direction) *channels.Channel {
|
|
|
|
return rc.channelManager.Channel(ctype, way)
|
|
|
|
}
|
2019-01-11 20:28:50 +00:00
|
|
|
|
|
|
|
// Close tearsdown a Process() and explicitly Closes the connection.
|
|
|
|
// Note, that if Process() is holding a connection this will trigger an Error
|
|
|
|
func (rc *Connection) Close() {
|
|
|
|
// Kill the Ricochet Connection.
|
|
|
|
log.Debugf("Closing Ricochet Connection for %v", rc.RemoteHostname)
|
|
|
|
rc.conn.Close()
|
2019-11-08 00:11:14 +00:00
|
|
|
rc.closingLock.Lock()
|
2019-01-11 20:28:50 +00:00
|
|
|
rc.closed = true
|
2019-11-08 00:11:14 +00:00
|
|
|
rc.closingLock.Unlock()
|
2019-01-11 20:28:50 +00:00
|
|
|
}
|