Merge branch 'trunk' into read-through

This commit is contained in:
Sarah Jamie Lewis 2021-08-26 19:19:02 +00:00
commit 1a7a6fe818
24 changed files with 196 additions and 42 deletions

View File

@ -7,4 +7,4 @@ title = "Cwtch Secure Development Handbook"
[output.html]
mathjax-support = true
default-theme = "navy"
default-theme = "ayu"

BIN
src/BASE_0.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

BIN
src/BASE_1.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

BIN
src/BASE_2.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

BIN
src/BASE_3.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

BIN
src/BASE_5.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.8 KiB

BIN
src/BASE_6.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.5 KiB

BIN
src/BASE_7.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

BIN
src/BASE_8.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

View File

@ -1,14 +1,16 @@
# Summary
- [Overview](./overview.md)
- [Overview and History](./overview.md)
- [Risk Model](./risk.md)
- [Cwtch Technical Basics](./cwtch-overview.md)
- [Open Questions](./open-questions.md)
- [Connectivity](./connectivity.md)
- [Tapir](./tapir.md)
- [Packet Format](./packet_format.md)md
- [Authentication Protocol](./authentication_protocol.md)
- [Cwtch Library](./cwtch.md)
- [Message Formats](./message_formats.md)
- [Groups](./groups.md)
- [Tapir](./tapir.md)
- [Packet Format](./packet_format.md)
- [Authentication Protocol](./authentication_protocol.md)
- [Connectivity](./connectivity.md)
- [Cwtch UI](./ui.md)
- [Profile Encryption & Storage](./profile_encryption_and_storage.md)
- [Android Service](./android.md)

View File

@ -1,3 +1,28 @@
# Android Service
(Currently under active development, will be documented here once it has been merged into trunk)
[Adapted from: Discreet Log #11: Integrating FFI processes with Android services](https://openprivacy.ca/discreet-log/11-android-ffi-service-integration/)
In addition to needing to make plain ol method calls into the Cwtch library, we also need to be able to communicate with (and receive events from) long-running Cwtch goroutines that keep the Tor process running in the background, manage connection and conversation state for all your contacts, and handle a few other monitoring and upkeep tasks as well. This isnt really a problem on traditionally multitasking desktop operating systems, but on mobile devices running Android we have to contend with shorter sessions, frequent unloads, and network and power restrictions that can vary over time. As Cwtch is intended to be metadata resistant and privacy-centric, we also want to provide notifications without using the Google push notification service.
The solution for long-running network apps like Cwtch is to put our FFI code into an Android Foreground Service. (And no, its not lost on me that the code for our backend is placed in something called a ForegroundService.) With a big of finagling, the WorkManager API allows us to create and manage various types of services including ForegroundServices. This turned out to be a great choice for us, as our gomobile FFI handler happened to already be written in Kotlin, and WorkManager allows us to specify a Kotlin coroutine to be invoked as the service.
If youd like to follow along, our WorkManager specifications are created in the handleCwtch() method of [MainActivity.kt](https://git.openprivacy.ca/cwtch.im/cwtch-ui/src/branch/trunk/android/app/src/main/kotlin/im/cwtch/flwtch/MainActivity.kt), and the workers themselves are defined in [FlwtchWorker.kt](https://git.openprivacy.ca/cwtch.im/cwtch-ui/src/branch/trunk/android/app/src/main/kotlin/im/cwtch/flwtch/FlwtchWorker.kt).
Our plain ol method calls to FFI routines are also upgraded to be made as WorkManager work requests, which allows us to conveniently pass the return values back via the result callback.
One initial call (aptly named Start) gets hijacked by FlwtchWorker to become our eventbus loop. Since FlwtchWorker is a coroutine, its easy for it to yield and resume as necessary while waiting for events to be generated. Cwtchs goroutines can then emit events, which will be picked up by FlwtchWorker and dispatched appropriately.
FlwtchWorkers eventbus loop is not just a boring forwarder. It needs to check for certain message types that affect the Android state; for example, new message events should typically display notifications that the user can click to go to the appropriate conversation window, even when the app isnt running in the foreground. When the time does come to forward the event to the app, we use LocalBroadcastManager to get the notification to MainActivity.onIntent. From there, we in turn use Flutter MethodChannels to forward the event data from Kotlin into the frontends Flutter engine, where the event finally gets parsed by Dart code that updates the UI as necessary.
Messages and other permanent state are stored on disk by the service, so the frontend doesnt need to be updated if the app isnt open. However, some things (like dates and unread messages) can then lead to desyncs between the front and back ends, so we check for this at app launch/resume to see if we need to reinitialize Cwtch and/or resync the UI state.
Finally, while implementing these services on Android we observed that WorkManager is very good at persisting old enqueued work, to the point that old workers were even being resumed after app reinstalls! Adding calls to pruneWork() helps mitigate this, as long as the app was shut down gracefully and old jobs were properly canceled. This frequently isnt the case on Android, however, so as an additional mitigation we found it useful to tag the work with the native library directory name:
private fun getNativeLibDir(): String {
val ainfo = this.applicationContext.packageManager.getApplicationInfo(
"im.cwtch.flwtch", // Must be app name
PackageManager.GET_SHARED_LIBRARY_FILES)
return ainfo.nativeLibraryDir
}
…then, whenever the app is launched, we cancel any jobs that arent tagged with the correct current library directory. Since this directory name changes between app installs, this technique prevents us from accidentally resuming with an outdated service worker.

BIN
src/card_header.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

BIN
src/cwtch phones.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

61
src/cwtch-overview.md Normal file
View File

@ -0,0 +1,61 @@
# Cwtch Technical Basics
This page presents a brief technical overview of the Cwtch protocol.
## A Cwtch Profile
Users can create one of more Cwtch Profiles. Each profile generates a random ed25519 keypair compatible with
Tor.
In addition to the cryptographic material, a profile also contains a list of Contacts (other Cwtch profile public keys +
associated data about that profile like nickname and (optionally) historical messages), a list of Groups (containing the group cryptographic material in addition to other associated data like the group nickname and historical messages).
## 2-party conversions: Peer to Peer
![](./BASE_3.png)
For 2 parties to engage in a peer-to-peer conversation both must be online, but only one needs to be reachable via
their onion service. For the sake of clarity we often label one party the "inbound peer" (the one who hosts the onion service) and the other party the
"outbound peer" (the one that connects to the onion service).
After connection both parties engage in an authentication protocol which:
* Asserts that each party has access to the private key associated with their public identity.
* Generates an ephemeral session key used to encrypt all further communication during the session.
This exchange (documented in further detail in [authentication protocol](./authentication_protocol.md)) is *offline deniable*
i.e. it is possible for any party to forge transcripts of this protocol exchange after the fact, and as such - after the
fact - it is impossible to definitely prove that the exchange happened at all.
After, the authentication protocol the two parties may exchange messages with each other freely.
## Multi-party conversations: Groups and Peer to Server Communication
**Note: Metadata Resistant Group Communication is still an active research area and what is documented here
will likely change in the future.**
When a person wants to start a group conversation they first randomly generate a secret `Group Key`. All group communication will be encrypted using this key.
Along with the `Group Key`, the group creator also decides on a **Cwtch Server** to use as the host of the group.
For more information on how Servers authenticate themselves see [key bundles](./key_bundles.md).
A `Group Identifier` is generated using the group key and the group server and these three elements are packaged up
into an invite that can be sent to potential group members (e.g. over existing peer-to-peer connections).
To send a message to the group, a profile connects to the server hosting the group (see below), and encrypts
their message using the `Group Key` and generates a cryptographic signature over the `Group Id`, `Group Server`
and the decrypted message (see: [wire formats](./message_formats.md) for more information).
To receive message from the group, a profile connected to the server hosting the group and downloads *all* messages (since
their previous connection). Profiles then attempt to decrypt each message using the `Group Key` and if successful attempt
to verify the signature (see [Cwtch Servers](./server.md) [Cwtch Groups](./groups.md) for an overview of attacks and mitigations).
### Servers are Peers
In many respects communication with a server is identical to communication with a regular Cwtch peer,
all the same steps above are taken however the server always acts as the inbound peer, and the outbound
peer always uses newly generated **ephemeral keypair** as their "longterm identity".
As such peer-server conversations only differ in the *kinds* of messages that are sent between the two parties,
with the server relaying all messages that it receives and also allowing any client to query for older messages.

View File

@ -37,8 +37,7 @@ version of the protobufs framework.
We have removed protobufs entirely from the project by porting this
functionality over the Tapir.
<!-- PoW as in Proof of Work? -->
### PoW Spam Prevention as a Metadata Vector
### Proof of Work (PoW) Spam Prevention as a Metadata Vector
**Status: Outdated**: Cwtch now uses Token Based Services to separate challenges like PoW from
resolving the tokens.

View File

@ -58,5 +58,3 @@ chain (which if they were attempting to censor other messages would
Finally: We are actively working on adding non-repudiation to Cwtch servers such
that they themselves are restricted in what they can censor efficiently.
<!-- I might have missed it but I think this is the first mention of each message being signed by the sender (and that the message contains unique identifiers)? Might be great to have a 'message_format.md' where message structure/constructure is explained/introduced earlier? I also can't tell how the session key is applied to the message at this point. -->

View File

@ -3,7 +3,7 @@
## Risk: Interception of Cwtch content or metadata through an IME on Mobile Devices
**Status: Unmitigated**
**Status: Partially Mitigated**
Any component that has the potential to intercept data between a person, and the Cwtch app is a
potential security risk.
@ -19,4 +19,7 @@ ecosystem.
A similar risk exists on desktop through the use of similar input applications (in addition to software keyloggers),
however we consider that fully outside the scope of Cwtch risk assessment (in line with other attacks on the security of the underlying
operating system itself).
operating system itself).
This is partially mitigated in Cwtch 1.2 through the use of `enableIMEPersonalizedLearning: false`. See
[this PR](https://git.openprivacy.ca/cwtch.im/cwtch-ui/pulls/142) for more information.

57
src/message_formats.md Normal file
View File

@ -0,0 +1,57 @@
# Message Formats
## Peer to Peer Messages
PeerMessage {
ID string // A unique Message ID (primarily used for acknowledgments)
Context string // A unique context identifier i.e. im.cwtch.chat
Data []byte // The context-dependent serialized data packet.
}
### Context Identifiers
* `im.cwtch.raw` - Data contains a plain text chat message (see: [overlays](./overlays.md) for more information)
* `im.cwtch.acknowledgement` - Data is empty and ID references a previously sent message
* `im.cwtch.getVal` and `im.cwtch.retVal` - Used for requesting / returning specific information about a peer. Data
contains a serialized `peerGetVal` structure and `peerRetVal` respectively.
peerGetVal struct {
Scope string
Path string
}
type peerRetVal struct {
Val string // Serialized path-dependent value
Exists bool
}
## Plaintext / Decrypted Group Messages
type DecryptedGroupMessage struct {
Text string // plaintext of the message
Onion string // The cwtch address of the sender
Timestamp uint64 // A user specified timestamp
// NOTE: SignedGroupID is now a misnomer, the only way this is signed is indirectly via the signed encrypted group messages
// We now treat GroupID as binding to a server/key rather than an "owner" - additional validation logic (to e.g.
// respect particular group constitutions) can be built on top of group messages, but the underlying groups are
// now agnostic to those models.
SignedGroupID []byte
PreviousMessageSig []byte // A reference to a previous message
Padding []byte // random bytes of length = 1800 - len(Text)
}
DecryptedGroupMessage contains random padding to a fixed size that is equal to the length of all fixed length fields + 1800.
This ensures that all encrypted group messages are equal length.
## Encrypted Group Messages
// EncryptedGroupMessage provides an encapsulation of the encrypted group message stored on the server
type EncryptedGroupMessage struct {
Ciphertext []byte
Signature []byte // Sign(groupID + group.GroupServer + base64(decrypted group message)) using the senders cwtch key
}
Calculating the signature requires knowing the groupID of the message, the server the group is associated with
and the decrypted group message (and thus, the Group Key). It is (ed25519) signed by the sender of the message, and can be
verified using their public cwtch address key.

View File

@ -1,7 +1,10 @@
# Message Overlays
<!-- Oh, great. Can this be earlier in the handbook? Like section 2. early? -->
[Adapted from: Discreet Log #8: Notes on the Cwtch Chat API](https://openprivacy.ca/discreet-log/08-chatapi/)
**Note: This section covers overlay protocols on-top of the Cwtch protcol. For information on the Cwtch Protocol
messages themselves please see [Message Formats](./message_formats.md)**
We envision Cwtch as a platform for providing an authenticated transport layer to higher-level applications.
Developers are free to make their own choices about what application layer protocols to use,
whether they want bespoke binary message formats or just want to throw an HTTP library on top and call it a
@ -65,7 +68,6 @@ The most simple over is a chat message which simply contains raw, unprocessed ch
```
{o:1,d:"got milk?"}
```
<!-- I was told there would unique identifiers and signatures? -->
## Invitations (Overlays 100 and 101)
@ -80,4 +82,6 @@ The wire format for these are equally simple:
This represents a departure from our original "overlays" thinking to a more action-oriented representation. The chat "overlay" can communicate that someone *did* something, even if it's paraphrased down to "added an item to a list," and the lists and bulletins and other beautifully chaotic data can have their state precomputed and stored separately.
<!-- What are lists or bulletin boards? -->
## Lists / Bulletin Boards
**Note: Expected to be Defined in Cwtch Beta 1.5**

View File

@ -9,6 +9,20 @@ processes.
![](https://docs.openprivacy.ca/cwtch-security-handbook/2.png)
## What is Cwtch?
Cwtch (/kʊtʃ/ - a Welsh word roughly translating to “a hug that creates a safe place”) is a decentralized, privacy-preserving, multi-party messaging protocol that can be used to build metadata resistant applications.
* **Decentralized and Open**: There is no “Cwtch service” or “Cwtch network”. Participants in Cwtch can host their own safe spaces, or lend their infrastructure to others seeking a safe space. The Cwtch protocol is open, and anyone is free to build bots, services and user interfaces and integrate and interact with Cwtch.
* **Privacy Preserving**: All communication in Cwtch is end-to-end encrypted and takes place over Tor v3 onion services.
* **Metadata Resistant**: Cwtch has been designed such that no information is exchanged or available to anyone without their explicit consent, including on-the-wire messages and protocol metadata.
### A Video Explainer
<video width="99%" controls>
<source src="https://cwtch.im/cwtch-explainer.mp4" type="video/mp4">
</video>
## A (Brief) History of Metadata Resistant Chat
In recent years, public awareness of the need and benefits of end-to-end

View File

@ -10,8 +10,6 @@ it isn't possible to provide a full list of profiles a user might have access to
To handle profiles that are "unencrypted" (i.e don't require a password to open) we currently create a profile
with a [defacto, hardcoded password](https://git.openprivacy.ca/cwtch.im/libcwtch-go/src/branch/trunk/constants/globals.go#L5).
<!-- Can cwtch be updated to randomly generate a password and store it on the devices keychain/keystore (for devices/OS with a keychain API)? Also I'm not sure how secure OS level keychains are but hard coded passwords always become campfire stories years later. -->
This isn't ideal, we would much rather wish to rely on OS-provided key material such that the profile is bound to a
specific device, but such features are currently patchwork - we also note by creating an unencrypted profile, people
who use Cwtch are explicitly opting into the risk that someone with access to the file system may be able to decrypt

View File

@ -1,5 +1,5 @@
# Cwtch Server
<!-- Can this be moved up to be section 3. of the handbook? -->
The goal of the Cwtch protocol is to enable group communication through
**Untrusted Infrastructure**.

View File

@ -98,27 +98,6 @@ we expect it to be protected via a preceeding app in an `ApplicationChain` e.g.
* No direct testing (tested via integration tests and unit tests)
### Ephemeral Connections
Occasionally it is desirable to have a peer connect to another / a service
without using their long term identity (e.g. in the case of connecting to
a Cwtch Server).
In this case we want to enable a convenient way to allow connecting with an
ephemeral identity.
<!-- This is confusing "avoid side channel around avoid
duplicate connections" -->
It turns out that doing this securely requires maintaining a completely separate
set of connections and applications in order to avoid side channel around avoid
duplicate connections (i.e. if we did mix them up then a service might be able
to exploit the fact that clients avoid duplicate connections by attempting to
connect to known-online peers and observing if they reject the connection
because they already have an outbound ephemeral connection open.)
Because of this, we don't provide an explicit Ephemeral Connect api and instead
recommend that peers maintain one long term service and multiple ephemeral
services.
## Known Risks
@ -172,6 +151,22 @@ handling of such instances (such as returning an error when they are found
allowing a handling application to retry the request if a connection with a
given capability isn't returned)
### Ephemeral Connections
Occasionally it is desirable to have a peer connect to a service
without using their long term identity (e.g. in the case of connecting to
a Cwtch Server).
In this case we want to enable a convenient way to allow connecting with an
**ephemeral identity**.
It turns out that doing this securely requires maintaining a completely separate
set of connections and applications in order to avoid side channels caused by duplicate connections handling.
As such the Cwtch Protocol Engine maintains two disctinct connection pools, one for avowed connections and another
for ephemeral connections. All connections to known Cwtch Servers are made through the ephemeral pool.
## Testing Status
Tapir features a number of well-defined integration tests which exercise not

View File

@ -58,8 +58,6 @@ In order to be suitable to deploy groups at a wide scale, the app requires a way
to prevent Cwtch from fetching information over such connections, and this
should likely be turned on by default.
<!-- Can messages from peers/servers be received/sent in a batch? -->
## Testing Status